The most underappreciated AI application in legal today: Retrieval‑Augmented Generation (RAG)

RAG is an AI framework that dramatically improves large language models’ accuracy by retrieving facts from trusted knowledge sources—known deal documents, internal playbooks, and proprietary databases—before generating an answer. Compared with other AI approaches, RAG delivers two primary benefits: (1) hallucinations are dramatically reduced, and (2) platforms such as Harvey AI implement guardrails that prevent disclosing sensitive information to third parties or using that information for model training.

I began to learn more about this technology after several conversations with some of the Madison community’s top entrepreneurs and AI experts—including Aaryush Gupta, Ronak Bhale, Scotty Cadet, Yash Arvind, Rehatibir Singh, Siddharth Singh, Slava Iudenko, Jenil Makwana, Dervis Gursoy, and Payton Smith. Their insights into RAG and their use cases have been invaluable and have helped me apply the technology to my practice.

In my experience, law firms may be particularly well positioned to benefit from RAG, given the depth of their curated deal knowledge and internal precedents. Decades of curated, precedential deal knowledge create a proprietary “data moat” that strengthens over time. From my own experience, I see dramatic improvements in the following:

(1) Budgets: The cost and predictability of services will continue to improve, leaving fewer clients burned by out-of-control outside legal costs.

(2) Time: Faster deal closings, better initial drafts, and improved accuracy and speed in due diligence review.

(3) Outcomes: Final drafts are more precise and error-free, business objectives are not delayed by legal, and complex deals are easier for all parties to understand.