At the Legal AI and Innovation Summit hosted by Inside Practice on November 21, 2024, three leaders in the intersection of law and technology took the stage to discuss “The Future (and current) State of Search and AI’s Impacts on Knowledge Work More Broadly”. This 4-part blog series recaps their insights and predictions for how search will continue to shape the legal profession.
The panel featured Oz Benamram, Chief Knowledge and Innovation Officer at Simpson Thacher & Bartlett LLP; Yannic Kilcher, CTO and Cofounder of DeepJudge; and moderator Ilona Logvinova, Director of Practice Innovation at Cleary Gottlieb Steen & Hamilton LLP. Together, they explored the rise of contextual and interactive search, how AI-powered tools are redefining document relevance and retrieval, and how these advancements are reshaping the practice of law.
The panel began with a quick history of search in legal practice, presented by Oz Benamram. Reflecting on his early experiences in the legal field, Benamram described how traditional document search systems failed to address the specific needs of law firms.
Going back to the 1990s, when it came to search, “the world was flat.” Systems like Lexis, WestLaw, Lycos, Yahoo were one-dimensional and only focused on document text; they were neither able to incorporate semantics nor metadata about the document. Then, in the e-commerce world, a second dimension emerged - people. E-commerce search evolved to incorporate both text and purchasers’ reviews and therefore recommendations. Some vendors began to bring these capabilities to law.
Benamram shared an early presentation he created to explain the concept of contextual search to his firm. He had articulated that law firm enterprise search needed to have a third dimension. In addition to the “what” (text of emails and documents) and the “who” (people and entities), there was a third, critically important element that needed to be incorporated: the “why” (clients and matters). When you combine these dimensions together, lawyers can know “how” to do things, Benamram explained. The approach focused on linking knowledge sources—such as documents, email, HR records, and client information—into a unified system.
Benamram recounted the challenges of building custom enterprise search solutions in the early 2000s. After several false starts with a vendor that was not focused on the legal sector, he eventually entered into a partnership with Recommind, a pioneering legal search vendor. “We worked together as a community” to build the product and learn from each other.
In this way Benamram began a long journey toward solving search for law firms, which the panel acknowledged is critical. Moderator Ilona Logvinova referred to search as “the foundational problem statement of our industry.”
Yannic Kilcher, CTO and Cofounder of DeepJudge, covered the technological advancements of the recent decade.
Kilcher described the paradigm shift brought about by neural networks and large language models (LLMs), such as BERT and GPT, which enabled machines to learn contextually from raw data. Previously, these systems required feature engineering. But now, technology such as “neural networks can make sense out of data in a much more human way,” Kilcher explained. These additional tools in the engineers' toolkits do not solve the search problem all on their own. They need to be combined with many other techniques to get good results.
Kilcher noted that his and his co-founders’ backgrounds in machine learning research, including time spent at Google during the development of the Transformer model, inspired DeepJudge to focus their product on contextual and interactive search tailored for legal professionals.
Kilcher cautioned against over-reliance on buzzwords like “AI search” and “vector databases,” which can oversimplify complex problems.
“Just adding an LLM or embedding a corpus in a vector database doesn’t solve the problem. It’s not solving all of your problems just because you add one of those things. It’s always been about implementing information retrieval well—using both modern techniques and still covering the basics like word frequencies and recency,” Kilcher explained.
Advancements like BERT have significantly improved search by enabling semantic understanding of a document’s content, rather than relying solely on keywords or titles. However, pure vector-based search has its limitations. The real innovation comes from combining multiple sources, enriching them with metadata (i.e. adding data about the data), and creating a multidimensional, comprehensive view. At its core, much of this still builds on the foundational principles of Information Retrieval (IR).
At around the 15-minute mark, the panel shifted to discussing the current challenges in legal search. Kilcher highlighted the inefficiencies of traditional legal search systems.
Search is challenging because it often feels like magic—you ask a question and expect the perfect answer or result. Expectations are especially high in enterprises, where information is scattered across many systems. Locating the right information is hard enough, but making sense of it—understanding what’s relevant, outdated, or just a draft—is even harder.
“It’s crazy. We hear all the time that people at law firms use Google to search the entire internet for something they know exists in their own data – simply because their internal systems can’t deliver,” Kilcher remarked.
Moderator Ilona Logvinova added that search must go beyond simply retrieving cases or documents. Law firm knowledge is not just words on a page, but valuable data that firms need to source, review, analyze and benchmark in a timely way, and that data is often distributed among systems and siloed.
Stay tuned for Part 2, coming next week!
