Brian Kennedy, Senior Practice Innovation Manager of Real Estate at Bryan Cave Leighton Paisner (BCLP), talks about Large Language Models (LLMs) and the issues they face including hallucinations. He discusses the legal tech products using LLMs, how law firms can spend their Generative AI dollars, and shares insights into how BCLP uses Generative AI.
Accuracy and potential hallucinations are issues in Generative AI for legal teams, but there are ways to mitigate these problems.
The legal tech market for Generative AI products offers both general multipurpose assistance and specialist products for law firms to choose from.
Deep dives
Basics of Generator AI for Legal Teams
Generator AI for legal teams involves using large language models to process natural language and generate human-sounding responses. These models are trained on massive datasets and use a transformer architecture to predict the words that should follow in a given context. Accuracy is a concern since the models are trained on information scraped from the web, which can be unreliable and incomplete. Additionally, the models can sometimes generate content that didn't actually exist, leading to potential hallucinations. However, there are ways to mitigate these issues, such as fine-tuning the models, using prompt engineering to guide their responses, and implementing guardrails that limit the sources and databases they can cite.
Accuracy Challenges and Concerns
The accuracy of generator AI for legal teams is influenced by the data used for pre-training the models, which includes potentially erroneous or incomplete information from the web. Legal data, such as know-how and precedents, may not be freely available online, leading to potential accuracy risks. The lack of transparency regarding the information used to generate an output adds to the concern. However, there are efforts to improve accuracy and address these challenges, such as creating legal tech tools based on curated legal content and ensuring the tools allow users to verify and cross-reference the generated output.
Legal Tech Landscape and Investment Approach
The legal tech market for generator AI products can be categorized into general multipurpose assistance and specialist products. The former provides a broad range of functionality, while the latter focuses on specific problems or functions. Law firms can choose between off-the-shelf products or developing their own bespoke solutions. The approach to investment in generator AI should consider finding a balance between broad applicability and specific use case optimization. Law firms will need to assess the accuracy levels required for their specific tasks and determine the best combination of tools and human review. Early investments in this evolving field are important, and collaboration with vendors and trial access to different tools can help firms determine the best approach.
In this episode, I chat with Brian Kennedy, Senior Practice Innovation Manager of Real Estate at Bryan Cave Leighton Paisner (BCLP), about Generative AI. We discuss what Large Language Models (LLMs) are, some issues plaguing LLMs including hallucinations, the types of legal tech products using LLMs, and how law firms can look at spending their Generative AI dollars with a final look at how BCLP uses Generative AI.
The points we discuss in this episode along with the corresponding time stamps are set out below:
1. Introduction (0:09) 2. What is a Large Language Model (LLM)? (02:02) 3. What are some accuracy issues LLMs face, and why? (3:25) 4. What are hallucinations (05:11) 5. How can hallucinations and accuracy problems be mitigated? (08:46) 6. How much can we trust Generative AI tools? (11:31) 7. What is the overview of the Legal Tech landscape for Generative AI products? (16:40) 8. What is the best approach for a law firm in spending their Generative AI dollars? (19:51) 9. What approach is your firm Bryan Cave Leighton Paisner (BCLP) taking to using Generative AI? (21:33)
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode