UnCommon Law 6. From Errors to Efficiency: Can AI Transform the Practice of Law?
Oct 31, 2024
Isabel Gottlieb, a Bloomberg Law reporter with a focus on AI in corporate legal departments, joins legal powerhouse John Quinn, founder of a leading law firm, and Stanford law professor Daniel Ho. They tackle the transformative potential of AI in legal practice, discussing both efficiency gains and significant accuracy concerns. The guests illuminate challenges like 'hallucinations' in AI responses and explore real-world implications, such as contract interpretation post-Hurricane Katrina. Together, they envision a future where human oversight and AI coalesce in law.
AI Snips
Chapters
Transcript
Episode notes
AI Hallucinates on Justice Scalia
- Google Gemini gave a hallucinated answer about Justice Scalia's opinion on the Chevron Doctrine despite his passing in 2016.
- This illustrates how AI chatbots can create plausible but false legal information, demanding cautious use.
High AI Legal Error Rates
- Large general AI models showed a 58% to 88% error rate answering legal questions.
- This shows that basic AI legal tools frequently generate incorrect information.
AI’s Contrafactual Bias Risks
- AI models tend to accept mistaken premises and provide answers without challenging them.
- This 'model sycophancy' or contrafactual bias increases the risk of AI giving misleading legal information.