Legal Innovation Spotlight

51 - Bridging the Gap Between AI Promises and Legal Practice Realities with Jack Shepherd

Jan 15, 2025

Jack Shepherd, a former lawyer and legal tech expert at iManage, discusses the gaps between AI promises and legal practice realities. He highlights the challenges of AI implementation, stressing the need for clear understanding and caution when adopting new technologies. Jack reveals the pitfalls of over-hyped tools and emphasizes the importance of basic data hygiene and targeted applications. He also talks about how generative AI often fails in critical legal tasks, advocating for pragmatic approaches to improve workflows and encourage knowledge-sharing among lawyers.

52:35

Episode guests

Jack Shepherd

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Generative AI has limitations in legal contexts due to comprehension issues and hallucinations, requiring cautious application in high-stakes scenarios.

Law firms should prioritize smaller, targeted enterprise search use cases to achieve better integration and tangible benefits despite previous failures.

Deep dives

The Role of Knowledge Management in Legal Tech

Knowledge consulting plays a crucial role in enhancing the efficiency of legal practices. By focusing on cleaning up data and improving knowledge sharing within law firms, professionals can ensure that high-quality examples are readily available for reference. This approach includes developing taxonomies and metadata schemas, as well as addressing cultural barriers to knowledge management. Ultimately, the goal is to streamline processes so that lawyers can work more effectively and focus on their core responsibilities.

Intro

2min

The Realities of AI in Legal Practice

20min

Exploring Insight Plus: A Tailored Approach to Knowledge Management in Legal Practices

2min

Navigating Document Management Challenges

8min

Navigating AI in Legal Practice

10min

Navigating Legal Tech Investment

6min

Navigating AI Adoption Challenges in Legal Practice

3min

In this episode, Ted sits down with Jack Shepherd to dissect the practical realities of implementing AI and knowledge management tools in the legal industry. Together, they challenge conventional assumptions, explore the pitfalls of over-hyped technologies, and highlight the understated value of focusing on basic, everyday challenges. With candid insights into AI adoption, enterprise search challenges, and the importance of clean data, this conversation is a must-listen for anyone seeking actionable strategies to tackle inefficiencies in legal workflows and knowledge management.

In this episode, Jack shares insights on how to:

Evaluate AI tools effectively in the legal industry
Address data hygiene challenges in law firm document management systems
Integrate generative AI into law firm workflows without over-promising results
Balance risk and reward when adopting enterprise search solutions in legal contexts
Encourage lawyer engagement with knowledge-sharing initiatives

Key takeaways:

Generative AI tools like LLMs are limited by hallucinations and a lack of comprehension, often failing in critical legal tasks like accurate citations or respecting legal hierarchies, underscoring the need for cautious adoption in high-risk contexts.
Many law firms invest heavily in enterprise search projects only to see them fail due to unclear ROI and poor integration, highlighting the importance of focusing on smaller, targeted use cases with tangible benefits like better document organization or locating key client information.
Effective use of AI in law requires clean, structured data, as experiments show poor data hygiene leads to irrelevant or misleading outputs, undermining both efficiency and trust.
The legal industry's focus on flashy, complex AI use cases often overlooks basic workflow inefficiencies—like time tracking, document versioning, or knowledge-sharing—which would yield quicker, more impactful results for lawyers.

About the guest, Jack Shepherd:

Jack Shepherd is a former lawyer and legal technology expert, with a particular interest in cutting through hype and confusion in technology to deliver value to practising lawyers and clients. He works at iManage, where he leads consulting initiatives on knowledge management projects.

“[The] accuracy thing is such a big deal around LLMs because the lawyer using a tool—they'll ask it a question, and I can guarantee you the first question they'll ask will be to test it. Then, if it gets even slightly wrong: 'No, I don't trust this. Never using it again.' You've got to be really careful about that.” – Jack Shepherd

Connect with Jack Shepherd:

Website: https://imanage.com/

LinkedIn: https://www.linkedin.com/in/jackwshepherd/

Connect with Ted Theodoropoulos:

Website: https://getinfodash.com/

LinkedIn: https://www.linkedin.com/in/tedtheo

YouTube: https://www.youtube.com/@getinfodash/podcasts