Strategies For Building A Product Using LLMs At DataChat

Mar 3, 2024

Jignesh Patel discusses the challenges of building a product using Large Language Models, the business and technical difficulties, and strategies for gaining visibility into the inner workings of LLMs while maintaining control and privacy of data. The episode explores the trade-offs in prompt engineering for AI model context building, potential applications of LLMs in information distillation, and the importance of balancing AI regulation and openness for innovation.

Ask episode

Chapters

Transcript

Episode notes

Introduction

00:00 • 4min

Navigating Challenges and Innovations in Data Analysis with LLMs

03:38 • 15min

Balancing Context Building in AI Model Development

18:54 • 3min

Exploring the Potential of Language Models in Information Distillation

22:01 • 21min

Balancing AI Regulation and Openness for Innovation

43:07 • 2min

Importance of Regulations and Data Quality in Machine Learning Adoption

44:48 • 4min

Summary
Large Language Models (LLMs) have rapidly captured the attention of the world with their impressive capabilities. Unfortunately, they are often unpredictable and unreliable. This makes building a product based on their capabilities a unique challenge. Jignesh Patel is building DataChat to bring the capabilities of LLMs to organizational analytics, allowing anyone to have conversations with their business data. In this episode he shares the methods that he is using to build a product on top of this constantly shifting set of technologies.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Your host is Tobias Macey and today I'm interviewing Jignesh Patel about working with LLMs; understanding how they work and how to build your own

Interview

Introduction
How did you get involved in machine learning?
Can you start by sharing some of the ways that you are working with LLMs currently?
What are the business challenges involved in building a product on top of an LLM model that you don't own or control?
- In the current age of business, your data is often your strategic advantage. How do you avoid losing control of, or leaking that data while interfacing with a hosted LLM API?
What are the technical difficulties related to using an LLM as a core element of a product when they are largely a black box?
- What are some strategies for gaining visibility into the inner workings or decision making rules for these models?
What are the factors, whether technical or organizational, that might motivate you to build your own LLM for a business or product?
- Can you unpack what it means to "build your own" when it comes to an LLM?
In your work at DataChat, how has the progression of sophistication in LLM technology impacted your own product strategy?
What are the most interesting, innovative, or unexpected ways that you have seen LLMs/DataChat used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working with LLMs?
When is an LLM the wrong choice?
What do you have planned for the future of DataChat?

Contact Info

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0