Balancing Context Building in AI Model Development

Summary
Large Language Models (LLMs) have rapidly captured the attention of the world with their impressive capabilities. Unfortunately, they are often unpredictable and unreliable. This makes building a product based on their capabilities a unique challenge. Jignesh Patel is building DataChat to bring the capabilities of LLMs to organizational analytics, allowing anyone to have conversations with their business data. In this episode he shares the methods that he is using to build a product on top of this constantly shifting set of technologies.
Announcements

• Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
• Your host is Tobias Macey and today I'm interviewing Jignesh Patel about working with LLMs; understanding how they work and how to build your own

Interview

• Introduction
• How did you get involved in machine learning?
• Can you start by sharing some of the ways that you are working with LLMs currently?
• What are the business challenges involved in building a product on top of an LLM model that you don't own or control? 
  • In the current age of business, your data is often your strategic advantage. How do you avoid losing control of, or leaking that data while interfacing with a hosted LLM API?
• What are the technical difficulties related to using an LLM as a core element of a product when they are largely a black box? 
  • What are some strategies for gaining visibility into the inner workings or decision making rules for these models?
• What are the factors, whether technical or organizational, that might motivate you to build your own LLM for a business or product? 
  • Can you unpack what it means to "build your own" when it comes to an LLM?
• In your work at DataChat, how has the progression of sophistication in LLM technology impacted your own product strategy?
• What are the most interesting, innovative, or unexpected ways that you have seen LLMs/DataChat used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working with LLMs?
• When is an LLM the wrong choice?
• What do you have planned for the future of DataChat?

Contact Info

• Website
• LinkedIn

Parting Question

• From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• DataChat
• CMU == Carnegie Mellon University
• SVM == Support Vector Machine
• Generative AI
• Genomics
• Proteomics
• Parquet
• OpenAI Codex
• LLama
• Mistral
• Google Vertex
• Langchain
• Retrieval Augmented Generation
• Prompt Engineering
• Ensemble Learning
• XGBoost
• Catboost
• Linear Regression
• COGS == Cost Of Goods Sold
• Bruce Schneier - AI And Trust

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0