Challenges and Lessons Learned in Building with Generative AI

Summary
Generative AI has rapidly gained adoption for numerous use cases. To support those applications, organizational data platforms need to add new features and data teams have increased responsibility. In this episode Lior Gavish, co-founder of Monte Carlo, discusses the various ways that data teams are evolving to support AI powered features and how they are incorporating AI into their work.
Announcements

• Hello and welcome to the Data Engineering Podcast, the show about modern data management
• Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. Trusted by teams of all sizes, including Comcast and Doordash. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino.
• Your host is Tobias Macey and today I'm interviewing Lior Gavish about the impact of AI on data engineers

Interview

• Introduction
• How did you get involved in the area of data management?
• Can you start by clarifying what we are discussing when we say "AI"?
• Previous generations of machine learning (e.g. deep learning, reinforcement learning, etc.) required new features in the data platform. What new demands is the current generation of AI introducing?
• Generative AI also has the potential to be incorporated in the creation/execution of data pipelines. What are the risk/reward tradeoffs that you have seen in practice?
  • What are the areas where LLMs have proven useful/effective in data engineering?
• Vector embeddings have rapidly become a ubiquitous data format as a result of the growth in retrieval augmented generation (RAG) for AI applications. What are the end-to-end operational requirements to support this use case effectively?
  • As with all data, the reliability and quality of the vectors will impact the viability of the AI application. What are the different failure modes/quality metrics/error conditions that they are subject to?
• As much as vectors, vector databases, RAG, etc. seem exotic and new, it is all ultimately shades of the same work that we have been doing for years. What are the areas of overlap in the work required for running the current generation of AI, and what are the areas where it diverges?
  • What new skills do data teams need to acquire to be effective in supporting AI applications?
• What are the most interesting, innovative, or unexpected ways that you have seen AI impact data engineering teams?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working with the current generation of AI?
• When is AI the wrong choice?
• What are your predictions for the future impact of AI on data engineering teams?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com with your 

Links

• Monte Carlo
  • Podcast Episode
• NLP == Natural Language Processing
• Large Language Models
• Generative AI
• MLOps
• ML Engineer
• Feature Store
• Retrieval Augmented Generation (RAG)
• Langchain

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA