AI Engineering Podcast

Viraj Mehta, CTO and co-founder of TensorZero, shares insights on optimizing AI applications with their innovative LLM gateways. He discusses how these gateways standardize communication and manage interactions between applications and AI models. The conversation dives into sustainable AI optimization and the challenges of integrating structured data inputs. Viraj also highlights the role of user feedback in enhancing AI interactions, as well as the architectural innovations that improve efficiency and usability for developers.

Ron Green, co-founder and CTO of Kung Fu AI, dives into the evolving AI landscape and the complexities of generative AI engines. He discusses the limitations of large language models and the critical need for human oversight and robust data management. Ron highlights innovative methods like Retrieval-Augmented Generation and the significance of targeted, domain-specific AI solutions. He expresses optimism for AI's future, predicting major advancements in the next 20 years that integrate seamlessly into everyday applications.

Jim Olson, CTO of ModelOp, specializes in generative AI governance and regulations. He discusses the importance of monitoring and inventory for compliance in high-risk areas like healthcare. Olson emphasizes the need for technical controls to manage data governance and the continuous monitoring of AI models to detect issues. He addresses the balance between innovation and regulation, particularly in light of evolving EU regulations, and highlights the necessity of building trust through effective governance solutions.

Vasilije Markovich, a data engineer and AI specialist from Montenegro, discusses enhancing large language models with memory. He highlights the challenges of context window limitations and forgetting in LLMs, introducing hierarchical memory to improve performance. Vasilije dives into his creation, Cognee, which manages semantic memory, emphasizing its potential applications and the blend of cognitive science with data engineering. He shares insights from building an AI startup, the importance of user feedback, and future developments in open-source AI technology.

Tanner Burson, VP of Engineering at Prismatic, dives into the transformative effects of generative AI on software development. He discusses how AI is reshaping developer roles and productivity, fueled by tools like GitHub's Copilot. Tanner outlines both the opportunities and challenges AI presents, emphasizing the crucial need for human oversight to ensure code quality. He also explores the microunits of AI integration in workflows, the growing importance of mentorship, and the balance between innovation and practical engineering skills in an AI-driven future.

Donnie Greenberg, Co-founder and CEO of Runhouse and former product lead for PyTorch at Meta, shares insights on simplifying machine learning infrastructure. He discusses the challenges of traditional MLOps tools and presents Runhouse's serverless approach that reduces complexity in moving from development to production. Greenberg emphasizes the importance of flexible, collaborative environments and innovative fault tolerance in ML workflows. He also touches on the need for integration with existing DevOps practices to meet the evolving demands of AI and ML.

Avthar Sewrathan, Head of AI at Timescale and expert in database infrastructure, shares insights into the innovative pgai Vectorizer toolchain. He reveals how this tool enables seamless management of AI workflows in Postgres, emphasizing the importance of keeping vector data updated. The discussion covers optimizing embedding strategies, the balance between user-friendliness and customization for developers, and the future of AI integration within databases. Avthar also touches on challenges in content moderation and semantic search, highlighting the need for continuous improvement and collaboration in the open-source community.

Philip Kiely, an AI infrastructure expert at BaseTen, dives into the complexities of running generative AI models in production. He shares insights on the importance of selecting the right model based on product requirements and discusses key deployment strategies, including architecture and performance monitoring. Challenges like model quantization and the balance between open-source and proprietary models are explored. Philip also highlights future trends such as local inference, emphasizing the need for compliance in sectors like healthcare.

Philip Rathle, CTO of Neo4J and an expert in knowledge graphs, dives deep into how GraphRAG revolutionizes AI retrieval systems. He explains how this innovative method blends knowledge graphs with vector similarity for clearer, more accurate AI outputs. Rathle discusses the technical aspects of data modeling and the importance of structured data in addressing traditional retrieval challenges. The conversation also touches on real-world applications of GraphRAG across various industries, highlighting its potential to transform AI interactions.

SummaryIn this episode of the AI Engineering podcast Praveen Gujar, Director of Product at LinkedIn, talks about the applications of generative AI in digital advertising. He highlights the key areas of digital advertising, including audience targeting, content creation, and ROI measurement, and delves into how generative AI is revolutionizing these aspects. Praveen shares successful case studies of generative AI in digital advertising, including campaigns by Heinz, the Barbie movie, and Maggi, and discusses the potential pitfalls and risks associated with AI-powered tools. He concludes with insights into the future of generative AI in digital advertising, highlighting the importance of cultural transformation and the synergy between human creativity and AI.AnnouncementsHello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systemsYour host is Tobias Macey and today I'm interviewing Praveen Gujar about the applications of generative AI in digital advertisingInterviewIntroductionHow did you get involved in machine learning?Can you start by defining "digital advertising" for the scope of this conversation?What are the key elements/characteristics/goals of digital avertising?In the world before generative AI, what did a typical end-to-end advertising campaign workflow look like?What are the stages of that workflow where generative AI are proving to be most useful?How do the current limitations of generative AI (e.g. hallucinations, non-determinism) impact the ways in which they can be used?What are the technological and organizational systems that need to be implemented to effectively apply generative AI in public-facing applications that are so closely tied to brand/company image?What are the elements of user education/expectation setting that are necessary when working with marketing/advertising personnel to help avoid damage to the brands?What are some examples of applications for generative AI in digital advertising that have gone well?Any that have gone wrong?What are the most interesting, innovative, or unexpected ways that you have seen generative AI used in digital advertising?What are the most interesting, unexpected, or challenging lessons that you have learned while working on digital advertising applications of generative AI?When is generative AI the wrong choice?What are your future predictions for the use of generative AI in dgital advertising?Contact InfoWebsiteLinkedInParting QuestionFrom your perspective, what is the biggest barrier to adoption of machine learning today?Closing AnnouncementsThank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story.To help other people find the show please leave a review on iTunes and tell your friends and co-workers.LinksGenerative AILLM == Large Language ModelDall-E)RLHF == Reinforcement Learning fHuman FeedbackThe intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0