No Priors: Artificial Intelligence | Technology | Startups

Launching AI products with Braintrust’s CEO Ankur Goyal

Oct 8, 2024

Ankur Goyal, the founder and CEO of Braintrust and former VP of Engineering at SingleStore, discusses the fascinating dynamics of AI development. He highlights trends in AI tooling and the rise of open-source models, emphasizing their slow adoption in enterprises. Ankur also shares insights on building resilient AI products, the importance of a startup’s early customer base, and the evolving role of CEOs in tech—balancing coding with leadership. His perspective on the future of AI integration is equally intriguing and thought-provoking.

38:28

Episode guests

Ankur Goyal

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Ankur Goyal emphasizes the importance of effective evaluations in AI development, highlighting sophisticated systems needed for comprehensive analysis of AI outputs.

The podcast discusses the shift towards simpler instruction-tuning methods over traditional fine-tuning, reflecting evolving trends in optimizing AI models.

Deep dives

Overview of Braintrust's Purpose

Braintrust functions as an end-to-end platform designed for the development of AI applications. It assists companies like Notion, Airtable, and Instacart in various areas such as evaluations, observability, and prompt development. This platform emerged from Ankur Goyal's experience in AI, particularly during his tenure with Empira and Figma, where he encountered recurring challenges in product development. The inception of Braintrust was driven by a need for effective solutions during the AI application lifecycle, highlighting the consistent problems that arise in both pre- and post-large language model (LLM) contexts.

Embrace Change: Evolving Data Infrastructure for AI

01:02

Embrace Emotion and Expertise for AI Success

01:18

Embrace Feedback to Foster Innovation

01:21

Intro

3min

Navigating AI Development Challenges

6min

Navigating AI Model Adoption

6min

Building a Startup: Lessons in Technical Excellence and Customer Focus

3min

Navigating AI: From Experimentation to Integration

10min

The CEO's Balancing Act in Tech

11min

Today on No Priors, Elad is joined by Ankur Goyal, founder and CEO of Braintrust. Braintrust enables companies like Notion, Airtable, Instacart, Zapier, and Vercel to deploy AI solutions at scale by efficiently evaluating and managing complex, non-deterministic AI applications. Ankur shares his insights into emerging trends in the use of AI tooling and coding languages, the rise of open-source, and the future of data infrastructure. Ankur also reflects on building resilient AI products, his philosophy on coding as a CEO, and the importance of a startup’s initial customer base.

Show Notes:

(0:00) Introduction

(0:38) Ankur’s path to Braintrust

(3:05) Braintrust’s solution

(5:46) AI tooling trends

(7:58) Instruction tuning vs. fine-tuning

(8:57) Open-source AI adoption

(10:42) Future of data infrastructure and synthetic data

(14:45) Designing technical interviews

(18:04) Rethinking agent-based approaches

(19:34) Building out an AI team

(23:35) Typescript as the language of AI

(25:12) The shift away from using frameworks

(26:02) Vendor consolidation among enterprises

(27:16) Coding as a CEO

(30:16) Collaborating with customers

(33:00) Future of Braintrust and evals

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more