The Peel with Turner Novak

How the Smartest Companies Use AI | Ankur Goyal, Braintrust

Jun 7, 2024

Ankur Goyal, CEO of Braintrust, discusses the importance of LLM evals, having two AI roadmaps, and the impact of AI on software development. Topics include AI advancements, tool calling, and advice for young software engineers. Goyal shares insights on selecting AI models, staying updated on AI developments, and the significance of purpose in career growth.

01:27:03

Creator website

Episode guests

Ankur Goyal

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Companies are adapting by having two product roadmaps: one for incremental AI improvements and another for transformative advancements.

Software development is shifting towards data-centric processes, moving away from code-centric approaches in favor of data-driven insights.

Brain Trust emphasizes autonomy and minimal meetings, focusing on granting individuals freedom and encouraging autonomy over micromanagement.

Deep dives

Product Roadmaps and AI Integration

Companies are adapting by having two product roadmaps: one assuming incremental AI improvements and another considering more transformative advancements. Customers forecast a significant increase in AI integration in engineering projects over the next years, marking a shift in software development paradigms towards data-centered processes.

Harnessing Feedback for AI Innovation

01:16

Embrace the New Era of AI Development

01:18

Embrace Uncertainty for Strategic Growth

01:22

Embrace Dual Roadmaps for AI Advancement

01:09

Intro

4min

Incorporating AI in Companies and the Importance of Evaluation for AI Systems

11min

Exploring AI Advancements and Tools for Developer Innovation

20min

The Significance of Tool Calling in AI Models

5min

Journey into Programming and Importance of Sleep and Transparency in Career Growth

11min

Developing AI for Understanding Unstructured Data

2min

Lessons Learned from Running Companies and Building Braintrust

23min

Handling Software Deployment and AI Initiatives for Enterprise Scale

9min

Innovative AI Product Development and Braintrust's Role

2min

Get Attio, the next generation of CRM: https://bit.ly/AttioThePeel

Ankur Goyal is the Founder and CEO of Braintrust, the end to end developer platform for building the world's best AI products. Their customers include companies like Instacart, Zapier, Notion, Airtable, Replit, and more.

We hit on the importance of LLM evals, advice for building AI products, why the best companies have two AI product roadmaps, and his non-conventional advice for founders.

Timestamps:

(00:00) Intro (04:04) Why everyone’s now an AI company (06:03) Reasons LLM evals are so important (08:10) Typescript becoming the language of AI (09:19) Replacing vibe checks with Braintrust (10:37) Making OpenAI’s protocols the standard (11:27) Why the best companies have two AI roadmaps (13:06) Building your product so each LLM release makes it better (14:54) Predicting AGI is impossible (15:54) Why people who work with LLMs aren’t worried about AI safety (16:52) The best developers are all-in on co-pilots (18:11) How AI is changing software development (21:09) Combining IDE, CIDC, and observability in one product (27:18) Are models more like CPU’s or relational databases? (30:14) How to pick an LLM (33:00) Advice for staying on top of new AI developments (34:30) Why tool calling is so important (38:02) Advice for young software engineers (40:25) Learning to code doing linear algebra homework (42:36) Lack of purpose interning in big tech (44:07) Working at MemSQL learning to be a founder (47:52) How to get a job at a startup (50:43) Building his first startups product on an international flight (52:39) Three lessons from his first failed startup (54:46) Don’t delegate what you’re good at (55:46) Why you should be careful listening to VCs advice (57:34) Tactics for successful delegation (59:36) Why Ankur doesn’t do any meetings (01:02:42) The importance of self-service in unlocking certain customer segments (01:05:14) How Braintrust got started (01:07:45) Advice on picking your target customers (01:10:35) How Braintrust hires with work trials (01:15:21) Balancing security with a modern UI (01:17:49) Why it’s hard to sell non-AI products right now (01:19:21) Advice for selling to large enterprises (01:23:10) Ankur’s favorite AI products

Referenced:

https://www.braintrustdata.com/

SICP Book

PDF: https://web.mit.edu/6.001/6.037/sicp.pdf

Hardcopy: https://www.amazon.com/Structure-Interpretation-Computer-Programs-Engineering/dp/0262510871

Linear’s guide to work trials: https://linear.app/blog/why-and-how-we-do-work-trials-at-linear

Where to find Ankur:

Twitter: https://twitter.com/ankrgyl

LinkedIn: https://www.linkedin.com/in/ankrgyl/

Where to find Turner:

Twitter: https://twitter.com/TurnerNovak

LinkedIn: https://www.linkedin.com/in/turnernovak/