Exploring the Limitations and Advantages of Language Models in Math Problem Solving

Jack Krawczyk is a Senior Director of Product at Google, building Bard. Bard is Google’s collaborative, conversational, and experimental AI tool that’s bridging the gap between humans and bots, while addressing ethical considerations around AI. After joining the project in 2020, Jack helped ship Bard in less than four years. Bard sources information directly from the web, and now enables users to inquire about and summarize YouTube videos.

—

In today’s episode, we discuss:

• Key lessons from Bard’s development process
• Ethics in AI
• How Bard shipped fast
• What separates Bard from competitors
• The future of LLM, Generative AI, and AGI
• Advice for aspiring AI developers

—

Referenced:

• Bard: https://bard.google.com/
• ChatGPT: https://chat.openai.com/
• Duet AI: https://cloud.google.com/duet-ai
• Free courses on machine learning by Andrew Ng: https://www.andrewng.org/courses/
• Google Assistant: https://assistant.google.com/
• Introducing Google Assistant to Bard: https://blog.google/products/assistant/google-assistant-bard-generative-ai/
• Large Language Model (LLM): https://en.wikipedia.org/wiki/Large_language_model
• Meena: https://blog.research.google/2020/01/towards-conversational-agent-that-can.html
• Sissie Hsiao (GM at Bard): https://www.linkedin.com/in/sissie-hsiao-b24243/
• Steve Stoute: https://www.linkedin.com/in/stevestoute/
• UnitedMasters: https://unitedmasters.com/

—

Where to find Jack Krawczyk:

• Twitter/X: https://twitter.com/JackK
• LinkedIn: https://www.linkedin.com/in/jack--k

—

Where to find Brett Berson:

• Twitter/X: https://twitter.com/brettberson
• LinkedIn: https://www.linkedin.com/in/brett-berson-9986094/

—

Where to find First Round Capital:

• Website: https://firstround.com/
• First Round Review: https://review.firstround.com/
• Twitter: https://twitter.com/firstround
• Youtube: https://www.youtube.com/@FirstRoundCapital
• This podcast on all platforms: https://review.firstround.com/podcast

—

Timestamps:

(00:00) Introduction

(02:17) Bard’s origin story

(03:54) Deciding on the application of Bard

(05:59) The ethical considerations around building Bard

(10:19) Why Bard launched to the public so early

(13:30) Risk-taking at big companies versus smaller ones

(16:20) Bard’s early user research

(21:21) Bard versus ChatGPT

(25:01) The cultural and product principles behind Bard

(30:56) Insight into Bard’s impressive development speed

(35:17) Deciding when to ship Bard

(41:41) Why Bard is different from other products Jack has built

(46:30) Evaluating Bard’s original spec

(48:02) Insight into Bard's product roadmap

(56:00) The toughest challenges Bard has faced

(57:50) What’s special about team-building at Bard

(62:54) Addressing Bard’s negative press

(67:49) Advice for aspiring LLM companies

(69:15) Advice for non-LLM companies

(71:05) The biggest barriers to advancing AI

(75:45) How product people can use or build with AI

(77:24) How AI is changing product leadership

(79:20) People who had an outsized impact on Jack