The Rise of TPU: Powering AlphaGo

The story of Groq, a semiconductor startup that makes chips for AI inference and was recently valued at $2.8 billion, is a classic “overnight success that was years in the making” tale. On this episode, I talk with founder and CEO Jonathan Ross.

He began the work that eventually led to Groq as an engineer at Google, where he was a member of the rapid eval team – “the team that comes up with all the crazy ideas at Google X.” For him, the risk involved in leaving to launch Groq in 2016 was far less than the risk of staying in-house and watching the project die.

Groq has had many “near-death” experiences in its eight years of existence, all of which Jonathan believes have ultimately put it in a much stronger position to achieve its mission: preserving human agency in the age of AI.

Groq is committed to giving everyone access to relatively low-cost generative AI compute, driving the price down even as they continue to increase speed. We talked about how the company culture supports that mission, what it feels like to now be on the same playing field as companies like Nvidia, and Jonathan’s belief that true disruption isn’t just doing things other people can’t do or don’t want to do, but doing things other people don’t believe can be done – even when you show them evidence to the contrary. 

Other topics we touched on include:

• Why the ability to customize on demand makes generative AI different 
• Managing your own and other people’s fear as a founder
• The problems of corporate innovation
• The role of luck in business
• How he thinks about long-term goals and growth

—

Brought to you by:

Mercury – The art of simplified finances. ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Learn more⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠.

DigitalOcean – The cloud loved by developers and founders alike. ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Sign up⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠.

Runway – The finance platform you don’t hate. ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Learn more⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠.

—

Where to find Jonathan Ross:

• X: ⁠https://x.com/JonathanRoss321⁠ 

• LinkedIn: ⁠https://www.linkedin.com/in/ross-jonathan/⁠

Where to find Eric:

• Newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://ericries.carrd.co/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ 

• Podcast: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://ericriesshow.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ 

• YouTube: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.youtube.com/@theericriesshow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ 

—

In This Episode We Cover:

(04:24) Jonathan’s involvement with the DeepMind Challenge Match between AlphaGo and Lee Sedol

(06:06) How Jonathan’s work Google and how it led him to that moment

(08:46) Why generative AI isn’t just the next internet or mobile

(10:12) The divine move in the DeepMind Challenge Match

(11:56) How Jonathan ended up designing chips without the usual background

(13:11) GPUs vs. TPUs

(14:33) What risk really is

(15:11) Groq’s mind-blowing AI demo 

(16:23) How Jonathan decided to leave Google and start Groq

(17:30) The differences between doing an innovation project at a company and starting a new company

(19:03) Nassim Taleb’s Black Swan theory

(21:02) Groq’s founding story

(24:12) The difference in attitude towards AI now compared to 2016 and how it affected Groq

(25:46) The moment the tide turned with LLMs

(28:28) The week-over-week jump from 8,000 users to 400,000 users

(30:32) How Groq used HBM and what is it (the memory used by GPUs)

(32:33) Jonathan’s approach to disruption

(35:38) Groq’s initial raise and focus on software

(36:13) How struggling to survive made Groq stronger

(37:13) Hiring for return on luck

(40:07) How Jonathan and Groq think about the long-term

(42:25) Founder control issues

(45:31) How Groq thinks about maintaining its mission and trustworthiness

(49:51) Jonathan’s vision for a capital market that would support companies like Groq

(52:58) How Groq manages internal cultural alignment

(55:59) Groq’s mission and to preserve human agency in the age of AI how it approaches achieving it

(59:48) Lightning round

You can find the transcript and references at ⁠⁠https://www.ericriesshow.com/⁠

—

Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠.

Eric may be an investor in the companies discussed.