This Week in Startups

What's Next for AI Infrastructure with Amin Vahdat | AI Basics with Google Cloud

May 1, 2025

In this discussion, Amin Vahdat, VP of ML at Google Cloud, shares his insights on the groundbreaking infrastructure behind AI. He explains how Google’s TPUs are revolutionizing real-time data processing and enhancing AI capabilities. Vahdat predicts 2025 as the pivotal 'Year of Inference' for startups. He highlights advancements in low-code development, the evolution of AI agents, and the transformative potential of cloud computing. Dive in to explore how technology is reshaping decision-making and creating opportunities for entrepreneurs!

27:34

Creator website

Episode guests

Amin Vahdat

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Google's TPU technology significantly enhances AI processing capabilities, empowering startups to tackle previously unimaginable projects with efficiency.

The shift from model training to inference highlights the necessity for immediate AI responsiveness, transforming how startups leverage AI for actionable insights.

Deep dives

The Scaling of AI Infrastructure

The infrastructure needed to support AI applications has grown significantly, with Google's cloud relying on custom accelerators like tensor processing units (TPUs). These TPUs can deliver the computing power equivalent to 100 standard servers in a single chip, allowing for massive parallel processing capabilities. This innovation supports complex queries and deep research by running numerous subqueries simultaneously to compile accurate answers. With the ability to coordinate thousands of these chips, Google has built one of the largest computing clusters, paving the way for enhanced AI applications and experiences.

Intro

3min

The Age of Inference in AI

15min

The Evolution and Impact of Tensor Processing Units in AI

3min

Exploring the Unconventional Uses of Technology in a Growing Marketplace

2min

The Rise of Autonomous AI Agents in Streamlining Processes

3min

Exploring the Impact of AI on Decision-Making and Forecasting

2min

In this episode of AI Basics, Jason sits down with Amin Vahdat, VP of ML at Google Cloud, to unpack the mind-blowing infrastructure behind modern AI. They dive into how Google’s TPUs power massive queries, why 2025 is the “Year of Inference,” and how startups can now build what once felt impossible. From real-time agents to exponential speed gains, this is a look inside the AI engine that’s rewriting the future.

Timestamps:

(0:00) Jason introduces today’s guest Amin Vahdat

(3:18) Data movement implications for founders and historical bandwidth perspective

(5:29) The shift to inference and AI infrastructure trends in startups and enterprises

(8:40) Evolution of productivity and potential of low-code/no-code development

(11:20) AI infrastructure pricing, cost efficiency, and historical innovation

(17:53) Google's TPU technology and infrastructure scale

(23:21) Building AI agents for startup evaluation and supervised associate agents

(26:08) Documenting decisions for AI learning and early AI agent development

Uncover more valuable insights from AI leaders in Google Cloud's 'Future of AI: Perspectives for Startups' report. Discover what 23 AI industry leaders think about the future of AI—and how it impacts your business. Read their perspectives here: https://goo.gle/futureofai

Check out all of the Startup Basics episodes here: https://thisweekinstartups.com/basics

Check out Google Cloud: https://cloud.google.com/

Follow Amin:

LinkedIn: https://www.linkedin.com/in/vahdat/?trk=public_post_feed-actor-name

Follow Jason:

X: https://twitter.com/Jason

LinkedIn: https://www.linkedin.com/in/jasoncalacanis

Follow TWiST:

Twitter: https://twitter.com/TWiStartups

YouTube: https://www.youtube.com/thisweekin

Instagram: https://www.instagram.com/thisweekinstartups

TikTok: https://www.tiktok.com/@thisweekinstartups