3min snip

Invest Like the Best with Patrick O'Shaughnessy cover image

Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]

Invest Like the Best with Patrick O'Shaughnessy

NOTE

The High Marginal Costs and Importance of MFU in AI

AI development faces high marginal costs unlike traditional software that benefits from zero marginal costs. The scaling laws imply that significant investments are required to achieve quality improvements in AI models. While marginal costs can decrease substantially over time due to various factors, they initially remain high at the forefront of model development. Consequently, infrastructure efficiency, highlighted by Model Flops Utilization (MFU), emerges as a crucial success factor for AI companies. Current MFU averages around 35-40%, indicating a notable gap between theoretical compute capacity and actual usage. Higher MFU allows companies to capitalize on efficiencies, enabling faster time-to-market, improved model quality, or reduced operational costs. For instance, even a modest increase in MFU can translate to significant cost savings in inference through techniques like quantization. The competitive landscape potentially renders AI models as commodities, yet as scaling laws progress, the associated costs could prohibitively elevate, transforming scale into a key competitive barrier. Therefore, MFU serves as a vital metric for evaluating AI research success and provides a basis for differentiation among leading companies in this rapidly evolving field.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode