The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Inside s1: An o1-Style Reasoning Model That Cost Under $50 to Train with Niklas Muennighoff - #721

Mar 3, 2025

Niklas Muennighoff, a PhD student at Stanford, dives into his groundbreaking work on the S1 reasoning model, designed to efficiently mimic OpenAI's O1 while costing under $50 to train. He elaborates on innovative techniques like 'budget forcing' that help the model tackle complex problems more effectively. The discussion highlights the intricacies of test-time scaling, the importance of data curation, and the differences between supervised fine-tuning and reinforcement learning. Niklas also shares insights on the future of open-sourced AI models.

49:29

Creator website

Episode guests

Niklas Muennighoff

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The S1 model introduces a budget forcing technique that optimizes computational effort during reasoning by regulating answer generation based on token budgets.

S1's open-source nature and minimal resource requirements foster accessibility and promote further experimentation in AI reasoning applications among researchers.

Deep dives

Comparison of S1 and R1 Approaches

The S1 and R1 models seek to replicate the functionality of OpenAI's O1 model, but they do so with different methodologies. R1 aims to replicate the entire pipeline established by O1, striving for a comprehensive reconstruction of its functionalities. In contrast, S1 is focused on achieving the core benefits of O1—strong reasoning performance and test time scaling—through a more minimalistic approach. This strategic difference has implications for the complexity and resource demands of each model, with S1 designed to be more accessible and cost-effective.

Intro

2min

Innovative Approaches to Model Output Aggregation

5min

Analyzing the Cost-Effective S1 Model and Its Relation to O1

3min

Optimizing Reasoning Model Training

16min

Optimizing Model Performance

6min

Exploring Open Source Model Replication and Base Model Selection

3min

Evaluating Language Models: Biases and Learning Strategies

14min

Today, we're joined by Niklas Muennighoff, a PhD student at Stanford University, to discuss his paper, “S1: Simple Test-Time Scaling.” We explore the motivations behind S1, as well as how it compares to OpenAI's O1 and DeepSeek's R1 models. We dig into the different approaches to test-time scaling, including parallel and sequential scaling, as well as S1’s data curation process, its training recipe, and its use of model distillation from Google Gemini and DeepSeek R1. We explore the novel "budget forcing" technique developed in the paper, allowing it to think longer for harder problems and optimize test-time compute for better performance. Additionally, we cover the evaluation benchmarks used, the comparison between supervised fine-tuning and reinforcement learning, and similar projects like the Hugging Face Open R1 project. Finally, we discuss the open-sourcing of S1 and its future directions.

The complete show notes for this episode can be found at https://twimlai.com/go/721.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more