Ronen Eldan

Join Ronen Eldan and Yuanzhi Li from Microsoft Research as they dive into the fascinating world of tiny language models. Learn how their Tiny Stories project showcases these models' surprising storytelling abilities while prioritizing data quality over sheer size. The duo discusses new training methods that mimic human language learning and explores the emergence of reasoning skills in AI. Discover the creative challenges of generating diverse narratives for young audiences and how understanding these small models can reshape the future of AI.

Ronen Eldan and Yuanzhi Li, researchers at Microsoft, dive into their groundbreaking work on the Tiny Stories dataset, aimed to advance natural language processing while being small enough for modest compute budgets. They explore the reasoning capabilities and interpretability of tiny language models, discussing how different model sizes influence performance. The duo also highlights challenges in generating child-friendly narratives and how these models can innovate storytelling. Their insights illuminate the intricate balance of knowledge and reasoning in AI training, redefining the potential of small AI models.