Mixture of Experts

Anastasia Stasenko, CEO of pleias and open-source AI advocate, joins AI experts Marina Danilevsky and Chris Hay to dive deep into the Paris AI Summit highlights. They discuss the global advancements in AI safety and a remarkable €109 billion investment to boost European infrastructure. The trio explores the implications of an intriguing new test-time scaling technique for AI models, as well as insights from Sam Altman's latest observations. They also dissect Anthropic's Economic Index, revealing the actual scope of AI adoption across industries.

Joining the conversation are Marina Danilevsky, a senior research scientist focused on AI ethics; Chris Hay, a distinguished engineer and CTO specializing in AI product development; and Nathalie Baracaldo, an expert in AI safety and security. They explore OpenAI’s recent deep research and the o3-mini model, discussing its capabilities and limitations. Insights on the upcoming AI Action Summit highlight its significance in navigating AI governance and ethical implications. The trio also dissects Anthropic’s Constitutional Classifiers and Microsoft’s new unit studying AI's impact on society.

In this engaging discussion, Kate Soule, Director of Technical Product Management at Granite, Chris Hay, Distinguished Engineer and CTO of Customer Transformation, and Aaron Baughman, IBM Fellow and Master Inventor dive into the realities behind DeepSeek R1. They debunk myths surrounding its hype and discuss the true implications of model distillation for AI competition. The trio explores the evolving landscape of open-source AI and how recent advancements can reshape industry strategy, shedding light on efficiency and innovation in model training.

Join experts Abraham Daniels, a senior technical product manager specializing in AI and open-source models, Kaoutar El Maghraoui, a principal research scientist leading AI hardware innovations, and Skyler Speakman, a senior research scientist focusing on AI technology. They unravel the implications of DeepSeek's open-source model launch, Mistral's IPO plans, and the controversial FrontierMath benchmarks. They also discuss IDC's findings on coding assistants, highlighting the shift towards specialized versus generalist tools in the programming landscape.

What would you do with $2 billion? In episode 38 of Mixture of Experts, join host Tim Hwang along with experts Chris Hay, Kaoutar El Maghraoui and Vyoma Gajjar to discuss the Anthropic valuation rumors. Next, Microsoft CEO Nadella created a new CoreAI group to build and run apps for customers. Then, NotebookLM upgraded some of its features, including podcast intervention. Finally, AI agents are making their way into the financial services industry. Can an agent invest all of your money? Tune-in to this week’s episode to find out. 00:01 -- What would you do with $2 billion? 00:51 -- Anthropic valuation 12:14 -- Microsoft CoreAI 25:01 -- NotebookLM upgrades 35:17 -- AI agents in finance The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity. 

What’s the most exciting CES AI announcement? In episode 37 of Mixture of Experts, host Tim Hwang is joined by Skyler Speakman, Volkmar Uhlig and Shobhit Varshney to debrief CES 2025. Specifically, the experts dive into NVIDIA’S Project DIGITS, among other announcements from the AI hardware giant. Next, a new enterprise AI development survey came out that detailing how developers really feel about AI implementation. Then, Apple Intelligence experienced some major hallucination fails, what does this tell us about Apple’s stake in the AI game? Finally, Sam Altman of OpenAI released a reflection blog, what does he say about the future of AI? All that and more on today’s Mixture of Experts.The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Is deep learning hitting a wall? It’s 2025 and Mixture of Experts is back and better than ever. In episode 36, host Tim Hwang is joined by Chris Hay, Kate Soule and Kush Varshney to debrief one of the biggest releases of 2024, OpenAI o3. Next, DeepSeek-V3 is here! Finally, will AI exist in 2027? The experts dissect the AI bet between Miles Brundage and Gary Marcus. All that and more on the first Mixture of Experts of 2025.The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.00:00 — Intro00:49 — OpenAI o314:40 — DeepSeek-V328:00 — The Brundage/Marcus bet

Will 2025 be the year of AI agents? In Episode 35 of Mixture of Experts, host Tim Hwang is joined by some show veterans to debrief 2024 in AI. This week, we review AI models, agents, hardware and product releases with some of the top industry experts. What was the best model of 2024? Is NVIDIA king? What are some of the AI trends in 2025? All that and more on this special edition of Mixture of Experts.The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.

Is pre-training a thing of the past? In Episode 34 of Mixture of Experts, host Tim Hwang is joined by Abraham Daniels, Vagner Santana and Volkmar Uhlig to debrief this week in AI. First, OpenAI cofounder Ilya Sutskever said that “peak data” was achieved, does this mean there is no longer a need to model pre-training? Next, IBM released Granite 3.1 with a slew of features, we cover them all. Then, there is a new way to steal AI models, how do we protect against model exfiltration. Finally, can NVIDIA Jetson for AI developers really increase hardware accessibility? Tune-in for more!The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.00:01 — Intro00:49— Is pre-training over?10:25 — Granite 3.122:23 — AI model stealing33:38—NVIDIA Jetson

Is o1 Pro worth the cost? In Episode 33 of Mixture of Experts, host Tim Hwang is joined by Marina Danilevsky, Kate Soule and Vyoma Gajjar. First, the experts debrief the 12 Days of OpenAI. Next, we review some of the top papers in NeurIPS, how are the experts keeping up with all these research papers? Then, we are back with another benchmark, can ARC Prize make AGI more tractable? Finally, Meta announced the launch of Llama 3.3 70B with the promise of 405B performance, can we have our cake and eat it too? Find out more on today’s Mixture of Experts!The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.