Evaluating Models with Tools and Agents

This week on The Data Stack Show, Alexander Patrushev joins John to share his journey from working on mainframes at IBM to leading AI infrastructure innovation at Nebius, with stops at VMware and AWS along the way. The discussion explores the evolution of AI and cloud infrastructure, the five pillars of successful machine learning projects, and the unique challenges of building and operating modern AI data centers—including energy consumption, cooling, and networking. Alexander also delves into the practicalities of infrastructure as code, the importance of data quality, and offers actionable advice for those looking to break into the AI field. Key takeaways include the need for strong data foundations, thoughtful project selection, and the value of leveraging existing skills and tools to succeed in the rapidly evolving AI landscape. Don’t miss this great conversation.

Highlights from this week’s conversation include:

• Alexander’s Background and Early Career at IBM (1:06)
• Moving From Mainframes to Virtualization at VMware (4:09)
• Transitioning to AWS and Machine Learning Projects (8:22)
• What Was Missed From Mainframes and the Rise of Public Cloud (9:03)
• Security, Performance, and Economics in Cloud Infrastructure (12:40)
• The Five Pillars of Successful Machine Learning Projects (15:02)
• Choosing the Right ML Project: Data, Impact, and Existing Solutions (18:01)
• Real-World AI and ML Use Cases Across Industries (19:42)
• Building Specialized AI Clouds Versus Hyperscalers (22:08)
• Performance, Scalability, and Reliability in AI Infrastructure (25:18)
• Data Center Energy Consumption and Power Challenges (28:41)
• Cooling, Networking, and Supporting Systems in AI Data Centers (30:06)
• Infrastructure as Code and Tooling in AI (31:50)
• Lowering Complexity for AI Developers and the Role of Abstraction (34:08)
• Startup Opportunities in the AI Stack (38:53)
• When to Fine-Tune or Post-Train Foundation Models (43:41)
• Comparing and Testing Models With Tool Use (47:49)
• Skills and Advice for Entering the AI Field (49:18)
• Final Thoughts and Encouragement for AI Newcomers (52:31)

The Data Stack Show is a weekly podcast powered by RudderStack, customer data infrastructure that enables you to deliver real-time customer event data everywhere it’s needed to power smarter decisions and better customer experiences. Each week, we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.