Practical AI

Evaluating models without test data

Sep 20, 2022

Charles Martin, an AI and data science consultant, created WeightWatcher, a unique tool for analyzing neural networks without needing test data. In this discussion, he dives into the challenges of evaluating machine learning models, especially in fields like retail and finance. He emphasizes the significance of human judgment and innovative methods for model assessment. Martin also highlights the integration of statistical methods from physics in AI, showcasing how they can enhance model training and optimization processes.

44:53

Creator website

Episode guests

Charles Martin

AI Chapters

Episode notes

Intro

6min

Challenges in Machine Learning Model Evaluation

10min

Monitoring and Evaluating Transformer Models in NLP

4min

Evaluating Machine Learning Models

11min

Optimizing Machine Learning Workflows for Better Model Convergence

3min

Navigating Hyperparameters and Open-Source in Deep Learning

7min

The Intersection of Theory and Engineering in AI

4min

WeightWatcher, created by Charles Martin, is an open source diagnostic tool for analyzing Neural Networks without training or even test data! Charles joins us in this episode to discuss the tool and how it fills certain gaps in current model evaluation workflows. Along the way, we discuss statistical methods from physics and a variety of practical ways to modify your training runs.

Join the discussion

Changelog++ members support our work, get closer to the metal, and make the ads disappear. Join today!

Featuring:

Charles Martin – GitHub, LinkedIn, X
Chris Benson – Website, GitHub, LinkedIn, X
Daniel Whitenack – Website, GitHub, X

Show Notes:

Something missing or broken? PRs welcome!

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.