Machine Learning Fundamentals, Part 2 with Shannon Wirtz

Nov 21, 2025

Ask episode

Chapters

Transcript

Episode notes

We continue our exploration of machine learning fundamentals with Shannon Wirtz, diving deeper into advanced model architectures, training techniques, and evaluation methods.

We start with ensemble learning - why combining multiple models often outperforms single models, and how techniques like Random Forest and XGBoost prevent overfitting through clever sampling strategies. From there, we explore neural networks, understanding how they learn directly from raw data through sequences of linear and nonlinear transformations.

The conversation covers the evolution from convolutional neural networks (perfect for images) to recurrent neural networks (for sequences) to Transformers (the architecture behind modern LLMs). We dive into how Transformers revolutionized natural language processing through parallelization and attention mechanisms, enabling the large language models we see today.

We then shift to the critical topic of model evaluation - exploring loss functions, gradient descent, learning rates, and the importance of proper train/validation/test splits. Shannon explains why you need separate validation and test sets, how k-fold cross-validation works, and the various metrics used to assess model performance beyond simple accuracy.

Topics include:

Ensemble learning: why combining models works (Random Forest, XGBoost)
Neural networks: linear and nonlinear transformations, neurons, and layers
Convolutional Neural Networks (CNNs): recognizing visual patterns and edges
Transformers: the architecture behind modern LLMs, attention mechanisms, and why they’re so powerful
Training and evaluation: loss functions, gradient descent, and learning rates
Train/validation/test splits and why you need all three
K-fold cross-validation: a more robust evaluation approach
Performance metrics including precision, recall, F1 score, AUC, and the confusion matrix
Model interpretation: white box vs black box models
Interpretation techniques including partial dependence plots, SHAP values, and individual conditional expectations
Learning resources: Andrew Ng’s courses, Kaggle, DataCamp, and hands-on projects

Shannon also shares his personal learning journey, from rote learning to practical hands-on experience, and discusses how he learns most effectively through immediate feedback and engaging projects.

Whether you’re looking to understand how modern AI systems work or seeking practical guidance on getting started with machine learning, this episode provides both theoretical depth and practical strategies for building and evaluating ML models.

This is Part 2 of a 2-part series. In Part 1 , we explored the foundations of machine learning - including core concepts, terminology, different learning approaches, and fundamental model types.

Show Links