The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Metric Elicitation and Robust Distributed Learning with Sanmi Koyejo - #352
Feb 27, 2020
Sanmi Koyejo, an assistant professor at the University of Illinois, dives into the intricacies of machine learning metrics and robust distributed learning. He highlights how traditional metrics fail in real-world decision-making, proposing innovative methods like pairwise preferences for better performance evaluation. The discussion also covers cognitive radio technology, promoting efficient spectrum use, and addresses the challenges of adversarial attacks in distributed training. Sanmi's interdisciplinary research uniquely blends cognitive science with machine learning, paving the way for more adaptive systems.
AI Snips
Chapters
Transcript
Episode notes
Complex Trade-offs in Decision Making
- Many real-world decision-making tasks involve complex trade-offs between different factors.
- Standard machine learning metrics often fail to account for these trade-offs.
Optimizing for Complex Metrics
- Classification problems often use the confusion matrix to measure model performance.
- Sanmi Koyejo's research focuses on building models that optimize complex metrics beyond accuracy, such as F-measure.
Metric Elicitation
- Metric elicitation aims to identify good metrics through interactions with experts.
- This approach seeks to capture expert trade-offs and create transportable metrics.