High Signal: Data Science | Career | AI Episode 15: Why Good Metrics Still Lead to Bad Decisions — and How to Fix It
33 snips
Apr 24, 2025 Eoin Mahony, data science partner at Lightspeed and former Uber science lead, shares his insights on effective metrics. He argues that metrics can mislead if their underlying mechanisms aren’t understood, a lesson he learned while optimizing NYC's Citi Bikes. Eoin discusses the pitfalls of relying too heavily on simplistic data and emphasizes the need for rigorous analysis in data-driven decision-making. He also dives into how generative AI can improve workflows while navigating the hype surrounding tech adoption, blending practical advice with real-world examples.
AI Snips
Chapters
Transcript
Episode notes
Misleading Positive Metrics
- Positive metrics can be meaningless without understanding the mechanism behind them.
- Misunderstanding measurement can lead to doing the opposite of what is intended.
Incentives Changed CityBike Use
- Eoin designed a bike incentive system for NYC CityBike to rebalance usage.
- This small behavioral nudge led to significant operational improvements and was highly adopted by users.
Keep Algorithms Simple and Practical
- Simplify algorithms to make them practical and actionable.
- Focus on easy-to-follow instructions to ensure operational success.