Stat Series: What Statistical Measure Are You Overusing? (And What to Do About It), Part One
Mar 1, 2024
auto_awesome
This podcast dives deep into the misleading nature of averages in statistical measures, discussing their addictive use and potential inaccuracies. It highlights the pitfalls of relying on averages for work forecasting and bug reporting, advocating for a more nuanced approach. The episode challenges the overuse of averages in estimation, pointing out biases and risks associated with solely relying on average values in data analysis.
Relying solely on averages for estimations can lead to inaccuracies and late deliveries due to overlooking variations and outliers in the data.
Shifting focus to understanding outlier risks and confidence intervals can provide more accurate delivery timelines than relying on average estimates.
Deep dives
The Misleading Nature of Averages
The average, often used as a heuristic to summarize data, can be misleading due to its representation of underlying information. Normal distribution implies a predictable pattern around the average, yet deviations from the average can significantly impact outcomes. In cases where data is not normally distributed, the average may not reflect the whole dataset accurately. For instance, a group with ages of 90 and 1 collectively having an average of 19 showcases the average's limitation.
Challenges with Using Averages for Estimations
Using averages for estimations can lead to inaccuracies and late deliveries due to the tendency to perceive averages as representative of outcomes. Quantitative methods like looking at past velocities for estimations can overlook the variations and potential outliers in the data. The negativity bias further compounds the issue, leading to a perception of constant delays when relying solely on averages for predictions. A shift towards evaluating outlier risks and confidence intervals rather than average estimates can offer more accurate delivery timelines.
Moving Beyond Averages in Estimations
To improve estimation accuracy, considering risks and outliers in delivery timelines can provide a more realistic approach than relying on averages. By identifying and accounting for outlier risks through historical data analysis, teams can offer more reliable estimates with a level of confidence. Shifting the focus from midline averages to understanding confidence in delivery timelines can lead to more realistic and dependable estimations, addressing the limitations associated with relying solely on averages.
On average, you're probably overusing this specific type of statistic. In today's episode, we discuss the king of all misleading numbers: averages!
There's so much to talk about with averages that we're splitting this into two parts. Disclaimer: I am not a mathematician. But we will talk about some of the interesting properties of averages and why they are so addictive to use for humans, but more practically what counterintuitive ways we might be using them incorrectly.
If you're using your sprint velocity to forecast work, this episode is for you!
If you’re an engineer and you would rather spend your time writing code than responding to comments in your issue tracker, send your team Jam.dev. Go to jam.dev to get started, it’s free.
📮 Ask a Question
If you enjoyed this episode and would like me to discuss a question that you have on the show, drop it over at: developertea.com.
If you want to be a part of a supportive community of engineers (non-engineers welcome!) working to improve their lives and careers, join us on the Developer Tea Discord community by visiting https://developertea.com/discord today!
🧡 Leave a Review
If you're enjoying the show and want to support the content head over to iTunes and leave a review! It helps other developers discover the show and keep us focused on what matters to you.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode