What meta‑analytic effect sizes mean in practice

Hello everyone! We weren’t able to record a podcast this week, because 1) Stuart was busy and 2) it’s Tom’s birthday. So by way of apology we’re re-releasing this one about some drama last year between Jon Haidt, sworn enemy of smartphones, and some guys who like meta-analyses. Hope you enjoy it!

A while back, The Studies Show covered the question of whether smartphones and social media cause mental health problems. Amazingly, that podcast didn’t settle the issue, and the debate has continued—and continued rather acrimoniously.

Psychologists—most notably Jonathan Haidt—are currently laying into each other, analysing, re-analysing, and meta-analysing datasets to try and work out whether “it’s the phones”. In this paid-only episode of The Studies Show, Tom and Stuart explain the story so far, and in the process get very disappointed by their heroes.

If you want to hear the whole episode and read the show notes, it’s easy to become a paid subscriber at thestudiesshowpod.com.

Show notes

* The summary of Jonathan Haidt’s upcoming book, Life After Babel

* The Google Doc on social media effects maintained by Haidt, Twenge, and Rausch

* Christopher Ferguson’s meta-analysis of causal social media effects studies

* Very useful online calculator to interpret effect sizes

* Study on the (non-)relation between reported and measured phone use

* Haidt & Rausch’s first article criticising the Ferguson meta-analysis and re-calculating the effects

* Anne Scheel’s critical tweet

* Matt Jané’s first article responding to Haidt & Rausch

* Haidt & Rausch respond to Jané (and criticise Ferguson again)

* Jané responds to Haidt & Rausch, again

* Haidt & Rausch’s second (or is it third?) article criticising the Ferguson meta-analysis (this is the one where they note the more basic errors)

* Article by Mike Males making the point that, whoever is right, the effects are all very small

Credits

* The Studies Show is produced by Julian Mayers at Yada Yada Productions. We’re very grateful to Malte Elson, Pete Etchells, and Matt Jané for talking to us for this episode—but any errors are our own.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sciencefictionspod.substack.com/subscribe