The Probability of Replicating a Study

2min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

Today, when I look at a paper in a top journal, I assume there's about a 40% chance it won't replicate. Do you think that I'm being overly pessimistic? "I would be pretty confident that you're going to get the same results," he says. The other thing is that some of the academic reformers believe this kind of study where we get the same result on a different population of people is as central to understanding the social world as some people seem to think.

Read the full transcript here.

What makes studying humans harder than studying other parts of the universe? Is social science currently improving its rigor, relevance, and self-reflection? Is it improving its predictive power over time? Why have sample sizes historically been so small in social science studies? Is social science actually able to accumulate knowledge? Have social scientists been able to move the "needle" on real-world problems like vaccine adoption? Is social science becoming more diverse? Specifically, does social science have a political bias? Are universities in crisis? Do the incentive structures in universities make them difficult or even impossible to reform?

Matt Grossmann is Director of the Institute for Public Policy and Social Research and Professor of Political Science at Michigan State University. He is also Senior Fellow at the Niskanen Center and a Contributor at FiveThirtyEight. He has published analysis in The New York Times, The Washington Post, and Politico, and hosts the Science of Politics podcast. He is the author or co-author of How Social Science Got Better, Asymmetric Politics, Red State Blues, The Not-So-Special Interests, Artists of the Possible, and Campaigns & Elections, as well as dozens of journal articles. You can find more about him on his website.

Staff