13. Joe Hilgard: Scientific fraud, reporting errors, and effects that are too big to be true

Joe Hilgard is Assistant Professor of Social Psychology at Illinois State University. In this conversation, we discuss his work on detecting and reporting scientific fraud. 

BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith. New conversations every other Friday. You can find the podcast on all podcasting platforms (e.g., Spotify, Apple/Google Podcasts, etc.).

Timestamps
0:00:05: Are we only catching the dumb fraudsters?
0:08:45: Why does Joe always sign his peer reviews?
0:11:51: Detecting errors during peer review
0:17:44: Retractions motivated by Joe's work
0:22:19: The whole Zhang affair
0:49:19: Ben found errors in a paper. Joe advises what to do next
1:04:06: How to separate negligible errors from serious errors that require action
1:11:37: When effects are too big to be true

Podcast links

• Website: https://bjks.buzzsprout.com/
• Twitter: https://twitter.com/BjksPodcast

Joe's links

• Website: http://crystalprisonzone.blogspot.com/
• Google Scholar: https://scholar.google.de/citations?user=FPOHtgQAAAAJ
• Twitter: https://twitter.com/JoeHilgard

Ben's links

• Website: www.bjks.blog/
• Google Scholar: https://scholar.google.co.uk/citations?user=-nWNfvcAAAAJ

References
Brown, N. J., & Heathers, J. A. (2017). The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science.
Callaway, E. (2011). Report finds massive fraud at Dutch universities. Nature News.
Friston, K. (2012). Ten ironic rules for non-statistical reviewers. Neuroimage.
Heathers, J. A., Anaya, J., van der Zee, T., & Brown, N. J. (2018). Recovering data from summary statistics: Sample parameter reconstruction via iterative techniques (SPRITE) . PeerJ Preprints.
Hilgard, Joe's blog post about the Zhang affair: http://crystalprisonzone.blogspot.com/2021/01/i-tried-to-report-scientific-misconduct.html
Hilgard, J. (2021). Maximal positive controls: A method for estimating the largest plausible effect size. Journal of Experimental Social Psychology.
Hilgard, J. (2019). Comment on Yoon and Vargas (2014): An implausibly large effect from implausibly invariant data. Psychological Science.
Lakens, Daniel: blog post on hungry judges: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html
Morey, R. D., Chambers, C. D., ... & Zwaan, R. A. (2016). The Peer Reviewers' Openness Initiative. Royal Society Open Science.
O'Grady: Write up in Science Magazine about the Zhang affair: https://science.sciencemag.org/content/371/6531/767
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2013). Life after p-hacking. In Meeting of the society for personality and social psychology, New Orleans, LA.
Simmons, J. What do true findings look like: Presentation slides available at https://osf.io/93fkq/
Stapel's autobiography freely available in English: http://nick.brown.free.fr/stapel
Yong, E. (2012). The data detective. Nature News.