Everything Hertz

Dan and James chat with Fiona Fidler (University of Melbourne), who is leading the repliCATS project, which aims to develop accurate techniques to elicit estimates of the replicability of research. This is also the first time they interview a guest live! Here's what they discuss... The story behind repliCATS Australia's best export, Tim Tams The SCORE project organised by DARPA Can anyone use the repliCATS methodology? Dan, Fiona, and James talk about did their honours theses (this is roughly the Australian equivalent of a Masters) What would a successful repliCATS project look like? What sort of heuristics do people use to assess replicability? The AIMOS conference The role of replicability in public policy This is Bob Katter Should we be keeping the replication crisis behind closed doors? Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, October 21) "Predicting the replicability of research ", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/KZPYG, Retrieved from https://osf.io/kzpyg/Special Guest: Fiona Fidler.Support Everything Hertz

Dan and James answer a listener question on how to navigate open science practices, such as preprints and open code repositories, in light of double-blind reviews. Stuff they cover: How common is double-blind review? How many journals don’t accept preprints? Bias in the review process How practical is blinded review? Do the benefits of preprints outweighs not having blinded review? James' approach to getting comments on his preprints Convincing your supervisor to adopt open science practices The preprint that James won’t submit for publication, for some reason We get reviews... Our first live guest! Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Citation Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, October 7) "Double-blind peer review vs. Open Science", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/7ZPMESupport Everything Hertz

Dan and James discuss the role of Google Scholar in citation patterns and whether we should limit academics to only publishing two papers a year. Links and details: James has a new Hertz-quarters The Metascience conference How is google scholar influencing citation patterns A slide from @Jevinwest's presentation on Google Scholars Is this a symptom of lazy citaton practices rather than the algorithm? What are the alternatives to google scholar? Should google open up the algorithm? GS will find your preprint and link it to the paywall link Why is Google Scholar free? What would make GS better? Using the Zotero plugin for to collected citation info in bulk from search results in GS Top recommended articles in GS are phenomenal GS is not great for meta-analysis GS reduces friction Should we limit academics to only publishing two papers a year, as suggested by former guest, Dorothy Bishop? Who would stand to lose from this? Is this idea practical? What if only a few countries or institutions implemented this? The Japanese math genius who posts papers on his own website Chaos in the brickyard paper Chaos in the brickyard cartoon Open data isn’t a new concept Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, September 16) "Chaos in the brickyard", Everything Hertz [Audio podcast] https://osf.io/xfd2p/Support Everything Hertz

We chat with Kristin Sainani (Stanford University) about a popular statistical method in sports medicine research (magnitude based inference), which has been banned by some journals, but continues to thrive in some pockets of scholarship. We also discuss the role of statistical inference in the current replication crisis. Links and info What is magnitude based inference and how did Kristin get involved in this? The response to Kristin’s critiques This is really an issue of small sample sizes Kristin’s Coursera course on scientific writing The readability of scientific articles is decreasing The role of statistical inference in the replication crisis Kristin has changed her mind about… Twitter Dan made international news by posting a picture of a bird on Twitter Kristin recommends this paper: P values are just the tip of the iceberg as well as the following books: Radium Girls and Bad Blood Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, September 2) "Shifting the goalposts in statistics (with Kristin Sainani)", Everything Hertz [Audio podcast], Retrieved from https://osf.io/3q25f/ Special Guest: Kristin Sainani.Support Everything Hertz

Dan and James discuss two listener questions on performing secondary data analysis and the potential for prestige to creep into open science reforms. More info and links: Why generate your own dataset when you can get a high impact paper using public data? Thanks to Stu Murray for the question Will people steal your ideas? The journal Scientific Data Are we now incentivising data mining rather than data collecting? Synthetic data Dan’s recent synthetic data preprint primer Ego and prestige got us into the mess we’re trying to fix with open science, but how can we stop this from happening again? Thanks to Robin Kok for the question, listen to our episode with him on e-health! Did all the people who co-authored the paper to change statistical significance the default p-value threshold to .005 actually do this in subsequent papers? Vagus nerve brain washing paper Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, August 19) "Mo data mo problems", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/TQ75JSupport Everything Hertz

We chat with Tom about whether psychology has a conflict-of-interest problem and how to best define such conflicts. Links and other stuff we cover... Tom's article on conflicts of interest in psychology How can we define a conflict an interest without falling down a rabbit hole? Communication statistics to the layperson How science journalism focuses on single studies rather than the larger story Tom’s new book: The AI does not hate you Win Tom’s book! Tweet your favourite Hertz episode and we’ll pick one at random, who'll get sent Tom's book How do journalists go about hearing from new voices for story comments? What has Tom changes his mind about? Tom’s book recommendation: Galileo's Middle Finger Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, August 5) "Conflicts of interest in psychology (with Tom Chivers)", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/F9WBMSpecial Guest: Tom Chivers.Support Everything Hertz

Dan and James apply the pomodoro principle by tackling four topics within a strict ten-minute time limit each: James' new error detection tool, academic dress codes, the "back in my day..." defence for QRPs, and p-slacking. Here are links and details... James won an award James’ new error detection tool, DEBIT Academic dress codes P-slacking The p-slacking paper Marcus Crede’s paper: A Negative Effect of a Contractive Pose Is Not Evidence for the Positive Effect of an Expansive Pose A preview of our next episode on conflicts of interest in psychology Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, July 15) "The pomodoro episode", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/VTDQ8Support Everything Hertz

We chat with Mike Morrison, a former User Experience (UX) designer who quit his tech career to research how we can bring UX design principles to science. We discuss Mike's recently introduced 'better poster' format and why scientists should think carefully about UX. Here's what we cover: What’s the story behind the “better poster?” The Better Poster video The Better Poster template The importance of minimising cognitive load Science isn’t badly designed, it’s not even designed at all What is good User Experience (UX)? The most important feature of SciHub Version 2 of the ‘better poster’ Weird poster designs that James has seen over the years The Fish Market study Common misunderstandings of the better poster Empirically investigating the performance of the Better Poster The meta-meta poster How better posters get better questions Mike’s next target: Better Presentations Andrew York’s Github paper A special give-away! Other links Mike on twitter [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, July 1) "Improving the scientific poster (with Mike Morrison)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/BNP7ESpecial Guest: Mike Morrison.Support Everything Hertz

Dan and James answer a listener question on whether they should stick it out for a few months in a toxic lab to get one more paper or if they should leave. Other stuff they cover: We don’t like cricket, oh no, we love it James is bad at tribalism We answer a listener question about a bad lab environment The “Dutch Bounce” The Golden Lab Child Demonstrating independence by writing a sole author paper What should you do if there’s a late authorship switch on your paper? Having an upfront conversation about authorship James on NPR Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, June 17) "Should I stay or should I go?", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/RX7FBSupport Everything Hertz

We chat with Kevin Mitchell (Trinity College Dublin) about what the field of psychology can learn from genetics research, how our research theories tend to be constrained by our research tools, and his new book, "Innate". Other stuff we cover: Kevin's book, "Innate: How the Wiring of Our Brains Shapes Who We Are" The story with link between Serotonin transporter gene 5-HTTLPR variation and it’s link to depression Gene wide association studies (GWAS) vs. candidate gene studies Correcting for multiple tests in GWAS What can psychology learn from genetics when it comes to improving methods? When did people lose faith in candidate gene studies? Common vs. rare gene mutations How our theories are shaped by our technology and resources The third overlooked area of genetic variation (other than nature and nurture): How the developmental program plays out Why intelligence is just an index of how well your brain is put together What happened to identifying new therapeutic targets via genetics? What Kevin has changed his mind about A few papers on mice behavior relevant to what Kevin mentioned, Matias et al (2017), Menegas et al (2018), Zalocusky et al (2016), Miyazaki et al (2018), Bercovici et al (2018), Orsini et al (2018) Robustness and Evolvability in Living Systems (Wagner, A) The Strategy of the Genes (Waddington, C) Cybernetics : Or the Control and Communication in the Animal and the Machine (Wiener, N) Other links [Dan on twitter](www.twitter.com/dsquintana) [James on twitter](www.twitter.com/jamesheathers) [Everything Hertz on twitter](www.twitter.com/hertzpodcast) [Everything Hertz on Facebook](www.facebook.com/everythinghertzpodcast/) Music credits: [Lee Rosevere](freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, June 3) "GWAS big teeth you have, grandmother (with Kevin Mitchell)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/RS4HXSpecial Guest: Kevin Mitchell.Support Everything Hertz