Everything Hertz

P-values are universal, but do we really know what they mean? In this episode, Dan and James discuss a recent paper describing the failure to correctly interpret p-values in a sample of academic psychologists. Some of the topics discussed: Common p-value misconceptions James tests Dan on his p-value knowledge p-values vs. effect size The problem of sample size with p-value interpretation The Facebook mood manipulation study Data peeking Equivalent p-values do not represent equivalent results Meta-analytical thinking Using significance as a categorical factor Statistical vs. clinical significance Clinical trial registration and 'secondary outcome creep' Dan and James answer listener questions Science communicator vs. scientist Grant titles and the 'Pub test' NASA and social media Links The article http://journal.frontiersin.org/article/10.3389/fpsyg.2016.01247/full Geoff Cumming's book (we got the name completely wrong - sorry Geoff!) http://www.amazon.com/Understanding-The-New-Statistics-Meta-Analysis-ebook/dp/B007M9D76G/ref=pd_sim_kstore_1?ie=UTF8&refRID=1QWKES82EP85DBAEKNT1 The story on research passing the 'pub' test https://theconversation.com/if-youre-going-to-ridicule-research-do-your-homework-64238 Real scientists http://realscientists.org Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.comSupport Everything Hertz

Science funding has a series of built in incentive structures, but what sort of science does this produce? Some of the topics discussed: Feedback from our 'Public health and Pokemon' episode (#22) Incentive structures in science What we should be doing in science compared to what we are doing Quantity vs. Quality The analysis of Trump's tweets for negativity vs. positivity Pre-registration How much detail do you need to go into when it comes to pre-registering an analysis APS journal badges - they're working! Data sharing makes you more careful with your data Solutions to the incentive problem have to come from the policy level The grant funding lottery system proposal The PhD oversupply Gaming the system James wants to mandate science communication Dan wants to include replication studies in PhD programs Scientist names that suit their research area Links The article on incentive structures https://medium.com/the-spike/how-a-happy-moment-for-neuroscience-is-a-sad-moment-for-science-c4ba00336e9c#.x3sea13i1 The Allen brain atlas http://observatory.brain-map.org Analysis of Trump's tweets http://varianceexplained.org/r/trump-tweets/ Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

Can you be a "serious academic" while still posting photos on Instagram? In this episode, James and Dan discuss a recent article bemoaning the infiltration of the "selfie epidemic" into academia. Some of the topics discussed: James and Dan share their thoughts on the article The REAL 'c' word.... Social media and conferences Snapchat + academics = snapademics Using Instagram stories to share you research Why "PHD comics" is so successful Criticism in academia Listener question 1: What's your favourite part of research? Listener question 2: What's your favourite technique or experiment to perform? Listener question 3: What's a funny story from being an academic? Links The article https://www.theguardian.com/higher-education-network/2016/aug/05/im-a-serious-academic-not-a-professional-instagrammer A response to the article https://www.theguardian.com/science/brain-flapping/2016/aug/05/im-a-non-serious-academic-i-make-no-apologies-for-this Snapademics https://twitter.com/snapademia Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

Pokemon Go is sweeping the world and getting people walking again! But is the Pokemon Go 'model' a golden opportunity to tackle obesity or just another fad? Some of the topics discussed: James plays "Pokemon or Cholesterol medication?" Dan tries to explain Pokemon Go to James James' first contact with Pokemon Go "trainers" Should health interventions be modeled on Pokemon Go? Other exercise augmented reality health apps What's the app's endgame? Can health authorities copy this model? We make a correction from episode 17: PLoS is in fact a non-profit journal, not a for-profit journal Dan and James answer two listener questions: i) The dumbest things they've ever done in the lab (both related to email faux pas) ii) How often should lab meetings be run The importance of PROPERLY piloting your experiment If you don't know the person in the meeting that takes up too much time, it's probably you. Links The quiz http://www.slate.com/articles/technology/gaming/2016/07/pokemon_or_cholesterol_medication_a_quiz.html Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

It's well established that steroid use is associated with many adverse healthy outcomes, but what does it actually do to your brain? Dan and James discuss an interesting new paper that compared brain structure in long-term steroid users and non-using weightlifters. Some of the topics discussed: A summary of the study How are steroids typically used? What are the differences in use between sports? The recruitment of 'real' users James gives Dan a surprise Norwegian test (he doesn't do too well) The things Dan and James liked about the study (hint: many things) Steroid use in women Dose-dependent effects of steroids Folk beliefs surrounding steroid use James' goal of making his cat as jacked as possible If you have a great study, there's no need to oversell James' experience of participating in a growth hormone trial Links The paper http://www.sciencedirect.com/science/article/pii/S000632231632529X Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

Can psychologists learn more by studying fewer people? Some of the topics discussed: Brexit and science Can the UK take the 'Norway' option? Horizon 2020 The impact on personnel and research training Italian coffee Listener feedback We're sorry for the chewing sound from episode 17! Intraindividual replication vs. larger sample sizes What sort of experiments are better suited to detailed within-subject studies? Is 'quantified self' data more valid than experimental data? What if you happen to recruit a 'weird' person? Links The paper http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4911349/ Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

Dan and James discuss a recent paper on intranasal oxytocin and spirituality Some of the topics discussed: A summary of a recent paper on oxytocin and spirituality Why within-subject designs are a better choice for oxytocin research The physiology of nasal administration How do you control for differences in nasal environment Hypothesis-driven vs. exploratory research Oxytocin pathway gene ANCOVA and Lord's paradox (yep, it's called that) Dan applauding the authors for posting ALL their data online James disagrees with Dan on approaches to pre-registering studies James promises never to chew during a recording (sorry!) Producing research vs. actually reading other people's research Our first citation (sort of) How the replication crisis is the fact that we're not doing any Heart rate variability in oysters (and other animals) Links The paper http://scan.oxfordjournals.org/content/early/2016/06/22/scan.nsw078 Dan's blog post on intranasal oxytocin administration considerations https://medium.com/@dsquintana/intranasal-drug-administration-in-psychiatry-80d076f1abdd#.dsdrrohdu Dan's paper on the promise and pitfalls of intranasal administration http://www.ncbi.nlm.nih.gov/pubmed/26552590 The data from the paper https://osf.io/mjhzw/ Our first (blog post) citation https://rhythmicpsychology.wordpress.com/2016/06/22/shock-shock-horror-horror/ Can we replicate the replication crisis? https://theconversation.com/is-psychology-really-in-crisis-60869 Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

Withholding data: bad science or scientific misconduct? Some of the topics discussed: Dan raises privacy issues surrounding sharing data What are the limits of AI to identify people from 'un-identifiable' data? The new age of sharing data What grinds Dan's gears Requesting data from people who said that they'd actually share their data James' experiences with requesting data Dan offers a solution for accessing data via an independent 3rd party When is a data request 'vexatious'? Hiding data vs. just being lazy with data management The importance of sharing your analysis code along with the data Version controlling manuscripts How much will open data actually improve science? 'Fake' journals masquerading as real ones Australia's windfarm commissioner (a real thing, unfortunalty) Australian coffee vs. the rest of the world Our Twitter followers numbers have surpassed our Facebook fans! Links The paper on requesting data http://www.ncbi.nlm.nih.gov/pubmed/22686633 The blog post on whether withholding your data is scientific misconduct http://blogs.lse.ac.uk/impactofsocialsciences/2015/07/03/data-secrecy-bad-science-or-scientific-misconduct/ Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

Do we really need scientific journals? Some of the topics discussed: James trolling predatory journals with jibberish papers on the 'DONG' effect How do these spammy journal invitation emails actually work? Formal journals vs. preprint servers The shift to preprints in psychology Why do some journals forbid preprints? An article defending the big publishers How much does it really cost to have an online journal? What if the public funders were to set up thier own journals? What's the role of post-pubication review? Do journals provide added value? James' brush with the 'established' media Links Publisher policies on preprints https://en.wikipedia.org/wiki/List_of_academic_journals_by_preprint_policy How much do universities pay for journal subscriptions? https://gowers.wordpress.com/2014/04/24/elsevier-journals-some-facts/ Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz

What are the defining characteristics of a good psychology study? We received this excellent question from a listener and decided to do a whole episode on this idea. Some of the topics discussed: When’s the last time you saw a psych study that only reported a t-test? Dan and James’ new paper on worry and heart rate variability Skepticism towards studies with many variables and ‘novel’ statistical approaches (that tend to always provide marvellous results) Repeated measures ANOVAs vs. linear mixed models Publishing convenient ideas, even if they’re wrong Fishing expeditions The ‘nocebo’ effect in action What are markers of study quality? Pre-registering studies - can it be gamed? The gradual improvement of psychology studies Links Dan and James' new paper on worry and heart rate variability http://www.ncbi.nlm.nih.gov/pubmed/27255891 The paper that said "Encouraging experimental psychologists to use LMMs was like giving shotguns to toddlers.” http://www.ncbi.nlm.nih.gov/pubmed/25657634 Facebook page https://www.facebook.com/everythinghertzpodcast/ Twitter account https://www.twitter.com/hertzpodcastSupport Everything Hertz