Are You Getting Realistic About What We Really Know About Effect Sizes?

i think what we need to be doing is being realistic about what we actually know here and the size of effects we're likely to expect. I see this as an evolution in the conversation. And like you said, it's not to say there shouldn't be any sort of debate or conversation or shift. But i'm saying that we need to first understand what a realistic effects is for some of these interventions. And secondly, can we get a better understanding of context and how context effects results?

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

In today's conversation, I am joined by Michael Hallsworth, managing director for the Americas division of the Behavioral Insights Team, or BIT. Michael was on the show nearly two years ago in episode 125 when he was sharing about his book which is aptly named Behavioral Insights.

The reason he is here today is to talk about a debate that has been going on across the behavioral science community for the bulk of this year: "Do nudges work?"

Michael wrote an article recently in Behavioral Scientist which laid out all the points in a very clear way and talked about the real question we should be asking (as well as the next steps for the field in the future). I knew he was the perfect guest to come on and speak about this. He does a great job summarizing everything here and I hope you get value from this conversation. It can be hard to look at ourselves, the fields we are in, or ourselves personally, and embrace opportunities for improvement. But, it is so necessary in order to grow, change and thrive. The field is built on solid science, and we have an opportunity to do even more going forward. Listen in to hear all of our thoughts on this hot topic.

Show Notes:

[00:42] In today's conversation, I am joined by Dr. Michael Hallsworth, managing director for the Americas division of the Behavioral Insights Team. He was on the show nearly two years ago.
[02:37] It can be hard to look at ourselves, the fields we are in, or ourselves personally, and embrace opportunities for improvement. But, it is so necessary in order to grow, change and thrive.
[05:00] Michael shares about himself and his background in behavioral science.
[06:56] Nudges guide people to decide while maintaining their freedom of choice.
[08:26] Publication bias is a problem that affects many disciplines (not just behavioral science) in the scientific literature.
[11:19] Singular data points are not generalizable in other contexts. Context and testing are key.
[13:22] First we need to understand what is a realistic effect size for some of these interventions and can we get a better understanding of how context affects results.
[15:42] A missing piece of this debate around if nudges do or don’t work is looking at some work that is not affected by publication bias.
[16:59] We do have evidence for the real-world effects of nudging that are not affected by publication bias. Those effects are smaller than the ones in the original study but they are still meaningful.
[19:06] Human behavior is complex. Results vary by context and group.
[21:21] There are factors going into a result that are meaningful which may mean that something doesn’t work in a different situation.
[22:56] Instead of making overall claims we should be talking about some of these ideas a bit like scientists have talked about incentives.
[23:53] We don’t need to oversimplify or oversell because the results are there but they vary in ways we don’t understand (yet).
[25:34] Moving forward we can run multi-size studies so we can explore these differences more systematically.
[27:42] When you take your results together, you should be able to see which idea is more supported.
[29:49] They found that if you thought something was more context-dependent those studies were less likely to replicate success.
[31:35] It’s not about what you know. It is about how you match it to context to produce a result.
[33:56] Behavioral Science “in the wild” is different from behavioral science in the lab.
[35:07] Within organizations it can be really hard to start tracing the threads between studies and looking into the connections.
[37:11] Michael shares about his upcoming manifesto.
[39:20] The overriding question he discusses in the manifesto is “How do we help behavioral science tackle some of those bigger questions?”
[40:14] “Do nudges work?” is the wrong debate. There are ways we can take behavioral science forward.
[42:02] Anything that has happened so far isn’t bad or wrong. We are learning from what has happened and now we are moving forward.
[44:47] You may not be able to predict in advance all the potential outcomes, particularly if you are intervening in a complex environment with lots of things going on (i.e., the real world with human people).
[46:11] Melina shares her closing thoughts.
[47:21] The summary of what you heard from Michael is that, yes, nudges do work. And, as I have always said on this show, nothing is perfectly generalizable.
[48:44] This idea of looking into the possibilities of more complex systems and being able to be even better at predicting what will happen when and why is fascinating, and something I look forward to being part of researching into the future.

Thanks for listening. Don’t forget to subscribe on Apple Podcasts or Android. If you like what you heard, please leave a review on iTunes and share what you liked about the show.

I hope you love everything recommended via The Brainy Business! Everything was independently reviewed and selected by me, Melina Palmer. So you know, as an Amazon Associate I earn from qualifying purchases. That means if you decide to shop from the links on this page (via Amazon or others), The Brainy Business may collect a share of sales or other compensation.

Let’s connect:

Melina@TheBrainyBusiness.com
The Brainy Business® on Facebook
The Brainy Business on Twitter
The Brainy Business on Instagram
The Brainy Business on LinkedIn
Melina on LinkedIn
The Brainy Business on Youtube
Join the BE Thoughtful Revolution – our free behavioral economics community, and keep the conversation going!

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!

Get the app

Home Top podcasts Popular guests Top books