#180 – Hugo Mercier on why gullibility and misinformation are overrated
Feb 21, 2024
auto_awesome
Cognitive scientist Hugo Mercier challenges the view of gullibility and misinformation, highlighting the perceptiveness of ordinary people. They explore how reasoning mechanisms facilitate beneficial communication and debunk myths about misinformation. Discussing AI's potential impact, they navigate belief formation, trust, and the dynamics of misinformation. The conversation delves into vaccine hesitancy, financial scams, astrology beliefs, and the challenges of discerning truth in an AI-influenced information environment.
People assess trust based on competence and trustworthiness, not blind gullibility.
Reflective beliefs without behavioral implications are often less consequential than intuitive beliefs.
Social influences and cognitive biases perpetuate participation in high-risk financial ventures.
Stock market predictions are challenging, with randomness being a significant factor.
Consumption of fake news tends to attract individuals already aligned with the content.
Deep dives
Trust based on competence and trustworthiness
People determine who to trust based on two main dimensions: competence and trustworthiness. Competence involves assessing the person's knowledge and expertise to know if what they say is likely true. Trustworthiness is about whether the person will share accurate beliefs and if their incentives align with yours. People often judge trustworthiness by short-term and long-term incentives of the individual.
Reflective Beliefs vs. Intuitive Beliefs
Reflective beliefs, not directly linked to behavior, are often less consequential than intuitive beliefs formed through perception and experience. People may claim reflective beliefs without behavioral implications, such as conspiracy theories. Reflective beliefs are usually not as costly for individuals and may serve social or personal goals, even if they are false.
Milgram Experiments Misconceptions
The Milgram experiments, often cited for obedience to authority, had participants who were skeptical and discussions were crucial in getting compliance. Not all participants obeyed completely, and doubts decreased compliance rates. The authority's prestige and setting played a significant role in the participant's decisions to follow instructions.
Attraction to High-Risk Financial Ventures
Many individuals are attracted to risky financial ventures like day trading and multi-level marketing scams, even though these activities often lead to financial losses. Day trading, where amateurs trade shares frequently, typically results in significant financial losses for the majority, despite the allure of quick profits. In contrast, multi-level marketing schemes promise earnings through product sales and recruitment, but statistically, only the first few participants profit. Despite the availability of information highlighting the risks and inefficiencies of these ventures through research, individuals continue to engage in them, a perplexing phenomenon.
Psychological Factors in Market Participation
The attraction to activities like day trading and multi-level marketing scams may stem from psychological factors such as the thrill of gambling or the desire for quick financial gains. Individuals engaging in speculative trading behaviors might be drawn to the excitement and potential rewards, overlooking the statistical evidence suggesting high failure rates. Additionally, the allure of joining a potentially profitable scheme, even if the probability of success is low and information on the risks is available, may indicate a psychological bias towards optimism and the hope for success despite unfavorable odds.
Influence of Social Factors and Cognitive Biases
Social influences and cognitive biases play a role in perpetuating participation in high-risk financial ventures like day trading and multi-level marketing. Stories of successful traders or scheme participants, selectively reported in media, may create an illusion of widespread profitability and success. Moreover, cognitive biases related to overconfidence or the tendency to follow the crowd can lead individuals to dismiss research findings on the inefficiency and risks of these activities, reinforcing their belief in potential gains.
The Challenge of Predicting Stock Market Movements
It is common for individuals to believe that patterns exist in stock market movements and that these patterns can be exploited for financial gain. However, the podcast explains that predicting stock market behavior is not as straightforward as many think. Despite some professionals and algorithm writers finding some predictability, most movements are considered random. The discussion emphasizes that while some people, especially experts, may profit from understanding complex finance dynamics, the average individual may be misled into thinking they can predict market outcomes reliably.
Impact of Misinformation and Fake News Consumption
The podcast highlights that the consumption of fake news and misinformation, particularly in the context of social media, tends to attract individuals who are already aligned with the content. Studies have shown that those consuming fake news are often politically extreme and use such information more for entertainment rather than being significantly influenced by it. This underscores how the demand for information largely shapes the informational environment, impacting the spread and reception of news, especially in democratic settings.
Impact of Misinformation
The podcast delves into the impact of misinformation on the spread of reliable information. It discusses the potential competition between engaging content created by resourced institutions versus less resourced groups. The conversation highlights the importance of trusting reputable sources to prevent the spread of misinformation and create a society where truth and facts prevail.
Future Role of AI in Information Environment
The episode explores potential ways that AI, specifically LLMs, could improve truth-seeking and combat misinformation. Ideas like individualized fact-checking on social media posts, information bots countering misinformation, and LLMs identifying misleading content are discussed. The conversation emphasizes the need for proactive measures to address the challenges posed by AI advancements in the information landscape to ensure the credibility and reliability of information sources.
And the discussion around misinformation and disinformation has shifted to focus on how generative AI or a future super-persuasive AI might change the game and make it extremely hard to figure out what was going on in the world — or alternatively, extremely easy to mislead people into believing convenient lies.
But this week’s guest, cognitive scientist Hugo Mercier, has a very different view on how people form beliefs and figure out who to trust — one in which misinformation really is barely a problem today, and is unlikely to be a problem anytime soon. As he explains in his book Not Born Yesterday, Hugo believes we seriously underrate the perceptiveness and judgement of ordinary people.
In this interview, host Rob Wiblin and Hugo discuss:
How our reasoning mechanisms evolved to facilitate beneficial communication, not blind gullibility.
How Hugo makes sense of our apparent gullibility in many cases — like falling for financial scams, astrology, or bogus medical treatments, and voting for policies that aren’t actually beneficial for us.
Rob and Hugo’s ideas about whether AI might make misinformation radically worse, and which mass persuasion approaches we should be most worried about.
Why Hugo thinks our intuitions about who to trust are generally quite sound, even in today’s complex information environment.
The distinction between intuitive beliefs that guide our actions versus reflective beliefs that don’t.
Why fake news and conspiracy theories actually have less impact than most people assume.
False beliefs that have persisted across cultures and generations — like bloodletting and vaccine hesitancy — and theories about why.
And plenty more.
Chapters:
The view that humans are really gullible (00:04:26)
The evolutionary argument against humans being gullible (00:07:46)
Open vigilance (00:18:56)
Intuitive and reflective beliefs (00:32:25)
How people decide who to trust (00:41:15)
Redefining beliefs (00:51:57)
Bloodletting (01:00:38)
Vaccine hesitancy and creationism (01:06:38)
False beliefs without skin in the game (01:12:36)
One consistent weakness in human judgement (01:22:57)
Trying to explain harmful financial decisions (01:27:15)
Astrology (01:40:40)
Medical treatments that don’t work (01:45:47)
Generative AI, LLMs, and persuasion (01:54:50)
Ways AI could improve the information environment (02:29:59)
Producer and editor: Keiran Harris Audio Engineering Lead: Ben Cordell Technical editing: Simon Monsour and Milo McGuire Transcriptions: Katy Moore
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
