118 — Science and Decision Making under Uncertainty, A Conversation with Prof. John Ioannidis

In this episode, I had the privilege of speaking with John Ioannidis, a renowned scientist and meta-researcher whose groundbreaking work has shaped our understanding of scientific reliability and its societal implications. We dive into his influential 2005 paper, Why Most Published Research Findings Are False, explore the evolution of scientific challenges over the past two decades, and reflect on how science intersects with policy and public trust—especially in times of crisis like COVID-19.

John had and has major impacts in our understanding of medical research, research quality in general, public health and how to handle critical situations under limited knowledge. His work was and is highly influential and extraordinarily important for our understanding where medical science, but really, science in general is standing, how we dealt with the Covid crisis, and how we could get our act together again.
He is professor of medicine at Stanford, expert in epidemiology, population health and biomedical data science.

We begin with John taking us back to 2005, when he published his paper in PLOS Medicine. He explains how it emerged from decades of empirical evidence on biases and false positives in research, considering factors like study size, statistical power, and competition that can distort findings, and why building on shaky foundations wastes time and resources.

“It was one effort to try to put together some possibilities, of calculating what are the chances that once we think we have come up with a scientific discovery with some statistical inference suggesting that we have a statistically significant result, how likely is that not to be so?”

I propose a distinction between “honest” and “dishonest” scientific failures, and John refines this. What does failure really mean, and how can they be categorised?

The discussion turns to the rise of fraud, with John revealing a startling shift: while fraud once required artistry, today’s “paper mills” churn out fake studies at scale. We touch on cases like Jan-Hendrik Schön, who published prolifically in top journals before being exposed, and how modern hyper-productivity, such as a paper every five days, raises red flags yet often goes unchecked.

“Perhaps an estimate for what is going on now is that it accounts for about 10%, not just 1%, because we have new ways of massive… outright fraud.”

This leads to a broader question about science’s efficiency. When we observe scientific output—papers, funding—grows exponentially but does breakthroughs lag? John is cautiously optimistic, acknowledging progress, but agrees efficiency isn’t what it could be. We reference Max Perutz’s recipe for success:

“No politics, no committees, no reports, no referees, no interviews; just gifted, highly motivated people, picked by a few men of good judgement.”

Could this be replicated in today's world or are we stuck in red tape?

“It is true that the progress is not proportional to the massive increase in some of the other numbers.”

We then pivot to nutrition, a field John describes as “messy.” How is it possible that with millions of papers, results are mosty based on shaky correlations rather than solid causal evidence? What are the reasons for this situation and what consequences does it have, e.g. in people trusting scientific results?

“Most of these recommendations are built on thin air. They have no solid science behind them.”

The pandemic looms large next. In 2020 Nassim Taleb and John Ioannidis had a dispute about the measures to be taken. What happened in March 2020 and onwards? Did we as society show paranoid overreactions, fuelled by clueless editorials and media hype?

“I gave interviews where I said, that’s fine. We don’t know what we’re facing with. It is okay to start with some very aggressive measures, but what we need is reliable evidence to be obtained as quickly as possible.”

Was the medicine, metaphorically speaking, worse than the disease? How can society balance worst-case scenarios without paralysis.

“We managed to kill far more by doing what we did.”

Who is framing the public narrative of complex questions like climate change or a pandemic? Is it really science driven, based on the best knowledge we have? In recent years influential scientific magazines publish articles by staff writers that have a high impact on the public perception, but are not necessarily well grounded:

“They know everything before we know anything.”

The conversation grows personal as John shares the toll of the COVID era—death threats to him and his family—and mourns the loss of civil debate. He’d rather hear from critics than echo chambers, but the partisan “war” mindset drowned out reason. Can science recover its humility and openness?

“I think very little of that happened. There was no willingness to see opponents as anything but enemies in a war.”

Inspired by Gerd Gigerenzer, who will be a guest in this show very soon, we close on the pitfalls of hyper-complex models in science and policy. How can we handle decision making under radical uncertainty? Which type of models help, which can lead us astray?

“I’m worried that complexity sometimes could be an alibi for confusion.”

This conversation left me both inspired and unsettled. John’s clarity on science’s flaws, paired with his hope for reform, offers a roadmap, but the stakes are high. From nutrition to pandemics, shaky science shapes our lives, and rebuilding trust demands we embrace uncertainty, not dogma. His call for dialogue over destruction is a plea we should not ignore.

Other Episodes

• Episode 116: Science and Politics, A Conversation with Prof. Jessica Weinkle
• Episode 112: Nullius in Verba — oder: Der Müll der Wissenschaft
• Episode 109: Was ist Komplexität? Ein Gespräch mit Dr. Marco Wehr
• Episode 107: How to Organise Complex Societies? A Conversation with Johan Norberg
• Episode 106: Wissenschaft als Ersatzreligion? Ein Gespräch mit  Manfred Glauninger
• Episode 103: Schwarze Schwäne in Extremistan; die Welt des Nassim Taleb, ein Gespräch mit Ralph Zlabinger
• Episode 94: Systemisches Denken und gesellschaftliche Verwundbarkeit, ein Gespräch mit Herbert Saurugg
• Episode 92: Wissen und Expertise Teil 2
• Episode 90: Unintended Consequences (Unerwartete Folgen)
• Episode 86: Climate Uncertainty and Risk, a conversation with Dr. Judith Curry
• Episode 67: Wissenschaft, Hype und Realität — ein Gespräch mit Stephan Schleim

References

• Prof. John Ioannidis at Stanford University 
• John P. A. Ioannidis, Why Most Published Research Findings Are False, PLOS Medicine (2005)
• John Ioannidis, A fiasco in the making? As the coronavirus pandemic takes hold, weare making decisions without reliable data (2020)
• John Ioannidis, The scientists who publish a paper every five days, Nature Comment (2018)
• Hanae Armitage, 5 Questions: John Ioannidis calls for more rigorous nutrition research (2018)
• John Ioannidis, How the Pandemic Is Changing Scientific Norms, Tablet Magazine (2021)
• John Ioannidis et al, Uncertainty and Inconsistency of COVID-19 Non-Pharmaceutical1Intervention Effects with Multiple Competitive Statistical Models (2025)
• John Ioannidis et al, Forecasting for COVID-19 has failed (2022)
• Gerd Gigerenzer, Transparent modeling of influenza incidence: Big data or asingle data point from psychological theory? (2022)
• Sabine Kleinert, Richard Horton, How should medical science change? Lancet Comment (2014)
• Max Perutz quotation taken from Geoffrey West, Scale, Weidenfeld & Nicolson (2017)
• John Ioannidis: Das Gewissen der Wissenschaft, Ö1 Dimensionen (2024)