How Science Works: Meta-Research, Publishing, Reproducibility, Peer Review, Funding | John Ioannidis

Short Summary: A rare, insider’s look at the messy realities of scientific research with Stanford’s Dr. John Ioannidis. The good, the bad, and the ugly about how scientific research actually works.

About the guest: John Ioannidis, MD, PhD is a professor at Stanford University in medicine, epidemiology, population health, and biomedical data science, with an MD from the University of Athens and a PhD from Harvard in biostatistics. He directs the Meta-Research Innovation Center at Stanford (METRICS), focusing on improving research methods and practices. Renowned for his paper “Why Most Published Research Findings Are False,” he’s among the most cited scientists globally, tackling biases and reproducibility in science.

Note: Podcast episodes are fully available to paid subscribers on the M&M Substack and everyone on YouTube. Partial versions are available elsewhere. Full transcript and other information on Substack.

Key Takeaways:

Science’s “replication crisis” isn’t new—it’s baked into how tough and bias-prone research is, hitting all fields, not just “soft” ones like psychology.

Ioannidis’s famous claim, “most published findings are false,” holds up: stats show many “significant” results are flukes due to weak studies or bias.

Peer review’s a mixed bag—only a third of papers improve, and unpaid, tired reviewers miss a lot, letting shaky stuff slip through.

Publishing’s a $30 billion game with 50,000+ journals; big players like Elsevier rake in huge profits from subscriptions and fees, often over $10,000 per paper.

Researchers game the system—think fake co-authorships or citation cartels—boosting metrics like the H-index, which tracks papers with matching citation counts.

Ioannidis’s early COVID-19 fatality rate (0.2-0.3%) was spot-on but sparked a firestorm as politics warped science into “clan warfare.”

NIH funding’s clogged by red tape and favors older researchers, starving young innovators and risky ideas that could shake things up.

He’s building tools like a public database of scientist stats (4 million downloads!) to spotlight gaming and push for transparent, fair research.

*Not medical advice.

00:00:00 Intro
00:06:21 The Replication Crisis Explained
00:13:12 Replication in Science: How Much and Why?
00:18:14 Why Most Published Research Findings Are False
00:25:13 Peer Review: Strengths and Weaknesses
00:33:07 The Explosion of Journals and Predatory Publishing
00:41:40 The Business of Scientific Publishing
00:48:45 Open Access Costs and the Funding Dilemma
00:57:00 Preprints & Potential Solutions
01:04:34 Gaming the System: Metrics and Misconduct
01:11:08 COVID-19 & Politicization of Science
01:18:31 Revisiting the Infection Fatality Rate
01:25:48 NIH Funding & Leadership Changes
01:32:13 Directs vs. Indirects in Research Grants
01:40:56 Hopes for NIH Reform with Jay Bhattacharya
01:46:37 Current Projects & Closing Thoughts