Metascience 101 - EP5: "How and Why to Run an Experiment"

IN THIS EPISODE: Professor Heidi Williams, Professor Paul Niehaus, Emily Oehlsen, and Jim Savage dive in on a practical “how-to” for experimentation and evaluation in metascience. They discuss how to keep metascience experimentation and evaluation relevant to policymakers.

“Metascience 101” is a nine-episode set of interviews that doubles as a crash course in the debates, issues, and ideas driving the modern metascience movement. We investigate why building a genuine “science of science” matters, and how research in metascience is translating into real-world policy changes. 

Chapters

(00:00:00) Introduction

(00:00:37) Research on research

(00:03:34) Identifying talent, RISE program

(00:14:44) Impact evaluation, GiveDirectly

(00:17:57) Decision making with impact evaluation

(00:31:45) Replications

(00:35:11) Existential vs. operational experiments and evaluation

(00:42:21) Outcome measurability: benchmarks, surrogates and new evidence

(00:52:01) GiveDirectly as example of evidence-based approach

(00:55:01) Government evidence-based as a way