I Think Scott's Being Too Easy on Himself Here, but It's Not Ok to Get So Many Relevant Facts Wrong

https://astralcodexten.substack.com/p/highlights-from-the-comments-on-aducanumab

These are highlights from the comments of Adumbrations Of Aducanumab, Details Of The Infant Fish Oil Story, and discussion of those posts elsewhere.

C_B writes:

I agree with this post's overall point that the FDA is not, on average, too lax, and that the Atlantic article's take that the aducanumab approval is a sign of them being too lax is a bad take.

That said, I think the beginning of this article really undersells how uniquely bad the aducanumab approval is. It's not just "pretty unclear whether it actually treats Alzheimers." Nobody in the field thinks there's any serious possibility that it treats Alzheimers.

- Here's Derek Lowe talking about it: https://blogs.sciencemag.org/pipeline/archives/2021/06/08/the-aducanumab-approval

- The FDA's advisory committee doesn't think it treats Alzheimers: https://alzheimersnewstoday.com/2020/11/11/fda-committee-votes-aducanumab-trial-data-fail-support-alzhimers-treatment-benefit/

- The trial was halted for futility: https://www.reuters.com/article/us-biogen-alzheimers/biogen-eisai-scrap-alzheimer-drug-trials-idUSKCN1R213G

- The details of the "positive results" are textbook p-hacking of exactly the sort that the whole replication crisis has been about. It's a post-hoc subgroup analysis where the subgroup was selected based on similarity to the patients who had the most positive results; i.e., trivially guaranteed to show "positive" results via group selection. You can read more details in the statistical reviewer's comments in the advisory committee's document (PDF, starting on p. 174): https://www.fda.gov/media/143502/download