“Categorizing Selection Effects” by romeostevensit

[Author's note: LLMs were used to generate and sort many individual examples into their requisite categories, as well as find and summarize relevant papers, and extensive assistance with editing]

The earliest recording of a selection effect is likely the story of Diagoras regarding the "Votive Tablets." When shown paintings of sailors who had prayed to Poseidon and survived shipwrecks, implying that prayer saves lives, Diagoras asked: “Where are the pictures of those who prayed, and were then drowned?”

Selection effects are sometimes considered the most pernicious class of error in data science and policy-making because they do not merely obscure the truth, they often invert it. They create "phantom patterns" where the exact opposite of reality appears to be statistically significant.

Unlike measurement errors, which can often be averaged out with more data, selection effects are structural. Adding more data from a biased process only increases the statistical significance of your wrong conclusion.

The danger of selection effects lies in their ability to violate the fundamental assumption of almost all statistical intuition: randomness. When we look at a dataset, our brains (and standard statistical software) implicitly assume that what we are seeing is a fair representation of the world.

[...]

---

Outline:

(01:54) Instrumental Selection

(02:57) Ontological Selection

(04:35) Process Selection

(05:30) Agentic Selection

(06:51) Anthropic Selection

(08:05) Cybernetic Selection

(09:18) Some major papers in the field:

---

First published:
December 3rd, 2025

Source:
https://www.lesswrong.com/posts/9hA32vhSHYmvnDeuY/categorizing-selection-effects

---

Narrated by TYPE III AUDIO.