The Nonlinear Library: LessWrong LW - On Fables and Nuanced Charts by Niko McCarty
Sep 9, 2024
13:47
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On Fables and Nuanced Charts, published by Niko McCarty on September 9, 2024 on LessWrong.
Written by Spencer Greenberg & Amber Dawn Ace for Asimov Press.
In 1994, the U.S. Congress passed the largest crime bill in U.S. history, called the Violent Crime Control and Law Enforcement Act. The bill allocated billions of dollars to build more prisons and hire 100,000 new police officers, among other things. In the years following the bill's passage, violent crime rates in the U.S. dropped drastically, from around 750 offenses per 100,000 people in 1990 to under 400 in 2018.
But can we infer, as this chart seems to ask us to, that the bill caused the drop in crime?
As it turns out, this chart wasn't put together by sociologists or political scientists who've studied violent crime. Rather, we - a mathematician and a writer - devised it to make a point: Although charts seem to reflect reality, they often convey narratives that are misleading or entirely false.
Upon seeing that violent crime dipped after 1990, we looked up major events that happened right around that time - selecting one, the 1994 Crime Bill, and slapping it on the graph. There are other events we could have stuck on the graph just as easily that would likely have invited you to construct a completely different causal story. In other words, the bill and the data in the graph are real, but the story is manufactured.
Perhaps the 1994 Crime Bill really did cause the drop in violent crime, or perhaps the causality goes the other way: the spike in violent crime motivated politicians to pass the act in the first place. (Note that the act was passed slightly after the violent crime rate peaked!)
Charts are a concise way not only to show data but also to tell a story. Such stories, however, reflect the interpretations of a chart's creators and are often accepted by the viewer without skepticism. As
Noah Smith and many others have argued, charts contain hidden assumptions that can drastically change the story they tell.
This has important consequences for science, which, in its ideal form, attempts to report findings as objectively as possible. When a single chart can be the explanatory linchpin for years of scientific effort, unveiling a data visualization's hidden assumptions becomes an essential skill for determining what's really true.
As physicist Richard Feynman once said: In science, "the first principle is that you must not fool yourself, and you are the easiest person to fool."What we mean to say is - don't be fooled by charts.
Misleading Charts
Bad actors have long used data visualizations to deliberately manipulate and mislead. How to Lie with Statistics, a classic book from 1954, describes tricks that unscrupulous actors use to distort the truth without fabricating results, such as by truncating the y-axis of a chart to make an effect look much larger than it is or by cherry-picking data.
Drug companies and special interest groups have employed these techniques for decades to win public support. Merck, for example, was accused of publishing
misleading data about the anti-inflammatory drug Vioxx to hide the fact that it could cause heart attacks and strokes, ultimately resulting in a multi-billion dollar settlement.
But even when no one is intentionally trying to mislead or manipulate, charts designed to make information clear can still lead to erroneous conclusions. Just consider the U.S. maternal mortality statistics, which seem to show maternal deaths rising from 0.4 deaths per 100,000 women in 2003 to close to 1 per 100,000 in 2020.
This graph is worrisome, particularly if you or your partner is pregnant (or expect to be). Why are so many more expectant and new mothers dying? Is there some new danger? Is the healthcare system getting worse? Coverage in Scientific American, NPR, and elsewhere suggested t...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On Fables and Nuanced Charts, published by Niko McCarty on September 9, 2024 on LessWrong.
Written by Spencer Greenberg & Amber Dawn Ace for Asimov Press.
In 1994, the U.S. Congress passed the largest crime bill in U.S. history, called the Violent Crime Control and Law Enforcement Act. The bill allocated billions of dollars to build more prisons and hire 100,000 new police officers, among other things. In the years following the bill's passage, violent crime rates in the U.S. dropped drastically, from around 750 offenses per 100,000 people in 1990 to under 400 in 2018.
But can we infer, as this chart seems to ask us to, that the bill caused the drop in crime?
As it turns out, this chart wasn't put together by sociologists or political scientists who've studied violent crime. Rather, we - a mathematician and a writer - devised it to make a point: Although charts seem to reflect reality, they often convey narratives that are misleading or entirely false.
Upon seeing that violent crime dipped after 1990, we looked up major events that happened right around that time - selecting one, the 1994 Crime Bill, and slapping it on the graph. There are other events we could have stuck on the graph just as easily that would likely have invited you to construct a completely different causal story. In other words, the bill and the data in the graph are real, but the story is manufactured.
Perhaps the 1994 Crime Bill really did cause the drop in violent crime, or perhaps the causality goes the other way: the spike in violent crime motivated politicians to pass the act in the first place. (Note that the act was passed slightly after the violent crime rate peaked!)
Charts are a concise way not only to show data but also to tell a story. Such stories, however, reflect the interpretations of a chart's creators and are often accepted by the viewer without skepticism. As
Noah Smith and many others have argued, charts contain hidden assumptions that can drastically change the story they tell.
This has important consequences for science, which, in its ideal form, attempts to report findings as objectively as possible. When a single chart can be the explanatory linchpin for years of scientific effort, unveiling a data visualization's hidden assumptions becomes an essential skill for determining what's really true.
As physicist Richard Feynman once said: In science, "the first principle is that you must not fool yourself, and you are the easiest person to fool."What we mean to say is - don't be fooled by charts.
Misleading Charts
Bad actors have long used data visualizations to deliberately manipulate and mislead. How to Lie with Statistics, a classic book from 1954, describes tricks that unscrupulous actors use to distort the truth without fabricating results, such as by truncating the y-axis of a chart to make an effect look much larger than it is or by cherry-picking data.
Drug companies and special interest groups have employed these techniques for decades to win public support. Merck, for example, was accused of publishing
misleading data about the anti-inflammatory drug Vioxx to hide the fact that it could cause heart attacks and strokes, ultimately resulting in a multi-billion dollar settlement.
But even when no one is intentionally trying to mislead or manipulate, charts designed to make information clear can still lead to erroneous conclusions. Just consider the U.S. maternal mortality statistics, which seem to show maternal deaths rising from 0.4 deaths per 100,000 women in 2003 to close to 1 per 100,000 in 2020.
This graph is worrisome, particularly if you or your partner is pregnant (or expect to be). Why are so many more expectant and new mothers dying? Is there some new danger? Is the healthcare system getting worse? Coverage in Scientific American, NPR, and elsewhere suggested t...