The Nonlinear Library EA - Extraordinary cost-effectiveness analyses call for theoretical buttressing by Seth Ariel Green
Jun 25, 2024
Writer and Rationalist in the EA community, Seth Ariel Green, discusses the importance of theoretical foundations in cost-effectiveness analyses. He delves into the comparative cost-effectiveness of Pure Earth and GiveDirectly, highlighting the need for a compelling theoretical basis behind extraordinary results in effective altruism.
03:31
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Extraordinary cost-effectiveness analyses call for theoretical buttressing, published by Seth Ariel Green on June 25, 2024 on The Effective Altruism Forum.
What needs to be true for an estimate to be reasonable?
Pure Earth is a GiveWell grantee that works to reduce lead and mercury exposure. In an August 2023 post, they provided a "preliminary analysis" suggesting that their lead reduction program in Bangladesh "can avert an equivalent DALY for just under $1." By contrast, they estimate that GiveDirectly "has a cost-effectiveness of approximately $836 per DALY-equivalent averted."
About 86% of the money GiveDirectly spends goes directly to recipients, so an $836 donation to GiveDirectly results in about~$719 going directly to a very poor person. In effect, Pure Earth is claiming that either GiveDirectly can give a person in poverty $719, or Pure Earth can spend $1 helping people in Bangladesh, and these would be about equally good for human welfare.
Pure Earth calls this an "extraordinary result" and forthrightly identifies places where their analysis might go wrong. But what I'm missing is a sense of why their analysis might be right -- a story about why this extraordinary opportunity exists. In other words, theory.
I find this to be a pretty common lacuna in effective altruist CEAs.[1] Our explanations tend to be technical but not contextual. They tend to focus on our assumptions about the world at large but lack granularity about social and political conditions. But for my tastes, any claim that that boils down to 'we can help poor people more effectively than they can help themselves' requires theoretical buttressing in the form of a plausible story.
What kinds of theories might suffice?
In the case of lead reduction in spices, six come to mind.
1. Poor people are generally bad at managing their own affairs and need external guidance. (I do not find this very plausible.)
2. There is extraordinary political gridlock that an external organization is especially well-suited to solving. (Pure Earth says that the "project's impact lies not in identifying and enforcing food safety regulations, but rather in expediting its implementation by several years.")
3. There is a narrow opportunity for positive impact stemming from deep insight about a particular context.
4. There is a market failure, e.g. a collective action problem, where a push from an NGO can create a self-sustaining equilibrium.
5. There is a cognitive bias and/or cultural failure that leads people to undervalue something that is good, or overvalue something that is bad, and they need a push in the right direction.
6. Unconditional cash transfers engender negative spillovers, whereas public health interventions typically have positive spillovers.
The story will vary from case to case. For New Incentives, I assume it's some version of theories 4, 5, and 6. For anti-malarial interventions, I am not sure (and am generally a skeptic). For Pure Earth, I am also not sure, because I am not an expert. As a potential donor, I am looking for explanation about aspects of the world that I don't understand. I can potentially back that story out of a spreadsheet. But I'd prefer to hear it directly.
1. ^
I use Pure Earth to illustrate because their CEA was on the far tail of the distribution, so it stuck in my memory, but this point could be made about many cost-effective analyses posted the forum. As far as I can tell, Pure Earth is doing great work on an important, neglected, and tractable issue.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
What needs to be true for an estimate to be reasonable?
Pure Earth is a GiveWell grantee that works to reduce lead and mercury exposure. In an August 2023 post, they provided a "preliminary analysis" suggesting that their lead reduction program in Bangladesh "can avert an equivalent DALY for just under $1." By contrast, they estimate that GiveDirectly "has a cost-effectiveness of approximately $836 per DALY-equivalent averted."
About 86% of the money GiveDirectly spends goes directly to recipients, so an $836 donation to GiveDirectly results in about~$719 going directly to a very poor person. In effect, Pure Earth is claiming that either GiveDirectly can give a person in poverty $719, or Pure Earth can spend $1 helping people in Bangladesh, and these would be about equally good for human welfare.
Pure Earth calls this an "extraordinary result" and forthrightly identifies places where their analysis might go wrong. But what I'm missing is a sense of why their analysis might be right -- a story about why this extraordinary opportunity exists. In other words, theory.
I find this to be a pretty common lacuna in effective altruist CEAs.[1] Our explanations tend to be technical but not contextual. They tend to focus on our assumptions about the world at large but lack granularity about social and political conditions. But for my tastes, any claim that that boils down to 'we can help poor people more effectively than they can help themselves' requires theoretical buttressing in the form of a plausible story.
What kinds of theories might suffice?
In the case of lead reduction in spices, six come to mind.
1. Poor people are generally bad at managing their own affairs and need external guidance. (I do not find this very plausible.)
2. There is extraordinary political gridlock that an external organization is especially well-suited to solving. (Pure Earth says that the "project's impact lies not in identifying and enforcing food safety regulations, but rather in expediting its implementation by several years.")
3. There is a narrow opportunity for positive impact stemming from deep insight about a particular context.
4. There is a market failure, e.g. a collective action problem, where a push from an NGO can create a self-sustaining equilibrium.
5. There is a cognitive bias and/or cultural failure that leads people to undervalue something that is good, or overvalue something that is bad, and they need a push in the right direction.
6. Unconditional cash transfers engender negative spillovers, whereas public health interventions typically have positive spillovers.
The story will vary from case to case. For New Incentives, I assume it's some version of theories 4, 5, and 6. For anti-malarial interventions, I am not sure (and am generally a skeptic). For Pure Earth, I am also not sure, because I am not an expert. As a potential donor, I am looking for explanation about aspects of the world that I don't understand. I can potentially back that story out of a spreadsheet. But I'd prefer to hear it directly.
1. ^
I use Pure Earth to illustrate because their CEA was on the far tail of the distribution, so it stuck in my memory, but this point could be made about many cost-effective analyses posted the forum. As far as I can tell, Pure Earth is doing great work on an important, neglected, and tractable issue.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org