The Nonlinear Library EA - Development RCTs Are Good Actually by ozymandias
Mar 28, 2024
16:25
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Development RCTs Are Good Actually, published by ozymandias on March 28, 2024 on The Effective Altruism Forum.
This post was cross-posted from the substack Thing of Things with the permission of the author.
In defense of trying things out
The Economist recently published
an article, "How poor Kenyans became economists' guinea pigs," which critiques development economists' use of randomized controlled trials. I think it exemplifies the profoundly weird way people think about experiments.
The article says:
In 2018, an RCT run by two development economists, in partnership with the World Bank and the water authority in Nairobi, Kenya's capital, tracked what happened when water supply was cut off to households in several slum settlements where bills hadn't been paid. Researchers wanted to test whether landlords, who are responsible for settling the accounts, would become more likely to pay as a result, and whether residents would protest.
Hundreds of residents in slum settlements in Nairobi were left without access to clean water, in some cases for weeks or months; virtually none of them knew that they were part of an RCT. The study caused outrage among local activists and international researchers.
The criticisms were twofold: first, that the researchers did not obtain explicit consent from participants for their involvement (they said that the landlord's contracts with the water company allowed for the cut-offs); and secondly, that interventions are supposed to be beneficial. The economists involved published an ethical statement defending the trial.
Their research did not make the cut-offs more likely, they explained, because they were a standard part of the water authority's enforcement arsenal (though they acknowledged that disconnections in slums had previously been "ad hoc"). The statement did little to placate the critics.
You know what didn't get an article in The Economist? All the times that slum dwellers in Nairobi were left without access to clean water for weeks or months without anyone studying them.
By the revealed preferences of local activists, international researchers, and The Economist, the problem isn't that people are going without clean water, or that the water authority is shutting off people's water - those things have been going on for decades without more than muted complaining. The ethical problem is that someone is checking whether this unthinkably vast amount of human suffering is actually accomplishing anything.
The water authority is presumably not shutting off people's water recreationally: it's shutting off people's water because they think it will get them to pay their water bills. Therefore, the possible effects of this study are:
The water authority continues to do the same thing it was doing all along.
The water authority learns that shutting off water doesn't get people to pay their bills, so it stops shutting off people's water, and they have enough to drink.
If you step back from your instinctive ick reaction, you'll notice that this study may well improve water access for slum dwellers in Nairobi, and certainly isn't going to make it any worse. But people are still outraged because, I don't know, they have a strongly felt moral opposition to random number generators.
I really don't understand the revulsion people feel about experimenting on humans. It's true that many scientists have done great evil in the name of science:
the Tuskegee syphilis experiment,
MKUltra, Nazi human experimentation, the Imperial Japanese Unit 731.[1] But the problem isn't the experiments. It's not somehow okay to deny people treatment for deadly diseases, force them to take drugs, or torture them if you happen to not write anything down about it.
If it's fine to do something, then it's fine to randomly assign people to two groups, only do it to half of ...
This post was cross-posted from the substack Thing of Things with the permission of the author.
In defense of trying things out
The Economist recently published
an article, "How poor Kenyans became economists' guinea pigs," which critiques development economists' use of randomized controlled trials. I think it exemplifies the profoundly weird way people think about experiments.
The article says:
In 2018, an RCT run by two development economists, in partnership with the World Bank and the water authority in Nairobi, Kenya's capital, tracked what happened when water supply was cut off to households in several slum settlements where bills hadn't been paid. Researchers wanted to test whether landlords, who are responsible for settling the accounts, would become more likely to pay as a result, and whether residents would protest.
Hundreds of residents in slum settlements in Nairobi were left without access to clean water, in some cases for weeks or months; virtually none of them knew that they were part of an RCT. The study caused outrage among local activists and international researchers.
The criticisms were twofold: first, that the researchers did not obtain explicit consent from participants for their involvement (they said that the landlord's contracts with the water company allowed for the cut-offs); and secondly, that interventions are supposed to be beneficial. The economists involved published an ethical statement defending the trial.
Their research did not make the cut-offs more likely, they explained, because they were a standard part of the water authority's enforcement arsenal (though they acknowledged that disconnections in slums had previously been "ad hoc"). The statement did little to placate the critics.
You know what didn't get an article in The Economist? All the times that slum dwellers in Nairobi were left without access to clean water for weeks or months without anyone studying them.
By the revealed preferences of local activists, international researchers, and The Economist, the problem isn't that people are going without clean water, or that the water authority is shutting off people's water - those things have been going on for decades without more than muted complaining. The ethical problem is that someone is checking whether this unthinkably vast amount of human suffering is actually accomplishing anything.
The water authority is presumably not shutting off people's water recreationally: it's shutting off people's water because they think it will get them to pay their water bills. Therefore, the possible effects of this study are:
The water authority continues to do the same thing it was doing all along.
The water authority learns that shutting off water doesn't get people to pay their bills, so it stops shutting off people's water, and they have enough to drink.
If you step back from your instinctive ick reaction, you'll notice that this study may well improve water access for slum dwellers in Nairobi, and certainly isn't going to make it any worse. But people are still outraged because, I don't know, they have a strongly felt moral opposition to random number generators.
I really don't understand the revulsion people feel about experimenting on humans. It's true that many scientists have done great evil in the name of science:
the Tuskegee syphilis experiment,
MKUltra, Nazi human experimentation, the Imperial Japanese Unit 731.[1] But the problem isn't the experiments. It's not somehow okay to deny people treatment for deadly diseases, force them to take drugs, or torture them if you happen to not write anything down about it.
If it's fine to do something, then it's fine to randomly assign people to two groups, only do it to half of ...