John List on Economic Field Experiments

Any work in social and behavioral science presumably – but not necessarily immediately - tells us something about humans in the real world. To come up with those insights, research usually occurs in laboratory settings, where the researchers control the independent variables and which, in essence, rules out research ‘in the wild.’ Enter John List.

“For years,” he tells interviewer David Edmonds in this Social Science Bites podcast, “economists thought that the world is so ‘dirty’ that you can’t do field experiments. They had the mentality of a test tube in a chemistry lab, and what they had learned was that if there was a speck of dirt in that tube, you’re in trouble because you can’t control exactly what is happening.”

Since this complex real world isn’t getting any cleaner, you could conclusively rule out field experiments, and that’s what the ‘giants’ of economics did for years. Or you could learn to work around the ‘dirt,’ which is what List started doing around the turn of the millennium. “I actually use the world as my lab,” the Kenneth C. Griffin Distinguished Service Professor of Economics at the University of Chicago says.

Since an early start centering on sports trading cards and manure-fertilized crop land (real field work, a self-described “bucolic” List happily acknowledges), his university homepage details a raft of field experiments:

“I have made use of several different markets, including using hospitals, pre-K, grammar, and high schools for educational field experiments, countless charitable fundraising field experiments to learn about the science of philanthropy, the Chicago Board of Trade, Costa Rican CEOs, the new automobile market, coin markets, auto repair markets, open air markets located throughout the globe, various venues on the internet, several auction settings, shopping malls, various labor markets, and partnered with various governmental agencies. More recently, I have been engaged in a series of field experiments with various publicly traded corporations—from car manufacturers to travel companies to ride-share.”

In the podcast, List explains, “I don’t anticipate or assume that I have a ‘clean test tube,’ but what I do is I randomly place people into a treatment condition or a control condition, and then what I look at is their outcomes, and I take the difference between those outcomes. That differences out the ‘dirt.’

“I can go to really dirty settings where other empirical approaches really take dramatic assumptions. All I need is really randomization and a few other things in place and then if I just take the simple difference, I can get an average treatment effect from that setting.”

His work – in journal articles, popular books like The Voltage Effect and The Why Axis, in findings applied immediately outside of academe – has earned him widespread praise (Gary Becker terms his output as “revolutionary”), a huge list of honors, and a recurring spot on Nobel shortlists.

For this podcast, List focuses on two of the many areas in which he’s conducted field experiments: charitable giving and the gig economy.

He describes one finding from working with different charities around the world over the last 25 years on what works best to raise money. For example, appeals to potential donors announcing their money would be matched when they gave, doubling or tripling a contribution’s impact. When he started, it was presumed that the greater the leverage offered by a match, the more someone would give, since their total gift would be that much greater.

“There was no science around it … it was art, or gut feeling.” It was also wrong.

List tested the assumption, offering four different appeals to four different groups: one with just an appeal for funding, one with a 1:1 match, one with a 2:1 match, and the last a 3:1 match. And the results bore out that matching a contribution amped up the results – but the leverage didn’t matter. “Just having the match matters, but the rate of the match does not matter.”

List was later the chief economist with ride-share behemoth Uber – and then with its competitor, Lyft. He coined the term Ubernomics for his ability to manipulate the tsunami of data the company generated. “It’s not only that you have access to a lot of data,” he says, “it’s also that you have access to generating a lot of new data. As a field economist, this is a playground that is very, very difficult to beat.”