Is There a Bias in Online Reviews?

Chris Nosko is a PhD economist. He did his PhD in economics at Harvard in the mid 2010s before going to Chicago Booth take a job as an assistant professor. But for a year prior to taking that job, between Harvard and Chicago, he did a postdoc fellowship at eBay where he, Thomas Blake and Steve Tadelis met and worked together on a project involved a serendipitous event at the company in which eBay quit paying for branded key words (e.g., “eBay Volvo decals”, “eBay typewriters”) on some but not all search engine auctions. They asked for the data on traffic to the site before and after eBay quit paying for branded keywords for all search engines (both those they kept paying and those they didn’t), ran a simple event study diff-in-diff and found evidence that search engine marketing at eBay was perhaps not causing increased traffic to the site. They convinced management to field a large RCT which confirmed their diff-in-diff results, and that study was published in Econometrica. Not a shabby way to start a career as an economist.

For many of us, a PhD in economics from Harvard, a successful partnership with eBay resulting in a study destined for a Top 5 and a tenure track job at Chicago Booth meant staying at Booth and having a career as an academic. No one outrightly says that the only meaningful life you can have as an economist is to be an academic, as it’s vulgar, opinionated and obviously false to talk that way about how someone else should live their life, but the norms are pretty powerful nonetheless. Well, starting around the time that Chris got his job at Booth, tech began experiencing a surge in hiring of PhD economists, largely driven by Amazon’s nearly insatiable appetite for them. Talking with people at Amazon, I have learned that behind this push was Pat Bajari, and behind Pat Bajari was Jeff Bezos who had long believed economics, and economists more specifically, had unique value. As Susan Athey said to me, though, in an interview earlier, Bajari though had to do pull a rabbit out of a hat. Whereas the first wave of economists to tech — people like Hal Varian, Susan Athey, Preston McAfee — had largely been micro theorists helping craft the foundations of a business model through auctions and advertising that would support search engines, arguably the core arteries of the internet itself — Bajari would have the task of bringing in young people, fresh out of grad school, and in Athey’s words, make them productive. And one of the people Bajari would ultimately tap do that was Chris Nosko, an assistant professor at Chicago Booth and someone trained in structural industrial organization, one of the economics’ more interesting experiments of fusing deep microeconomic theory with econometric estimation.

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Nosko was a Ariel Pakes student at Harvard and was well versed in so many different parts of economics and modern technology that it almost seems predestined that he would ultimately leave Chicago Booth permanently and go to Amazon when Bajari finally convinced him to, but that’s all selection on the dependent variable reasoning. When we look back in time at decisions we made, our mind tends to forget that there was a moment when we could’ve gone left instead of right. The same with Chris — there was a decision that had to be made to leave a career as an academic. The decision materialized into what it materialized, but to pretend it was easy, or that it didn’t have risk, or that Chris didn’t try to manage that risk in some ways is really unfair to our earlier selves or even our future selves who are in situations facing, not probabilistic risk but more like Knightian uncertainty in which no one truly has a clue what possibly could happen.

But Chris did leave. Sort of. He took “a leave of absence” from Booth in 2015 and took a job at Amazon, then permanently left Booth in 2016. He spent four years at Amazon before leaving for Uber, one of the more impressive firms to ever exist for creating an actual open marketplace solving two sided matching problems through algorithms and prices. Algorithms, prices and rules — three ways, no doubt there are others, in which modern economies coordinate productive activity. Is it really so surprising that economics might be valued by tech firms given the complex coordination they try to solve using all three?

Thank you for reading Scott's Substack. This post is public so feel free to share it.

Chris has been at Uber for four years. He is now Vice President and Head of Science and Analytics for Uber Product there.  Within tech, economists sort into tons of different jobs with titles that to an academic don’t make a ton of sense — just like so much of what academics’ lives takes place within administrative units that make little sense to anyone else. If Chris isn’t the chief economist, though, at Uber, I figure he’s probably up there. And he’s my guest this week on The Mixtape with Scott as part of my longer, unfolding series I call “Economists in tech”. Our conversation covered a lot of ground. We talked about growing up in rural Oregon, falling into programming early on and working a few years between high school and college during the early wave tech boom of the late 1990s and early 2000s as a programmer. It wasn’t exactly what he would do later, as that was more web design and less machine learning and statistics, but the aptitude of programming is very portable and his deep knowledge of tech sectors was anyway established or at least re-invested in while there. We talked about his love for his liberal arts education at the University of Chicago where he did his undergraduate degree, and his broad navigation of economics as a field and a career. 

All in all, it was a fun opportunity to talk to Chris, to learn more about his own path, about the world out there outside of academia, what economists do in tech, and how all of these things fit together for both economics but maybe more importantly just for Chris himself. I think a lot of people are going to find Chris’s story very interesting and personally intriguing as they may see him in themselves. You can read some of Chris’s work here. Thanks again for tuning in! I hope you enjoy this week’s interview as much as I did! If you are enjoying these, please consider supporting me by sharing the podcast and/or becoming a paying subscriber!

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.