Exploring AI Models and Evaluation Criteria

This week on the podcast, Matthew Jackson from Stanford University is the guest and it was such a delight for me to talk to him and get to know his story a little better. I’d met him before, but only briefly, but I’d read a lot of his work because I once developed and taught a class on networks for our masters of economics students. His textbook on the economic and social networks is excellent but he also has a general interest book on networks if you’re wanting something more accessible.

As the podcast is technically both listening to the stories of living economists and an oral history project, maybe it is worth noting this (though I think it’s obvious to most listeners) that Matt is a micro theorist whose work has empirical content. Not all micro theory does and not all empirical work is necessarily theoretically driven, which is why I make that technical distinction. Networks are also, I think, so clearly an important part of human existence. We make friends, we catch diseases, we learn about opportunities (and maybe as importantly, don’t learn about opportunities) because of networks. And so in a very real sense, even the classical definition of economics proposed by Lionel Robbins, that economics is the study of the allocation of scarce resources by people with unlimited desires, can alone justify the study of networks if networks, as opposed to merely markets and market prices, are actually an important part of that resource allocation process itself. It’s so interesting — as someone nearly 50 to consider all the ways economics evolved over the last 50 years and continues to evolve while still remaining at its core connected to core questions like “how do humans manage to survive on this planet given they have so little time and so little resources?”

Anyway, one last thing. At the end of the podcast, I ask Matt about his new work on artificial intelligence. The paper is at PNAS and is currently unlocked. It’s entitled “A Turing Test of Whether AI Chatbots are Behaviorally Similar to Humans” and it’s by Matt, Qiaozhu Mei, Yutong Xie, and Walter Yuan. They had ChatGPT-4 play a variety of classic games, like dictator games, prisoner’s dilemma, and so on. And they mapped the way the chatbot played to the way humans have planed these games in the lab. The one thing that I found really interesting in what they found was that ChatGPT-4 is altruistic. “It” appears to play the game altruistically in the sense that it attempts to maximize a weighted average of both its payouts and its opponent’s payouts. What then should we expect if we in the long run end up with a network of chatbots? Hard to say what the general equilibrium will be as game theoretic equilibria are often surprising and not immediately intuitive and usually depend on institutions and incentives, but still it’s quite fascinating to me. I hope you liked this interview!

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.