Perverse Incentives Select for People Who Are the Best at Exploiting a Given System

Get the app

Paul Smaldino & C. Thi Nguyen on Problems with Value Metrics & Governance at Scale (EPE 06)

COMPLEXITY

chevron_right

notes

NOTE

Perverse Incentives Select for People Who Are the Best at Exploiting a Given System

The original deans and administrators burn out due to their dislike of the US News and World Report rankings and are replaced by individuals driven by ranking success. This shift reflects a difference in mentality between valuing money as a means of support versus valuing money as the sole purpose of life. Similarly, pursuing publications and citations for a job versus making them the ultimate goal shows a significant distinction. However, these differences are connected through a temporal dynamic where initially people adapt their behavior to succeed in a flawed system. The system then filters out those who can best exploit it, resulting in the selection of individuals with specific values.

00:00

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

There are maps, and there are territories, and humans frequently confuse the two. No matter how insistently this point has been made by cognitive neuroscience, epistemology, economics, and a score of other disciplines, one common human error is to act as if we know what we should measure, and that what we measure is what matters. But what we value doesn’t even always have a metric. And even reasonable proxies can distort our understanding of and behavior in the world we want to navigate. Even carefully collected biometric data can occlude the other factors that determine health, or can oversimplify a nuanced conversation on the plural and contextual dimensions of health, transforming goals like functional fitness into something easier to quantify but far less useful. This philosophical conundrum magnifies when we consider governance at scales beyond those at which Homo sapiens evolved to grasp intuitively: What should we count to wisely operate a nation-state? How do we practice social science in a way that can inform new, smarter species of political economy? And how can we escape the seductive but false clarity of systems that rain information but do not enhance collective wisdom?

Welcome to COMPLEXITY, the official podcast of the Santa Fe Institute. I’m your host, Michael Garfield, and every other week we’ll bring you with us for far-ranging conversations with our worldwide network of rigorous researchers developing new frameworks to explain the deepest mysteries of the universe.

This week on the show we talk to SFI External Professor Paul Smaldino at UC Merced and University of Utah Professor of Philosophy C. Thi Nguyen. In this episode we talk about value capture and legibility, viewpoint diversity, issues that plague big governments, and expert identification problems…and map the challenges “ahead of us” as SFI continues as the hub of a five-year international research collaboration into emergent political economies. (Find links to all previous episodes in this sub-series in the notes below.)

Be sure to check out our extensive show notes with links to all our references at complexity.simplecast.com. If you value our research and communication efforts, please subscribe, rate and review us at Apple Podcasts or Spotify, and consider making a donation — or finding other ways to engage with us — at santafe.edu/engage.

If you’d like some HD virtual backgrounds of the SFI campus to use on video calls and a chance to win a signed copy of one of our books from the SFI Press, help us improve our science communication by completing a survey about our various scicomm channels. Thanks for your time!

Lastly, we have a bevy of summer programs coming up! Join us June 19-23 for Collective Intelligence: Foundations + Radical Ideas, a first-ever event open to both academics and professionals, with sessions on adaptive matter, animal groups, brains, AI, teams, and more. Space is limited! The application deadline has been extended to March 1st.

OR apply to the Graduate Workshop on Complexity in Social Science.

OR the Complex ity GAINS UK program for PhD students.

(OR check our open listings for a staff or research job!)

Join our Facebook discussion group to meet like minds and talk about each episode.

Podcast theme music by Mitch Mignano.

Mentioned & Related Links:

Transparency Is Surveillance
by C. Thi Nguyen

The Seductions of Clarity
by C. Thi Nguyen

The Natural Selection of Bad Science
by Paul Smaldino and Richard McElreath

Maintaining transient diversity is a general principle for improving collective problem solving
by Paul Smaldino, Cody Moser, Alejandro Pérez Velilla, Mikkel Werling

The Division of Cognitive Labor
by Philip Kitcher

The Unreasonable Effectiveness of Mathematics in The Natural Sciences
by Eugene Wigner