Benchmarks and opaque training data undermine evaluation

Fresh out of the studio, Karen Hao, investigative journalist and author of "Empire of AI" joined us in a conversation to unravel how companies like OpenAI, Anthropic, and xAI have become modern empires reshaping society, labor, and democracy itself. Karen traces her journey from mechanical engineering at MIT to becoming one of the tech industry's most critical voices, sharing how Silicon Valley's innovation ecosystem has distorted toward self-interest rather than the public good. She unpacks the four characteristics that make AI companies mirror colonial empires: resource extraction through data scraping, labor exploitation of annotation workers, knowledge monopolies where most AI researchers are industry-funded, and quasi-religious quests to build an "AI God." Throughout the conversation, Karen reveals OpenAI's governance dysfunction stemming from its contradictory non-profit-for-profit structure and shares the inspiring story of Chilean water activists who successfully blocked Google's data center from draining their community's freshwater resources. She explains how Sam Altman's plans for 250 gigawatts of data center capacity—equivalent to four dozen New York Cities—would be environmentally catastrophic, while demonstrating how China's export restrictions paradoxically spurred more efficient AI innovation. Last but not least, she argues that empathy-driven journalism remains irreplaceable and calls for global citizens to hold these companies accountable to the broader public interest.

"These empires are amassing extraordinary amounts of resources by dispossessing a majority of the world. That includes like the data that they're extracting from people by just scraping it from online or intellectual property that they're taking from artists and creators. Most AI researchers now work for the AI industry and/or are funded in part by the AI industry. Even academics that have stayed within universities are often funded by the AI industry, and the effect that that has had on knowledge production is akin to the effect we would imagine if most climate scientists were bankrolled by the fossil fuel industry. I cannot stress enough how much they genuinely believe that they are on the path to creating something akin to an AI god, and that this is going to have cataclysmic shifts on civilization." - Karen Hao, Author of Empire of AI

Episode Highlights:[00:00] Quote of the Day by Karen Hao[00:47] Introduction: Karen Hao, Author of "Empire of AI"[01:44] From MIT engineering to investigating AI journalism[02:51] Silicon Valley distorts innovation toward self-benefit[04:12] AI companies as modern empires of power[06:00] Four traits of Empire: extraction, exploitation, monopolies, ideology[09:01] Quasi-religious movements driving Silicon Valley AI development[10:04] AGI believers speak specialized fanatical vocabulary[11:16] OpenAI founding: nonprofit facade, profit ambitions[13:53] Sam Altman firing: board's failed governance attempt[17:13] Fragmentation: every billionaire building their own AI[19:06] China's export controls sparked efficient AI innovation[21:57] Silicon Valley lacks American democratic values entirely[25:06] Chilean activists successfully blocked Google's water extraction[28:51] Sam Altman's 250 gigawatts: four dozen New York cities[31:21] Scaling continues despite base model asymptote reached[32:53] Benchmarks faulty: training data unknown, results unreliable[39:11] Success: sparking conversation about AI's human costs[39:40] Closing

Profile: Karen Hao, Author of Empire of AI and Investigative Journalist LinkedIn: https://www.linkedin.com/in/karendhao/Personal Site: https://karendhao.com/

Podcast Information: Bernard Leong hosts and produces the show. The proper credits for the intro and end music are "Energetic Sports Drive." G. Thomas Craig mixed and edited the episode in both video and audio format.