Achieving Language Benchmark Does Not Imply General Intelligence

1min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

Attaining proficiency in language benchmarks does not equate to being proficient in all tasks. A benchmark in language does not signify being close to accomplishing a wide range of tasks in the economy. The ability to excel in one specific area, like chess, does not require possessing all capacities of general intelligence. History shows that focusing solely on mastering a specific task, such as chess, does not necessarily lead to achieving general artificial intelligence.

In this episode, Nathan sits down with Robin Hanson, associate professor of economics at George Mason University and researcher at Oxford’s Future of Humanity Institute. They discuss the comparison of human brains to LLMs and legacy software systems, what it would take for AI and automation to significantly impact the economy, our relationships with AI and the moral weight it has, and much more. Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api

LINKS:

Robin’s Book, The Age of Em: https://ageofem.com/
Robin’s essay on Automation: https://www.overcomingbias.com/p/no-recent-automation-revolutionhtml
Robin’s Blog: https://www.overcomingbias.com/
AI Scouting Report: https://www.youtube.com/watch?v=0hvtiVQ_LqQ&list=PLVfJCYRuaJIXooK_KWju5djdVmEpH81ee&pp=iAQB
Dr. Isaac Kohane Episode: https://www.youtube.com/watch?v=pS5Vye671Xg

SPONSORS:

The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

X/SOCIAL:

@labenz

@robinhanson (Robin)

@CogRev_Podcast

TIMESTAMPS

(00:00) Preview

(07:10) Why our current time is a “dream time” and the move back to a Malthusian world

(13:30) What sort of world should we be striving for?

(13:40) Sponsor - Brave

(17:50) Distinguishing value talk from factual talk

(18:00) Comparing and contrasting Ems to LLMs

(22:30) The comparison of human brains to legacy software systems

(30:52) Sponsor - Netsuite

(41:01) AIs in medicine

(53:30) A several century innovation pause

(55:30) Achieving full human level AI in the next 60-90 years

(1:03:55) Chess and routine benchmarks not a good predictor of AI performance in the economy

(1:07:44) Reaching and exceeding human-level AI in the next 1000 years

(1:11:40) Losing technologies tied to scale economies

(1:12:00) Why AI is hard to maintain in the long run

(1:12:20) Standard deviation in automation

(1:14:05) Computing power grows exponentially but automation grows steadily

(1:15:50) AI art generation and deepfakes

(01:21:42) The economics of AI-powered coding

(1:33:51) Merging LLMs

(1:36:02) Rot in software and the human brain

(1:40:18) Parallelism in LLMs and brain design

(1:41:00]) Moral weight for AIs, enslavement, and cooperation with AI

(1:47:10) What would change Robin’s mind about the future

(1:49:18) Wrap

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.