Enhancing Language Models with Relaxed Recursive Transformers

Our 189th episode with a summary and discussion of last week's big AI news!

Hosted by Andrey Kurenkov and Jeremie Harris.
Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai

Read out our text newsletter and comment on the podcast at https://lastweekin.ai/.

In this episode:

* OpenAI's acquisition of chat.com and internal shifts, including hardware lead hire and hardware model leaks, signal significant strategy pivots and challenges with model scaling and security. 
* Saudi Arabia plans a $100 billion AI initiative aiming to rival UAE's tech hub, highlighting the region's escalating AI investments. 
* U.S. penalties on GlobalFoundries for violating sanctions against SMIC underline ongoing challenges in enforcing AI-chip export controls. 
* Anthropic collaborates with Palantir and AWS to integrate CLAWD into defense environments, marking a significant policy shift for the company.

Sponsors:

• The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence.
• The AI safety book “Uncontrollable" which is not a doomer book, but instead lays out the reasonable case for AI safety and what we can do about it. Max TEGMARK said that “Uncontrollable” is a captivating, balanced, and remarkably up-to-date book on the most important issue of our time" - find it on Amazon today!

If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form.

Timestamps + Links:

• (00:00:00) Intro / Banter
• (00:01:28) News Preview
• (00:02:10) Response to listener comments
• (00:05:02) Sponsor Break
• Tools & Apps
  • (00:07:31) OpenAI Introduces ‘Predicted Outputs’ Feature: Speeding Up GPT-4o by ~5x for Tasks like Editing Docs or Refactoring Code
  • (00:11:55) Anthropic’s Haiku 3.5 surprises experts with an “intelligence” price increase
  • (00:17:10) Introducing FLUX1.1 [pro] Ultra and Raw Modes
  • (00:19:11) X is testing a free version of Grok AI chatbot in select regions
• Applications & Business
  • (00:21:39) OpenAI acquired Chat.com
  • (00:23:40) Saudis Plan $100 Billion AI Powerhouse to Rival UAE Tech Hub
  • (00:28:28) Meta’s former hardware lead for Orion is joining OpenAI
  • (00:31:38) OpenAI Accidentally Leaked Its Upcoming o1 Model to Anyone With a Certain Web Address
  • (00:35:50) Nvidia Rides AI Wave to Pass Apple as World’s Largest Company
• Projects & Open Source
  • (00:37:53) ‘Unrestricted’ AI group Nous Research launches first chatbot — with guardrails
  • (00:41:48) FrontierMath: The Benchmark that Highlights AI’s Limits in Mathematics
  • (00:46:29) Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
• Research & Advancements
  • (00:49:55) Applying “Golden Gate Claude” mechanistic interpretability techniques to protein language models.
  • (00:58:3) Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
  • (01:05:55) From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code
  • (01:10:22) OpenAI reportedly developing new strategies to deal with AI improvement slowdown
• Policy & Safety
  • (01:19:52)  What Donald Trump’s Win Means For AI
  • (01:28:44) Fab Whack-A-Mole: Chinese Companies are Evading U.S. Sanctions
  • (01:33:57) US fines GlobalFoundries for shipping chips to sanctioned Chinese firm
  • (01:36:55) Anthropic teams up with Palantir and AWS to sell its AI to defense customers
• (01:39:23) Outro