#195 - OpenAI o3 & for-profit, DeepSeek-V3, Latent Space
Last Week in AI
Intro
This chapter explores important recent developments in artificial intelligence, particularly focusing on OpenAI's DeepSeek V3 project. The hosts share personal experiences to create a relatable atmosphere while teasing future discussions on the significance of the topics covered.
Our 195th episode with a summary and discussion of last week's* big AI news!
*and sometimes last last week's
Recorded on 01/04/2024
Join our brand new Discord here! https://discord.gg/nTyezGSKwP
Note: apologies for Andrey's slurred speech and the jumpy editing, will be back to normal next week!
Hosted by Andrey Kurenkov and Jeremie Harris.
Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai
Read out our text newsletter and comment on the podcast at https://lastweekin.ai/.
Sponsors:
- The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence.
In this episode:
- OpenAI teases new deliberative alignment techniques in its O3 model, showcasing major improvements in reasoning benchmarks, whilst surprising with autonomy in hacks against chess engines.
- Microsoft and OpenAI continue to wrangle over the terms of their partnership, highlighting tensions amid OpenAI's shift towards a for-profit model.
- Chinese AI companies like DeepSeek and Quen release advanced open-source models, presenting significant contributions to AI capabilities and performance optimization.
- Sakana AI introduces innovative applications of AI to the search for artificial life, emphasizing the potential and curiosity-driven outcomes of open-ended learning and exploration.
If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form.
Timestamps + Links:
- (00:00:00) Intro / Banter
- (00:03:07) News Preview
- (00:03:54) Response to listener comments
- (00:05:00) Sponsor Break
- Tools & Apps
- Applications & Business
- (00:24:24) OpenAI announces plan to transform into a for-profit company
- (00:33:17) Microsoft and OpenAI Wrangle Over Terms of Their Blockbuster Partnership
- (00:37:36) Elon Musk’s xAI gets investment from Nvidia in recent funding round: report
- (00:39:43) Sam Altman’s nuclear energy startup signs one of the largest nuclear power deals to date
- (00:41:13) OpenAI Search Leader Departs After Less Than a Year
- (00:42:43) Senior OpenAI Researcher Radford Departs
- Projects & Open Source
- (00:45:21) DeepSeek-AI Just Released DeepSeek-V3: A Strong Mixture-of-Experts (MoE) Language Model with 671B Total Parameters with 37B Activated for Each Token
- (00:54:14) Qwen Team Releases QvQ: An Open-Weight Model for Multimodal Reasoning
- (00:58:09) LightOn and Answer.ai Releases ModernBERT: A New Model Series that is a Pareto Improvement over BERT with both Speed and Accuracy
- Research & Advancements
- Policy & Safety
- (01:10:27) Nonprofit group joins Elon Musk’s effort to block OpenAI’s for-profit transition
- (01:14:35) OpenAI Researchers Propose 'Deliberative Alignment' : A Training Approach that Teaches LLMs to Explicitly Reason through Safety Specifications before Producing an Answer
- (01:22:06) o1-preview autonomously hacked its environment rather than lose to Stockfish in our chess challenge. No adversarial prompting needed.
- (01:27:22) Elon Musk’s xAI supercomputer gets 150MW power boost despite concerns over grid impact and local power stability
- (01:29:06) DOE: Data centers consumed 4.4% of US power in 2023, could hit 12% by 2028
- Synthetic Media & Art
- (01:36:15) Outro
The AI-powered Podcast Player
