#196 - Nvidia Digits, Cosmos, PRIME, ICLR, InfAlign
Last Week in AI
Intro
The hosts kick off the episode by sharing their backgrounds in AI and outlining the podcast's format. They humorously explore the uncertainty of AI news, encouraging listeners to engage with the latest developments in the field.
Our 196th episode with a summary and discussion of last week's* big AI news!
*and sometimes last last week's
Recorded on 01/10/2024
Join our brand new Discord here! https://discord.gg/nTyezGSKwP
Hosted by Andrey Kurenkov and Jeremie Harris.
Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai
Read out our text newsletter and comment on the podcast at https://lastweekin.ai/.
Sponsors:
- The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence.
In this episode:
- Nvidia announced a $3,000 personal AI supercomputer called Digits, featuring the GB10 Grace Blackwell Superchip, aiming to lower the barrier for developers working on large models.
- The U.S. Department of Justice finalizes a rule restricting the transmission of specific data types to countries of concern, including China and Russia, under executive order 14117.
- Meta allegedly trained Llama on pirated content from LibGen, with internal concerns about the legality confirmed through court filings.
- Microsoft paused construction on a section of a large data center project in Wisconsin to reassess based on new technological changes.
If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form.
Timestamps + Links:
- (00:00:00) Intro / Banter
- (00:04:52) Sponsor Break
- Tools & Apps
- Applications & Business
- (00:16:16) NVIDIA Is Reportedly Focused Towards “Custom Chip” Manufacturing, Recruiting Top Taiwanese Talent
- (00:21:54) AI start-up Anthropic closes in on $60bn valuation
- (00:25:38) Why OpenAI is Taking So Long to Launch Agents
- (00:30:08) TSMC Set to Expand CoWoS Capacity to Record 75,000 Wafers in 2025, Doubling 2024 Output
- (00:33:10) Microsoft 'pauses construction' on part of data center site in Mount Pleasant, Wisconsin
- (00:37:23) Google folds more AI teams into DeepMind to ‘accelerate the research to developer pipeline’
- Projects & Open Source
- Research & Advancements
- (00:50:16) PRIME: Online Reinforcement Learning with Process Rewards
- (00:58:29) ICLR: In-Context Learning of Representations
- (01:07:38) Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
- (01:11:44) METAGENE-1: Metagenomic Foundation Model for Pandemic Monitoring
- (01:15:45) TransPixar: Advancing Text-to-Video Generation with Transparency
- (01:18:03) The amount of compute used to train frontier models has been growing at a breakneck pace of over 4x per year since 2018, resulting in an overall scale-up of more than 10,000x! But what factors are enabling this rapid growth?
- Policy & Safety
- (01:23:45) InfAlign: Inference-aware language model alignment
- (01:28:44) Mark Zuckerberg gave Meta’s Llama team the OK to train on copyrighted works, filing claims
- (01:33:19) Anthropic gives court authority to intervene if chatbot spits out song lyrics
- (01:35:57) US government says companies are no longer allowed to send bulk data to these nations
- (01:39:10) Trump announces $20B plan to build new data centers in the US
Remember Everything You Learn from Podcasts
