Latent Space: The AI Engineer Podcast cover image

Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit

Latent Space: The AI Engineer Podcast

00:00

Unveiling Replicode V1 3B

This chapter introduces Replicode V1 3B, a new coding model with 2.7 billion parameters trained on 525 billion tokens of permissively licensed code. It explores its unique vocabulary designed for programming, performance enhancements over previous versions, and the challenges faced during its development. The chapter also discusses the ambitious shift in training methodology, emphasizing the balance between data volume and computational efficiency.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app