The Orthogonal Bet Rohit Krishnan on Training AI to Write Better
8 snips
Oct 15, 2025 Join Rohit Krishnan, a creative force behind the Strange Loop Canon newsletter and an AI practitioner, as he dives into the world of artificial intelligence and writing. He reveals the intriguing flaws of large language models in generating prose and discusses the role of reinforcement learning in training AI like his project, Walter. Rohit also explores how AI can reshape our future work dynamics, likening it to managing playful video games. Above all, he emphasizes the importance of experimentation and play in unlocking AI's full potential.
AI Snips
Chapters
Books
Transcript
Episode notes
LLMs Struggle With What They're Built For
- Large language models perform poorly at high-quality writing despite being trained on text.
- Rohit suggests this gap is puzzling and worth targeting with specialized reward design.
Social Media As A Reward Source
- Social media provides abundant but noisy reward signals for training writing.
- Rohit found small models can learn better writing from social engagement data despite noise.
Great Writing Hides Trainable Patterns
- High-quality writing has deeper, subtler patterns than surface token statistics.
- Rohit proposes extracting surprisal and sampling patterns from great literature to create training gradients.
