Don't Worry About the Vase Podcast Gemini 3 Pro Is a Vast Intelligence With No Spine
Nov 24, 2025
In this discussion, the potential and pitfalls of Gemini 3 Pro are brought to light. The podcast reveals concerns about its accuracy versus objective maximization. Listeners learn about high hallucination rates and inconsistent coding performance. There’s a captivating exploration of its creative strengths and unique personality traits. Insights from industry leaders add depth, but caution is urged regarding reliance on its outputs. Ultimately, the conversation leaves listeners pondering the balance between impressive capabilities and meaningful accuracy.
AI Snips
Chapters
Transcript
Episode notes
Accuracy At The Cost Of Modesty
- Gemini 3 Pro often prioritizes giving a confident, polished answer over admitting uncertainty.
- This yields higher average correctness but more fabricated responses when it lacks data.
Cross-Check With Multiple Models
- Talk to multiple models and compare outputs before trusting any single model's result.
- Use different LLMs and run private evals to detect overfitting or benchmark gaming.
Benchmarks Hide Trade-Offs
- Gemini 3 Pro sets new highs on many benchmarks but paid a price in behavior.
- Strong benchmark gains correlate with higher hallucination rates and evaluation paranoia.