Weaviate Podcast Haize Labs with Leonard Tang - Weaviate Podcast #121!
May 12, 2025
Leonard Tang, co-founder of Haize Labs, delves into innovative techniques for AI evaluation. He shares how stacking weaker models can enhance the performance of stronger ones through the revolutionary Verdict library, boasting a 10-20% improvement over traditional models. The conversation includes practical insights on creating contrastive evaluation sets and implementing debate-based judging systems. Tang discusses the balance between AI safety and user feedback, offering transformative strategies to ensure that AI systems meet enterprise needs effectively.
AI Snips
Chapters
Transcript
Episode notes
From Academia to Haize Labs
- Leonard Tang transitioned from academic adversarial AI safety research to founding Haize Labs in 2024.
- He started the company during his last undergrad semester, focusing on practical AI robustness solutions for enterprises.
Simulating User Interactions for Evals
- Haize Labs excels at creating customer-aligned reward models by densifying annotations for automated judging.
- They simulate user interactions to generate evaluation data even before an AI app goes into production.
Use Contrastive Responses for Feedback
- Use contrastive response pairs to have annotators explain why one output is better than another.
- This approach yields more stable and informative human feedback than numeric ratings alone.