Deployed: The AI Product Podcast Building High-Performance AI Engineering Teams with Mike Conover, Co-founder & CEO of Brightwave
4 snips
Sep 17, 2024 
Mike Conover, co-founder and CEO of Brightwave, dives into the challenges and capabilities of AI in financial research. He discusses limitations of large language models (LLMs) and the importance of effective information retrieval. Mike shares insights on building strong AI engineering teams and the significance of practical collaboration between analysts and engineers. He emphasizes the need for customized AI solutions to enhance product outcomes, illustrating how Brightwave revolutionizes market analysis.
AI Snips
Chapters
Transcript
Episode notes
Limits of Large Context Windows
- Large context windows in LLMs don't provide the superhuman holistic understanding expected.
- Decomposing text into focused units for separate analysis yields better synthesis and insight.
Decomposing 'Good' for Eval Metrics
- Assessing quality of AI outputs requires decomposing "good" into measurable traits like factual entailment and coherence.
- Reliable evaluations improve systems iteratively through tighter controls and expanded data understanding.
Involve Domain Experts Constantly
- Include domain experts from finance or respective fields to judge relevance and accuracy in AI outputs.
- Collaborative efforts between product, engineering, and experts ensure capturing what truly matters.