AWS for Software Companies Podcast Ep077: Developing an AI Strategy for Software Companies
Feb 4, 2025
25:46
In this AWS panel discussion, Naveen Rao, VP of AI of Databricks and Vijay Karunamurthy, Field CTO of Scale AI share practical insights on implementing generative AI in enterprises, leveraging private data effectively, and building reliable production systems.
Topics Include:
- Sherry Marcus introduces panel discussion on generative AI adoption
- Scale AI helps make AI models more reliable
- Databricks focuses on customizing AI with company data
- Companies often stressed about where to start with AI
- Board-level pressure driving many enterprise AI initiatives
- Start by defining specific goals and success metrics
- Build evaluations first before implementing AI solutions
- Avoid rushing into demos without proper planning
- Enterprise data vastly exceeds public training data volume
- Customer support histories valuable for AI training
- Models learning to anticipate customer follow-up questions
- Production concerns: cost, latency, and accuracy trade-offs
- Good telemetry crucial for diagnosing AI application issues
- Speed matters more for prose, accuracy for legal documents
- Cost becomes important once systems begin scaling up
- Organizations struggle with poor quality existing data
- Privacy crucial when leveraging internal business data
- Role-based access control essential for regulated industries
- AI can help locate relevant data across legacy systems
- Models need organizational awareness to find data effectively
- Private data behind firewalls most valuable for AI
- Customization gives competitive advantage over generic models
- Current AI models primarily do flexible data recall
- Next few years: focus on deriving business value
- Future developments in causal inference expected post-5 years
- Complex multi-agent systems becoming more important
- Scale AI developing "humanity's last exam" evaluation metric
- Discussion of responsibility and liability in AI decisions
- Companies must stand behind their AI system outputs
- Existing compliance frameworks can be adapted for AI
Participants:
- Naveen Rao – VP of AI, Databricks
- Vijay Karunamurthy – Field CTO, Scale AI
- Sherry Marcus Ph.D. - Director, Applied Science, AWS
See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/