Accenture AI Leaders Podcast AI Leaders Podcast #76: Scaling AI in the Enterprise
4 snips
Sep 30, 2025 Naveen Rao, Vice President of AI at Databricks and a pioneer in AI chips, shares his insights on scaling AI in large organizations. He discusses the challenges of operationalizing AI, including infrastructure failures and the importance of a robust data foundation. The conversation dives into quality in generative AI, emphasizing the need for reliable models tailored to specific industries. Naveen also highlights the significance of user interface design and data governance to ensure effective AI integration in workflows.
AI Snips
Chapters
Transcript
Episode notes
Correctness Is The Enterprise Barrier
- Correctness and grounding in real data are the biggest blockers for enterprise generative AI.
- Hallucinations undermine trust and make many high-stakes enterprise use cases infeasible.
From Cut‑Paste To AI‑Enabled SQL
- At Databricks users were copying SQL query results into a chat to summarize data, which was slow and unscalable.
- They added SQL extensions to call AI directly, unlocking large-scale batch inference and smoother workflows.
Build For Constant Hardware Failures
- Design systems to handle hardware failures gracefully when scaling across thousands of GPUs.
- Expect frequent failures as scale grows and build software that continues without full shutdowns.