Building Box's AI Platform: Enterprise Lessons in Scaling LLMs | Ben Kus, CTO of Box
Feb 20, 2025
auto_awesome
Ben Kus, CTO of Box, leads their AI initiatives by integrating secure and scalable AI solutions into enterprise content management. He discusses the transformative potential of generative AI, sharing how Box navigated the shift from skepticism to enthusiasm among customers. Ben emphasizes the importance of collaboration in AI implementation and maintaining strict quality standards while ensuring data security. He also highlights the evolving expectations within enterprises and the need for adaptive engineering teams to stay ahead in the rapidly evolving AI landscape.
The integration of agentic AI into enterprises revolutionizes how unstructured data is processed, enhancing efficiency and workflow management.
Generative AI empowers organizations like Box to provide users with actionable insights from vast unstructured content, transforming operations significantly.
To successfully adopt AI technologies, companies must foster a culture of flexibility and collaborative experimentation among engineering teams, adapting to rapid advancements.
Deep dives
Understanding Agentic AI
The evolution of agentic AI signifies a shift in how enterprises process unstructured data, such as documents and contracts. Initially, businesses sought ways to utilize AI for efficiency—reducing the time it takes to analyze documents. However, the realization that AI can further enhance workflows emerged; for instance, agents can autonomously analyze a set of documents and provide structured summaries, alleviating the need for manual review. This evolution emphasizes that agentic AI not only speeds up processes but can fundamentally change how businesses interact with their data.
Generative AI and Unstructured Data
Generative AI has transformed the handling of unstructured data, providing users with insights that were previously difficult to attain. Companies like Box have leveraged this technology to enhance capabilities such as document search, video transcription, and metadata extraction. By integrating generative AI directly into their platforms, businesses enable users to ask complex questions of their data and receive coherent, contextual answers. This shift allows organizations to turn vast amounts of unstructured content into actionable intelligence, fundamentally altering operations.
AI Agents and Complex Tasks
AI agents represent a new frontier in processing that enables the execution of more complex tasks than traditional AI applications. Unlike basic AI chat interfaces, agentic AI can perform multiclass functions by overseeing workflows—effectively managing various tasks from start to finish. This capability transforms user interactions with AI from simple query-response dynamics to encompassing full-service transaction handling, allowing for more sophisticated project management and execution. Enterprises are seeing the potential for AI agents to streamline operations by automating various workflows that typically require human intervention.
Adapting Organizational Structures
As companies integrate agentic AI into their operations, they may need to rethink their organizational structures and workflows. Engineering teams, for instance, must adapt to embrace the flexibility required to harness rapidly evolving AI technologies. Traditional practices may not suffice; instead, there is a need to encourage experimentation and responsiveness to create meaningful AI solutions that address real business challenges. A focus on collaboration and agile methodology within teams can help achieve this transformation, providing a strong foundation for integrating future AI advancements.
Quality Assurance in AI Applications
Maintaining quality in AI functionalities, particularly with generative AI, necessitates robust evaluation strategies to ensure accuracy and reliability. Companies are employing techniques like LLM grading, where AI evaluates its outputs against benchmarks, ensuring that responses meet user expectations. As the complexity of AI outputs arises, it becomes crucial to derive insights from both AI assessments and real user feedback to continuously improve the system's accuracy. Building a sustainable evaluation process allows organizations to maintain high standards in their AI applications and ensure they meet the needs of enterprises effectively.
On this episode of Deployed: The AI Product Podcast, we sit down with Ben Kus, CTO of Box, to unpack how they built a secure, scalable AI platform within their enterprise content management system. Ben shares candid insights from Box's journey integrating AI capabilities while maintaining enterprise-grade security for sensitive customer data. Learn practical strategies for evaluating AI quality without accessing customer data, building internal platforms that engineering teams want to use, and designing architecture that can evolve with rapid AI advances. Whether you're integrating LLMs into an established product or building new AI features, this conversation offers valuable lessons on balancing innovation with enterprise requirements. Join Ian, Co-Founder and CEO of Freeplay, for this in-depth discussion on scaling AI responsibly in the enterprise.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode