How to Power the AI Boom (Robert Nishihara, Co-Founder & CEO of Anyscale)
Nov 17, 2023
auto_awesome
Robert Nishihara, Co-Founder & CEO of Anyscale, discusses building a distributed computing system for AI, the challenges of managing deployment models, hiring and building a team, importance of repetition and learning from founding stories, and optimism for the future of technology.
Anyscale aims to build the fastest and most cost-efficient infrastructure for running large language models (LLMs) and other AI applications, similar to how Microsoft was the underlying operating system for the PC era.
Ray, an open-source compute framework developed at UC Berkeley, is being used by major organizations like OpenAI and Cohere, highlighting the growing prominence of open source models in AI infrastructure.
Deep dives
The Rise of Generative AI
AI, specifically generative AI, is experiencing exponential growth and improvement, with models from Open AI and other sources rapidly advancing. The widespread adoption of multimodal models that integrate text, image, and video data is expected to lead to increased data intensity and infrastructure challenges. As the world moves towards multimodal models, the need for scalable compute and efficient infrastructure becomes paramount. Open source models are gaining prominence, and it is predicted that all models will soon reach a level of quality suitable for most applications.
The Vision Behind AnyScale
AnyScale's goal is to provide the fastest and most cost-efficient infrastructure for running large language models (LLMs) and other AI applications. Founded by Robert Nishihara, AnyScale builds on Ray, an open-source compute framework developed at UC Berkeley. Ray is already used by major organizations like OpenAI and Cohere. AnyScale's vision is to become the underlying operating system for the AI era, similar to how Microsoft was for the PC era.
The Transition from Academia to Startup
While in grad school, Robert Nishihara and his co-founders realized the need for better tools to support distributed computing. This led to the creation of Ray, an open-source project aimed at making scalable compute easy. As they continued their academic journey, the team recognized the limitations of their academic environment and felt the desire to move faster. Graduating from Berkeley, they decided to start AnyScale to accelerate the development and adoption of Ray as a scalable solution for AI infrastructure.
Challenges and Lessons Learned
Building AnyScale and navigating the transition from academia to startup came with its own set of challenges. The team had to learn and adapt to different roles within the company, such as product management, sales, marketing, and HR. Initially, they had to backpedal on deployment models, realizing the importance of simplicity and focus to ensure speed and execution. Communicating a clear strategy and aligning the team became a priority, emphasizing the importance of repetition and simplicity in messaging.
In this episode of B2BaCEO, I speak with Robert Nishihara, co-founder and CEO of Anyscale. Anyscale’s aspiration is to build the fastest, most cost-efficient infrastructure for running LLMs and AI workloads. When it is successful, Anyscale will be to the AI era what Microsoft was for the PC era: the underlying operating system on which all AI applications are developed and run.
Anyscale is built on Ray, an open-source compute framework that Robert and his co-founders developed as PhD students at UC Berkeley. Under the guidance of Professor Ion Stoica, who also co-founded Conviva and Databricks, the team sought to make distributed computing broadly accessible. Anyscale was then launched as a fully managed platform for Ray, making even the toughest problems in distributed computing easy for developers to tackle. Today, Anyscale is a billion-dollar business powering mission-critical AI use cases at companies like Amazon, Cohere, Hugging Face, NVIDIA, OpenAI, and Visa.
If you’ve been a PhD student at UC Berkeley, created a popular open-source framework, and built a billion-dollar business on top of it, you’ve likely learned a thing or two along the way. Robert’s story offers valuable lessons for fellow founders and builders at all stages of the startup journey.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode