E114: How OctoML Helps Developers Build with Llama 2 & Stable Diffusion
Nov 7, 2023
auto_awesome
Tianqi Chen, Co-Founder and Chief Technologist of OctoML, discusses the importance of supporting multiple models, advancements in Llama and Stable Diffusion, building TVM and OctoML communities, predictions on GenAI in enterprise, and challenges in starting a MLAI company.
OctoML was founded to continue the development of the TVM open-source project and provide a compute platform for accelerating the deployment of AI models into production.
TVM offers unique technical capabilities to optimize ML models for various platforms, addressing pain points in model optimization and supporting standard and specialized variants.
Deep dives
OctoML's Origin Story and Apache TVM
OctoML was born out of the recognition of the challenges faced when deploying machine learning (ML) models on different environments. This led to the creation of Apache TVM, a project aimed at alleviating the pain points of ML product realization. The team behind OctoML started the company to continue the development of the TVM open-source project and provide a compute platform for accelerating the deployment of AI models into production.
The Unique Approach of TVM as a Compiler
TVM is not a typical ML framework, but rather a compiler that optimizes ML models for various platforms. It offers unique technical capabilities to enable productivity cycles and adapt to the evolving landscape of ML engineering. The motivation behind TVM was to automate the process of building and optimizing models, making the solution more agnostic and scalable. TVM started as an early adopter in the ML compilation space and has since grown to support a wide range of models and platforms.
The Growth Trajectory and Success of TVM
TVM has gained traction due to its unique technical advantages and the growth of the ML field. Its open-source nature and continuous innovation have resonated with users and the community. TVM has found success by addressing pain points in ML model optimization and supporting both standard models and specialized variants. The platform's versatility and ability to quickly integrate new hardware and models have made it invaluable for users looking to achieve optimal performance and productivity.
OctoML's Products and The Future of ML Acceleration
OctoML offers a compute platform for accelerating AI models and optimizing their performance and cost efficiency. They focus on features such as speed, cost, hardware agnosticism, and accessibility. OctoML's products include self-optimizing compute services, fine-tuning capabilities, and the ability to deploy ML models on various devices. Looking ahead, OctoML anticipates trends such as diversified backend execution, hybrid AI models, and fine-tuning becoming more prevalent. They aim to continue building innovative solutions that enable users to leverage the full potential of ML acceleration.
Tianqi Chen is Co-Founder and Chief Technologist of OctoML, the compute infrastructure platform for tuning and running generative models in the cloud. OctoML was founded by the creators of Apache TVM, the machine learning compiler framework for CPUs, GPUs, and accelerators.
OctoML has raised $132M from investors including Amplify, Addition, Madrona, and Tiger.
In this episode, we discuss the importance of supporting multiple models, the advancements from LLaMA and Stable Diffusion this year, building the TVM and OctoML communities, predictions on GenAI in the enterprise (hybrid ML, for example), whether GenAI is over-invested in & more!
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode