Efficient Deployment of Models at the Edge // Krishna Sridhar // #284

8 snips

Jan 17, 2025

In this engaging discussion, Krishna Sridhar, an engineering leader at Qualcomm and former co-founder of Tetra AI, dives into the efficient deployment of AI models at the edge. He shares insights on using Qualcomm AI Hub to optimize models for on-device performance, highlighting its application in real-time sports tracking and mobile photography. Krishna also explores the balance between hardware and software optimization in modern devices. Plus, he reveals how innovations in edge computing are transforming everyday AI applications while ensuring user privacy.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

From Oil Taxes to AI

Krishna Sridhar's PhD thesis focused on optimizing taxes for oil companies' drilling operations.
He later realized he wanted to contribute to more socially beneficial applications of AI.

INSIGHT

AI Behind Smartphone Photos

Modern smartphones execute approximately 25 AI models when taking a picture.
These models handle various tasks like framing, face detection, and recoloring within milliseconds.

INSIGHT

On-Device AI Challenges

On-device AI allows for innovative, privacy-sensitive features.
The challenge lies in mapping fast-evolving AI and hardware ecosystems.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Krishna Sridhar is an experienced engineering leader passionate about building wonderful products powered by machine learning. Efficient Deployment of Models at the Edge // MLOps Podcast #284 with Krishna Sridhar, Vice President of Qualcomm.

Big shout out to Qualcomm for sponsoring this episode! // Abstract Qualcomm® AI Hub helps to optimize, validate, and deploy machine learning models on-device for vision, audio, and speech use cases. With Qualcomm® AI Hub, you can: Convert trained models from frameworks like PyTorch and ONNX for optimized on-device performance on Qualcomm® devices. Profile models on-device to obtain detailed metrics including runtime, load time, and compute unit utilization. Verify numerical correctness by performing on-device inference. Easily deploy models using Qualcomm® AI Engine Direct, TensorFlow Lite, or ONNX Runtime. The Qualcomm® AI Hub Models repository contains a collection of example models that use Qualcomm® AI Hub to optimize, validate, and deploy models on Qualcomm® devices. Qualcomm® AI Hub automatically handles model translation from source framework to device runtime, applying hardware-aware optimizations, and performs physical performance/numerical validation. The system automatically provisions devices in the cloud for on-device profiling and inference. The following image shows the steps taken to analyze a model using Qualcomm® AI Hub. // Bio Krishna Sridhar leads engineering for Qualcomm™ AI Hub, a system used by more than 10,000 AI developers spanning 1,000 companies to run more than 100,000 models on Qualcomm platforms. Prior to joining Qualcomm, he was Co-founder and CEO of Tetra AI which made its easy to efficiently deploy ML models on mobile/edge hardware. Prior to Tetra AI, Krishna helped design Apple's CoreML which was a software system mission critical to running several experiences at Apple including Camera, Photos, Siri, FaceTime, Watch, and many more across all major Apple device operating systems and all hardware and IP blocks. He has a Ph.D. in computer science from the University of Wisconsin-Madison, and a bachelor’s degree in computer science from Birla Institute of Technology and Science, Pilani, India. // MLOps Swag/Merch https://shop.mlops.community/ // Related Links Website: https://www.linkedin.com/in/srikris/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Krishna on LinkedIn: https://www.linkedin.com/in/srikris/