I'm wondering if you can talk to some of the different tasks that are involved in being able to actually take some source piece of data and generate a vector embedding from it. I think when most folks think about embedding generation, they like to think that as just a one step process. There's also ways where you can do it frame by frame or you can chop up a video into, let's say, 10 frame segments and generate embeddings based off of those. And then maybe we have some sort of summarization algorithm later on, which will turn all of those into a single, maybe larger embedding.
Summary
Data is one of the core ingredients for machine learning, but the format in which it is understandable to humans is not a useful representation for models. Embedding vectors are a way to structure data in a way that is native to how models interpret and manipulate information. In this episode Frank Liu shares how the Towhee library simplifies the work of translating your unstructured data assets (e.g. images, audio, video, etc.) into embeddings that you can use efficiently for machine learning, and how it fits into your workflow for model development.
Announcements
- Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
- Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
- Your host is Tobias Macey and today I’m interviewing Frank Liu about how to use vector embeddings in your ML projects and how Towhee can reduce the effort involved
Interview
- Introduction
- How did you get involved in machine learning?
- Can you describe what Towhee is and the story behind it?
- What is the problem that Towhee is aimed at solving?
- What are the elements of generating vector embeddings that pose the greatest challenge or require the most effort?
- Once you have an embedding, what are some of the ways that it might be used in a machine learning project?
- Are there any design considerations that need to be addressed in the form that an embedding takes and how it impacts the resultant model that relies on it? (whether for training or inference)
- Can you describe how the Towhee framework is implemented?
- What are some of the interesting engineering challenges that needed to be addressed?
- How have the design/goals/scope of the project shifted since it began?
- What is the workflow for someone using Towhee in the context of an ML project?
- What are some of the types optimizations that you have incorporated into Towhee?
- What are some of the scaling considerations that users need to be aware of as they increase the volume or complexity of data that they are processing?
- What are some of the ways that using Towhee impacts the way a data scientist or ML engineer approach the design development of their model code?
- What are the interfaces available for integrating with and extending Towhee?
- What are the most interesting, innovative, or unexpected ways that you have seen Towhee used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Towhee?
- When is Towhee the wrong choice?
- What do you have planned for the future of Towhee?
Contact Info
Parting Question
- From your perspective, what is the biggest barrier to adoption of machine learning today?
Closing Announcements
- Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links
The intro and outro music is from
Hitman’s Lovesong feat. Paola Graziano by
The Freak Fandango Orchestra/
CC BY-SA 3.0
