This chapter discusses various approaches to validate machine learning systems for safety-critical applications, the challenges of understanding model decisions, the importance of monitoring and measuring changes over time, the management of change and model evolution, the parallels between the DevOps movement and validation, and the challenges of designing and verifying machine learning systems.
Summary
Software systems power much of the modern world. For applications that impact the safety and well-being of people there is an extra set of precautions that need to be addressed before deploying to production. If machine learning and AI are part of that application then there is a greater need to validate the proper functionality of the models. In this episode Erez Kaminski shares the work that he is doing at Ketryx to make that validation easier to implement and incorporate into the ongoing maintenance of software and machine learning products.
Announcements
- Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
- Your host is Tobias Macey and today I'm interviewing Erez Kaminski about using machine learning in safety critical and highly regulated medical applications
Interview
- Introduction
- How did you get involved in machine learning?
- Can you start by describing some of the regulatory burdens placed on ML teams who are building solutions for medical applications?
- How do these requirements impact the development and validation processes of model design and development?
- What are some examples of the procedural and record-keeping aspects of the machine learning workflow that are required for FDA compliance?
- What are the opportunities for automating pieces of that overhead?
- Can you describe what you are doing at Ketryx to streamline the development/training/deployment of ML/AI applications for medical use cases?
- What are the ideas/assumptions that you had at the start of Ketryx that have been challenged/updated as you work with customers?
- What are the most interesting, innovative, or unexpected ways that you have seen ML used in medical applications?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Ketryx?
- When is Ketryx the wrong choice?
- What do you have planned for the future of Ketryx?
Contact Info
Parting Question
- From your perspective, what is the biggest barrier to adoption of machine learning today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
The intro and outro music is from
Hitman's Lovesong feat. Paola Graziano by
The Freak Fandango Orchestra/
CC BY-SA 3.0
