Shopify has hundreds of machine learning engineers and data scientists that we have to support with this platform. Some companies just give the Kubernetes access to their data scientists so they can run QPCTL commands. For us, we wanted to reduce the amount of access that we are giving to different users who may not feel comfortable in running these type of commands. We included everything inside of an API that we can leverage for both our users and also for our automatic pipelines.
Summary
Shopify uses machine learning to power multiple features in their platform. In order to reduce the amount of effort required to develop and deploy models they have invested in building an opinionated platform for their engineers. They have gone through multiple iterations of the platform and their most recent version is called Merlin. In this episode Isaac Vidas shares the use cases that they are optimizing for, how it integrates into the rest of their data platform, and how they have designed it to let machine learning engineers experiment freely and safely.
Announcements
- Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
- Your host is Tobias Macey and today I'm interviewing Isaac Vidas about his work on the ML platform used by Shopify
Interview
- Introduction
- How did you get involved in machine learning?
- Can you describe what Shopify is and some of the ways that you are using ML at Shopify?
- What are the challenges that you have encountered as an organization in applying ML to your business needs?
- Can you describe how you have designed your current technical platform for supporting ML workloads?
- Who are the target personas for this platform?
- What does the workflow look like for a given data scientist/ML engineer/etc.?
- What are the capabilities that you are trying to optimize for in your current platform?
- What are some of the previous iterations of ML infrastructure and process that you have built?
- What are the most useful lessons that you gathered from those previous experiences that informed your current approach?
- How have the capabilities of the Merlin platform influenced the ways that ML is viewed and applied across Shopify?
- What are the most interesting, innovative, or unexpected ways that you have seen Merlin used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Merlin?
- When is Merlin the wrong choice?
- What do you have planned for the future of Merlin?
Contact Info
Parting Question
- From your perspective, what is the biggest barrier to adoption of machine learning today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links
The intro and outro music is from
Hitman's Lovesong feat. Paola Graziano by
The Freak Fandango Orchestra/
CC BY-SA 3.0
