How Shopify Built A Machine Learning Platform That Encourages Experimentation
AI Engineering Podcast
00:00
Introduction
Isaac Vides is a staff data engineer at Shopify. He works on building Merlin, the company's new machine learning platform. Isaac talks about his background in DevOps and how he became interested in ML.
Transcript
Play full episode
Transcript
Episode notes
Summary
Shopify uses machine learning to power multiple features in their platform. In order to reduce the amount of effort required to develop and deploy models they have invested in building an opinionated platform for their engineers. They have gone through multiple iterations of the platform and their most recent version is called Merlin. In this episode Isaac Vidas shares the use cases that they are optimizing for, how it integrates into the rest of their data platform, and how they have designed it to let machine learning engineers experiment freely and safely.
Announcements
Parting Question
Shopify uses machine learning to power multiple features in their platform. In order to reduce the amount of effort required to develop and deploy models they have invested in building an opinionated platform for their engineers. They have gone through multiple iterations of the platform and their most recent version is called Merlin. In this episode Isaac Vidas shares the use cases that they are optimizing for, how it integrates into the rest of their data platform, and how they have designed it to let machine learning engineers experiment freely and safely.
Announcements
- Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
- Your host is Tobias Macey and today I'm interviewing Isaac Vidas about his work on the ML platform used by Shopify
- Introduction
- How did you get involved in machine learning?
- Can you describe what Shopify is and some of the ways that you are using ML at Shopify?
- What are the challenges that you have encountered as an organization in applying ML to your business needs?
- Can you describe how you have designed your current technical platform for supporting ML workloads?
- Who are the target personas for this platform?
- What does the workflow look like for a given data scientist/ML engineer/etc.?
- What are the capabilities that you are trying to optimize for in your current platform?
- What are some of the previous iterations of ML infrastructure and process that you have built?
- What are the most useful lessons that you gathered from those previous experiences that informed your current approach?
- How have the capabilities of the Merlin platform influenced the ways that ML is viewed and applied across Shopify?
- What are the most interesting, innovative, or unexpected ways that you have seen Merlin used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Merlin?
- When is Merlin the wrong choice?
- What do you have planned for the future of Merlin?
Parting Question
- From your perspective, what is the biggest barrier to adoption of machine learning today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Shopify
- Shopify Merlin
- Vertex AI
- scikit-learn
- XGBoost
- Ray
- PySpark
- GPT-3
- ChatGPT
- Google AI
- PyTorch
- Dask
- Modin
- Flink
- Feast Feature Store
- Kubernetes
The AI-powered Podcast Player
Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
