Exploring the Sorting Hat Project: Enhancing Data Preparation Automation

Today we’re joined by Arun Kumarm, an associate professor at UC San Diego. We had the pleasure of catching up with Arun prior to the Workshop on Databases and AI at NeurIPS 2021, where he delivered the talk “The New DBfication of ML/AI.” In our conversation, we explore this “database-ification” of machine learning, a concept analogous to the transformation of relational SQL computation. We discuss the relationship between the ML and database fields and how the merging of the two could have positive outcomes for the end-to-end ML workflow, and a few tools that his team has developed, Cerebro, a tool for reproducible model selection, and SortingHat, a tool for automating data prep, and how tools like these and others affect Arun’s outlook on the future of machine learning platforms and MLOps.

The complete show notes for this episode can be found at twimlai.com/go/553