Databricks and Distributed Time Series Data

Ben: A couple of our customers were trying to do something that everybody's always wanted to do with generating forecasts. They started using MLflow and they start writing out for you know 500,000 models each of those 500,000 events like the artifacts themselves. It quickly overwhelmed the systems and most of their computation time was spent just writing to a database. That morphed into us asking internally in engineering could we make this run like stupidly fast? Could we write some way of making it so that we could do a million profit models in less than an hour? We came up with an open source package which does handle the aspect of interfacing with tracking services on MLflow.