SK Learn - One Stop Shop Versus Spark?

MLflow is built as its own project. And it's built in a diagnostic way where it's not Spark centric. It's just a framework that you can build whatever flavor of model you want. We're actually using sklearn library for doing the validation checks on that instead of using Spark because we don't need to. But the principle of how that was built was we're not taking sides on any packages that people are doing. Like, hey, we can support these two different data types or actually there's like five data types for that API.