Where ML and DevOps Meet - ML 108
Adventures in Machine Learning
00:00
Databricks - How to Scale a DataBricks Cluster
The way I would first approach it is if it is inherently parallelizable and you don't have any cross dependencies across rows. You can kind of get away with a really simple, just create a synthetic grouping key on your data. And then once it's in the executor, the worker on Spark, you can do whatever you want. So that's that's what you can do with Databricks.
Transcript
Play full episode
