AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is Automation a Good Idea?
Everything we write into the object store is immutable by design. We use content addressable storage to not only point to the right block, but actually even do things like deduplication. If I'm running multiple pipelines that are actually trying to run the same code and generate the same piece of data, you can do things that deduplicate all of that. The general idea is, yes, anything you can parallelize, you should.