AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Benefits of a Simulated a-B Test
The main thing is that the simulator does not have to match reality, but the simulator has its own sense of something like a fully online experiment. And then we can really use these simple simulation environments to learn more about when certain methods are better or worse than some others. The ranking that you will get from your simulated A-B test might not be perfectly aligned with the real world. But for a system with similar learning dynamics as the real world, this method is able to actually get much better performance than this. And that is worth something.