How to Counterfactually Query Your Learned Transition Model

I think maybe with the causal structure to learning your world model is more like you have the explainability, but then you still kind of need to go from the high dimensional observations into the high level variable space. Yeah, it makes me think about say model based reinforcement learning where you're trying to learn the transition dynamics as well as how to act in it. So once you have these transition models, can you just unroll a counterfactual by just you know, proposing some kind of action or is it because it's out of the distribution space that the model is just going to compounding air?