Are You Thinking About Playing Around With Pre-Trained Encoders in RL?

I'm not really too familiar with the literature, but using foundation models from language and from vision try to use that in RL. I'd be curious to combine like a pre-trained encoder that's pre-trained on a whole bunch of different visual data with your like auxiliary self-survised plus policy network. And yeah, another direction for that is also how can we actually leverage these tools in a model-based setting? If you have a model of the environment, you can actually start doing data augmentation, not just in observations, but also in the action space or in the dynamics of itself. What do you feel like are some controversial opinions that other people maybe don't agree