Is There an Alternative to Offline RL?

I don't think the problem with clever prompting is that it's too simple or primitive, I think the problem might actually be that it might be too complex. What do you think of other types of reinforcement learning setups? We want simplicity because simplicity makes it easy to make things work at a large scale. These language models, aside from what we talked about earlier with translating images into language, can we use the embeddings that are learned for robotics type problems?