Is There a Transition Point for Distributed Training?

I recommend people just always use scripts directly, right? Like write your own Python scripts, do it that way. So I definitely see optimization being important to training and influencing. But how about just sort of deep into early development, we were still experimenting. And then you get something you like and you're going to make this transition to production. What sort of concerns come to mind or do you have to kind of keep it in good shape?"