Pre-Trained Models - Can I Apply This to Any Kind of Domain?

Roy: Can we come up with something around this for pre-training? So say you have a very large corpus as a common crawl or the internet. And then can I kind of say, well, in the end, I'm actually just going to use this in finance or in sports?" He says it's definitely an interesting line of research. "I mean, it really depends on your motivation because I mean, if you have the model already trained, you can think, okay, why not use it for your medical"