AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is There a Better Approach to Parameter Efficient Training?
There hasn't been that much work on that front, I would say this is something that needs a lot more attention. It's not like a layered architecture, like a CNN, where you can just chop off the end layers and retrain from that point. So that kind of incremental updating doesn't work so easily. And also sort of fundamental problems that need to be addressed by language models.