Is There a Way to Improve Model Performance?

LSTMs are not known for being amazingly good at composition. We don't know why when you do interaction again, when the models talk to each other and try to do well on the task, you don't just immediately forget about everything you've learned. And so I came across another paper called Knowledge Evolution that proposed this very cool sounding idea of how knowledge within a model evolves towards better generalization. So they basically generate a random binary mask which is the same shape as the model. That's a hyperparameter you can tune, but they fix it after every generation.