AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
How to Control a Convergence?
The idea is that beyond just resulting in a large error, it wolt prevent convergens altogether. So my colleagues actually propose two methods. The first one is a dampening regularizer which pushes the weights to be close to the centre of beans instead of quantensation tresl. Other approach that they have is about freezing the weights. What does that mean? This means, if you see that the weights are oscillating, then you freeze them. You just don't update them, because you know that this will have a negative impact.