AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is Retrained Learning Used to Predict the Next Word in a Sequence?
T der has the nice benefit of having a sort of rolling in information from the future back into previous predictions. It was used alongside the typical lc m predicting the next word in a sequence. O the same time as the ellis tem is learning to predict the next word, we constrain one of the hidden representations to also be useful for predicting the part of speech of the nextword in the sequence. So actually taking that label, what is the next part of speech in the sequence, putting that label in the hidden representation in the next layer, and then using it for prediction of the next wirge saw an improvement.