The Return of LSTM: Sepp Hochreiter's Extended XLSTM Model

The long short-term memory (LSTM) model, invented by Sepp Hochreiter in the 1990s, revolutionized language processing applications, including Siri and Google Translate. However, the emergence of transformers, particularly the generative pre-trained transformer (GPT) architecture, led to LSTM being largely overshadowed due to its ability to handle data in a more parallelizable manner. Hochreiter has now introduced the extended LSTM (XLSTM), which aims to retain the functional advantages of LSTM while addressing scalability issues faced during training and application. This new model boasts improvements in memory efficiency, allowing for improved processing of longer sequences, thus potentially regaining relevance in modern AI applications.