xLSTM is a new Recurrent Neural Network architecture based on LSTM, with Exponential Gating, normalization techniques, and a Matrix Memory, showing promising results in Language Modeling. Minimal installation is required, with detailed instructions provided. The xLSTM can be used in various applications, such as the xLSTMBlockStack for non-language tasks and the xLSTMLMModel for token-based applications. Experimental tasks showcase the benefits of xLSTM over mLSTM and sLSTM, with tasks like Parity and Multi-Query Associative Recall. The codebase is available on GitHub and should be cited according to the provided format. Unexpectedly, the training loop lacks early stopping or test evaluation.
https://github.com/NX-AI/xlstm