LeanRL is a lean PyTorch-based library that aims to speed up Reinforcement Learning (RL) training times by providing optimized implementations of popular RL algorithms. It is a fork of CleanRL, focusing on using PyTorch 2 features like torch.compile and cudagraphs to run RL scripts at full speed with minimal impact on user experience. The library provides single-file implementations, optimized PyTorch scripts, and tools like tensordict and torch.vmap for faster execution. By utilizing torch.compile and cudagraphs, LeanRL achieves speed-ups of up to 6.8x in PyTorch speed compared to the CleanRL implementation. The library does not support poetry but offers tips to accelerate code and instructions on how to get started. The repo is MIT licensed and acknowledges CleanRL as its precursor.
https://github.com/pytorch-labs/LeanRL