In this article, the author, a Machine Learning engineer, discusses the process of automatic differentiation, which is crucial for training neural networks. They introduce a Tensor class that allows for symbolic calculation on variables and demonstrate how to perform addition and multiplication operations using this class. They then introduce a binary tree structure to track the calculation history of each tensor. The author also explains how to add automatic differentiation by implementing the derivative for basic operations. They demonstrate the process by deriving a function and calculating the partial derivatives using both their Tensor class and the Sympy library. The author acknowledges possible improvements and optimizations for the Tensor class.
https://vmartin.fr/understanding-automatic-differentiation-in-30-lines-of-python.html