The author’s five-year-old daughter initially thought math was stupid, but the author and their partner have been successful in making her more excited about math. She has developed an interest in fractals and enjoys watching deep zoom videos of Mandelbrot and Mandelbulb fractal sets. This sparked the author’s curiosity about the connection between fractals and artificial neural networks. They discovered that there are similarities in how fractals and neural networks are generated, both involving repeatedly applying a function to its own output. They found that the boundary between hyperparameters for successful or unsuccessful neural network training exhibits fractal structure. The author provides visualizations and videos of these fractal landscapes. They also explain the connection between fractals and neural networks and discuss the challenges of meta-learning. Overall, the author finds the fractals generated by neural networks to be beautiful and mesmerizing.
https://sohl-dickstein.github.io/2024/02/12/fractal.html