GameNGen is a groundbreaking real-time game engine powered entirely by a neural model, allowing for interactive simulations of games like DOOM at high quality. The neural model achieves impressive results, with next frame prediction comparable to lossy JPEG compression. Human raters struggle to distinguish between actual gameplay and simulations, highlighting the model’s accuracy. The training process involves an RL-agent playing the game to collect data, followed by training the diffusion model to predict future frames based on past actions and observations. Surprisingly, corrupting context frames with noise during training helps maintain visual stability over long periods. This innovative approach to game engine development showcases the power of neural models in the gaming industry.
https://gamengen.github.io