In this paper, the authors address the challenge of high-fidelity and real-time view synthesis of dynamic 3D scenes at 4K resolution. While some methods have achieved impressive rendering quality, their speed is limited when dealing with high-resolution images. To overcome this limitation, the authors propose a 4D point cloud representation called 4K4D that supports hardware rasterization and allows for unprecedented rendering speed. This representation is based on a 4D feature grid, which ensures regularity and robust optimization. Additionally, the authors introduce a new hybrid appearance model that enhances rendering quality without compromising efficiency. They also present a differentiable depth peeling algorithm for learning the proposed model from RGB videos. Experimental results demonstrate that the proposed representation can be rendered at over 400 FPS on the DNA-Rendering dataset at 1080p resolution and 80 FPS on the ENeRF-Outdoor dataset at 4K resolution, achieving a rendering quality that surpasses previous methods and setting a new standard in the field.
https://zju3dv.github.io/4k4d/