Shape of Motion presents a groundbreaking method for reconstructing dynamic scenes from a single monocular video, overcoming the challenges of previous approaches by explicitly modeling 3D motion. Using a compact set of SE(3) motion bases and data-driven priors, the method decomposes scenes into rigidly-moving groups and achieves globally consistent representations. Results show the method’s state-of-the-art performance in 3D/2D motion estimation and novel view synthesis on dynamic scenes. However, the method struggles with fast motion and occlusions, relying on potentially inaccurate off-the-shelf methods like mono-depth estimation. The project acknowledges support from DARPA and IARPA, while also paying tribute to a beloved feline companion.
https://shape-of-motion.github.io/