Towards Geometric Reasoning for Dynamic and Unconstrained 3D Scenes

Abstract

Dynamic scene reconstruction is fundamental to real-time photorealistic rendering for applications such as virtual reality, telepresence, and robotic perception. Although early methods like Neural Radiance Fields (NeRFs) have achieved impressive results in novel view synthesis, their heavy computational demands severely hinder real-time deployment. This thesis presents four novel methods that collectively advance dynamic 3D scene reconstruction, offering substantial improvements in speed, controllability, visual fidelity, and the robustness to handle challenging, unconstrained environments.

First, we propose DyLiN, the first Light Field Network (LFN) framework capable of modeling dynamic scenes with topological variation. DyLiN incorporates a ray-based deformation MLP and hyperspace lifting network, trained via knowledge distillation from dynamic NeRFs, achieving superior visual quality with order-of-magnitude faster inference. Next, we develop CoGS, an extension of 3D Gaussian Splatting (3DGS) that enables dynamic, controllable scene modeling without dense supervision. CoGS leverages explicit Gaussian representations alongside novel regularization losses to allow reasonable manipulation of scene elements.

We further present GS-LK, a hybrid method integrating Gaussian Splatting with a differentiable Lucas-Kanade pipeline for improved geometric consistency in monocular video. Finally, Focus4DGS introduces uncertainty-aware modeling in 4D Gaussian Splatting to enhance robustness in uncontrolled environments. It achieves this by separating high-confidence reconstructions from ambiguous motion regions, effectively finding the subset of dynamic objects that can be reconstructed from those that cannot.

Collectively, these contributions advance the field toward real-time, high-fidelity dynamic scene modeling by addressing core limitations of existing methods. Extensive experiments on synthetic and real datasets demonstrate superior performance in rendering speed, image quality, and scene controllability, making these methods suitable for deployment in interactive and resource-constrained applications.