Abstract:
Dynamic scene reconstruction is fundamental to real-time photorealistic rendering for applications such as virtual reality, telepresence, and robotic perception. Although early methods like Neural Radiance Fields (NeRFs) have achieved impressive results in novel view synthesis, their heavy computational demands severely hinder real-time deployment. This thesis presents four novel methods that collectively advance dynamic 3D scene reconstruction, offering improvements in speed, controllability, fidelity, and robustness in unconstrained settings.

First, we propose DyLiN, the first Light Field Network (LFN) framework capable of modeling dynamic scenes with topological variation. DyLiN incorporates a ray-based deformation MLP and hyperspace lifting network, trained via knowledge distillation from dynamic NeRFs, achieving superior visual quality with order-of-magnitude faster inference. Next, we develop CoGS, an extension of 3D Gaussian Splatting (3DGS) that enables dynamic, controllable scene modeling without dense supervision. CoGS leverages explicit Gaussian representations alongside novel regularization losses to allow reasonable manipulation of scene elements.

We further present GS-LK, a hybrid method integrating Gaussian Splatting with a differentiable Lucas-Kanade pipeline for improved geometric consistency in monocular video. Finally, Focus4DGS introduces uncertainty-aware modeling in 4D Gaussian Splatting to enhance robustness in uncontrolled environments. It achieves this by separating high-confidence reconstructions from ambiguous motion regions, effectively finding the subset of dynamic objects that can be reconstructed from those that cannot.

Committee:
Prof. László Jeni (advisor)
Prof. Jun-Yan Zhu
Mosam Dabhi