SmokeSeer: 3D Gaussian Splatting for Smoke Removal and Scene Reconstruction

Abstract

The presence of smoke in real-world scenes can severely degrade the qual-
ity of images and hamper visibility. Recently introduced methods for
image restoration either rely on data-driven priors that are susceptible to
hallucination, or are limited to static low-density smoke. We introduce
See-Through Smoke, a method to perform simultaneous 3D scene recon-
struction and smoke removal from a video capturing multiple views of a
scene. To achieve this task, our method uses thermal and RGB images,
leveraging the fact that the reduced scattering in thermal images enables
us to see through the smoke. We build upon 3D Gaussian splatting
to fuse information from the two image modalities, and decompose the
scene explicitly into smoke and non-smoke components. Unlike prior
approaches, See-Through Smoke handles a broad range of smoke densities
and can adapt to temporally varying smoke. We validate our approach on
synthetic data and introduce a new real-world multi-view smoke dataset
with RGB and thermal images.