Towards Controllable Sampling and Diverse Score Distillation in Diffusion Models

Abstract

Denoising diffusion models have emerged as a powerful paradigm for gen-
erative modeling, which has been widely used for perception, generation,
and action. These models can be utilized through sampling or score
distillation; however, existing methods lack controllability in sampling and
suffer from limited diversity in score distillation. In this thesis, we propose
two complementary mechanisms to enhance the controllability and diver-
sity of diffusion-based generation. First, we introduce a score rescaling
approach that enables users to steer the sampling diversity of diffusion
models without requiring any modifications to training. This method,
validated across diverse tasks—including pose estimation, depth predic-
tion, image generation, and robotic manipulation—demonstrates that
adjusting the sampling distribution can lead to significant performance
improvements. Second, we address the inherent mode-seeking limitation in
score distillation for 3D optimization. Inspired by the diffusion sampling
process, we propose a novel formulation that encourages optimization to
follow diverse generation paths, thereby improving sample diversity while
maintaining fidelity. We further introduce an approximation to adapt this
formulation to practical settings where generation trajectories cannot be
strictly preserved. We empirically validate our approach across multiple
applications, including 2D optimization, text-to-3D generation, and single-
view reconstruction. Together, these contributions advance the flexibility
and effectiveness of diffusion models, broadening their applicability in
generative modeling and beyond.