RI PhD Thesis Defense - Nupur Kumari - Robotics Institute Carnegie Mellon University

PhD Thesis Defense

April

21
Tue
Nupur Kumari PhD Student Robotics Institute,
Carnegie Mellon University
Tuesday, April 21
2:00 pm to 3:30 pm
NSH 3305
RI PhD Thesis Defense – Nupur Kumari
Who: Nupur Kumari
Date: April 21, 2026
Time: 2:00 PM
Location: NSH 3305 (3rd floor)

Type: Ph.D. Thesis Defense

Title: Customizing Text-to-Image Diffusion Models
Abstract
Recent advances in Generative AI highlight its growing potential to reshape content creation workflows, with the potential to support highly personalized use cases for everyday users and independent creators. However, realizing these applications, requires going beyond text conditioning for which current large-scale models are typically pre-trained for, owing to the abundance of text-annotated data on the web. In contrast, practical applications often involve modifying existing content using multimodal cues, where a core challenge is the scarcity of paired input–output data.

In this talk, I will briefly overview my research on post-training methods that address this challenge along three directions:
(1) Learning from few samples, via parameter-efficient fine-tuning of the pre-trained model on a few user-provided data. This is computationally efficient but requires fine-tuning for each new task instance.

(2) Learning from large-scale synthetic datasets, where I propose a pipeline to create large-scale paired datasets using the capabilities of pre-trained generative models themselves to enable end-to-end training. However, constructing such datasets requires careful curation, filtering, and risk becoming outdated as base pre-trained models evolve.

(3) Learning from discriminative models, which leverages vision–language models to evaluate task success and provide direct gradient-based feedback to the generative model. We show its potential as a scalable and robust framework for efficient customization of generative models for downstream tasks without relying on paired synthetic datasets.

Thesis Committee:
Jun-Yan Zhu (Chair)
Deva Ramanan
Shubham Tulsiani
Phillip Isola (MIT)