Abstract:

Generative visual AI has advanced by scaling data and compute, but its next bottleneck is oversight: the expert signals that evaluate, reward, and teach models what “good” looks like. Providing such oversight is increasingly difficult because foundation vision-language models now match or surpass most humans at the skills being judged.

This thesis develops scalable oversight for generative visual AI, turning limited expert judgment into abundant and reliable evaluation, reward, and training signals. First, I present VQAScore and GenAI-Bench, evaluation standards for text-to-visual generation adopted by 100+ frontier labs such as DeepMind, and show that a stronger judge directly improves generation through inference-time scaling. Second, I extend oversight to video, where precise language must be built rather than collected: CameraBench (NeurIPS’25 Spotlight) defines a structured cinematic vocabulary with professional creators, and CHAI (CVPR’26 Highlight) introduces critique-based human-AI oversight in which experts critique model drafts rather than write from scratch. One recipe of specification, oversight, and post-training simultaneously improves captioning, reward modeling, and video generation, enabling a small open model to surpass proprietary models like GPT and Gemini. Finally, I present Moodio, a deployed AI film studio with hundreds of daily active users, whose cinematic video retrieval outperforms SOTA embedding models and measurably helps creators generate more compelling videos. I conclude with ongoing work on personalized oversight for end-to-end creative agents, toward visual storytelling for everyone.

Thesis Committee Members:

• Deva Ramanan (Chair)
• Deepak Pathak
• Graham Neubig
• Ali Farhadi (University of Washington and Microsoft)