Jun-Yan Zhu

Associate Professor

Home Department: RI

Office: 216 Elliot Dunlap Smith Hall

Personal Homepage

Administrative Assistant: Christine Downey

Labs: Computer Vision @ CMU, Generative Intelligence Lab

At the Generative Intelligence Lab, we are studying the collaboration between Human Creators and Generative Models, with the goal of building intelligent machines capable of helping everyone tell their visual stories. We are studying the following questions:

Interaction between creators and generative models: How can we help creators control the model outputs more easily? We develop algorithms and interfaces for controllable visual synthesis (e.g., images, videos, 3D, visual+tactile)
Rewriting and searching generative models: How can creators repurpose existing models for new tasks, concepts, and styles? How could they rewrite the rules of models? Which model shall they use as a starting point?
Co-existence of creators and generative models: How can we allow creators to opt in or out of generative models at any time? If opting in, how do we credit creators for contributing training data?
Synthetic data generation with generative models: How can we use generative models to produce useful data for improving computer vision and robotics systems?

Biography: Jun-Yan Zhu is the Michael B. Donahue Associate Professor of Computer Science and Robotics at CMU’s School of Computer Science. Before joining CMU, he was a Research Scientist at Adobe Research and a postdoctoral researcher at MIT CSAIL. He obtained his Ph.D. from UC Berkeley and his B.E. from Tsinghua University. He studies computer vision, computer graphics, and computational photography. His current research focuses on collaboration between human creators and generative models. He has received a Sloan Research Fellowship, a Packard Fellowship for Science and Engineering, the NSF CAREER Award, Samsung AI Researcher of the Year, the ACM SIGGRAPH Outstanding Doctoral Dissertation Award, the UC Berkeley EECS David J. Sakrison Memorial Prize for outstanding doctoral research, and several industry faculty awards. His co-authored work has received the ICCV 2025 Best Paper Award (the Marr Prize), the Frontiers of Science Awards (2025, 2026), the ICRA 2024 Best Paper Award on Human-Robot Interaction, the CVPR Best Paper Finalists (2019, 2022), the NVIDIA Pioneer Research Award, and the SIGGRAPH 2019 Real-Time Live Best of Show Award and Audience Choice Award, and was named one of Popular Science’s 100 Greatest Innovations of 2019. His work and commentary have been covered in The New Yorker, The New York Times, BBC, CNN, Reuters, and The Economist.