Seminar
Neural Certificates for Safe Robotic System Planning and Control
Abstract: Achieving safety, scalability, and high performance in complex systems, such as multi-agent systems (MAS) control, is a central challenge in many real-world robotic deployments due to its computational complexity as a large-scale constrained optimal control problem. To address this, we introduce a novel graph control barrier function (GCBF) as a core tool for large-scale [...]
A Manipulation Journey
Abstract: The talk will revisit my career in manipulation research, focusing on projects that might offer some useful lessons for others. We will start with my beginnings at the MIT AI Lab and my MS thesis, which is still my most cited work, then continue with my arrival at CMU, a discussion with Allen Newell, [...]
Seeing Deep Inside Scattering Tissue Using Efficient, Noise-Robust Wavefront Shaping
Abstract: Scattering limits our ability to see inside biological tissue, as light penetration is severely distorted by tissue components with varying refractive indices. One promising method to overcome scattering aberration is wavefront shaping. This technique involves placing a spatial light modulator (SLM) in the microscope's optical path to correct the wavefront emitted from a point [...]
Bringing Dexterity to Robot Hands in the Real World
Abstract: Dexterous manipulation is a grand challenge of robotics, and fine manipulation skills are required for many robotics applications that we envision. In this overview talk, I will discuss my view of some major factors that contribute to dexterity and discuss how we can incorporate them into our robots and systems. Bio: Nancy Pollard [...]
Toward Generalist Humanoid Robots: Recent Advances, Opportunities, and Challenges
Abstract: In an era of rapid AI progress, leveraging accelerated computing and big data has unlocked new possibilities to develop generalist AI models. As AI systems like ChatGPT showcase remarkable performance in the digital realm, we are compelled to ask: Can we achieve similar breakthroughs in the physical world — to create generalist humanoid robots capable [...]
From Video Generation to Video World Models
Abstract: Video diffusion models have achieved remarkable success in content creation, yet they still fall short of simulating interactive worlds that respond to users in real time. This talk examines the fundamental challenges preventing these models from evolving into true world simulators. I will present a series of works — CausVid, Self-Forcing, MotionStream, and State-Space [...]
Just Asking Questions
Abstract: In the age of deep networks, "learning" almost invariably means "learning from examples". We train language models with human-generated text and labeled preference pairs, image classifiers with large datasets of images, and robot policies with rollouts or demonstrations. When human learners acquire new concepts and skills, we often do so with richer supervision, especially [...]
How to Coordinate Thousands of Robots Efficiently and Robustly
Abstract: Large-scale robot fleets are increasingly deployed in warehouses, factories, transportation systems, and emerging robotics applications. Coordinating hundreds or thousands of robots in shared, cluttered spaces creates fundamental challenges in maintaining safety, preventing deadlocks, and minimizing congestion. In this talk, I will present our recent work on scalable imitation learning methods for coordinating 10k robots, automatic environment [...]
What Can We Learn from a Million Models?
Abstract: Machine learning has transformed many fields by learning from large collections of data. Yet, it is rarely applied to its own outputs: the models themselves. Today, with millions of publicly available models, a natural question arises: what can we do with so many models? In this talk, I will motivate two core applications that [...]
Should we skip attention?
Abstract: Transformers are ubiquitous. They influence nearly every aspect of modern AI. However, the mechanics of their training remain poorly understood. This poses a problem for the field due to the immense amounts of data, computational power, and energy being invested in the training of these networks. I highlight a recent intriguing empirical result from [...]