Abstract: The physical properties and dynamics that decide how an object or environment responds to a robot's actions are often impossible to determine from visual observation alone. An object's mass distribution and friction, the kinematics of an articulated object: these latent factors dictate the correct action, yet they leave little or no trace in a single [...]

Abstract: Visual imitation learning is a promising approach to training robot manipulation policies capable of completing a wide variety of tasks. A key requirement for these manipulation policies is to exhibit robust generalization capabilities when deployed in the real world, where the objects, scenes, and sensors a robot encounters differ from those seen during training. In [...]

Abstract: Hierarchical imitation learning has become an effective approach for robotic manipulation: a high-level policy predicts a sub-goal end-effector pose, while a low-level policy executes the actions needed to reach it. This decomposition improves generalization and provides an interpretable interface, but the design of the high-level goal predictor remains an open question. This thesis studies [...]

Abstract: Generalist robot policies, vision-language-action models that combine a large pretrained vision-language model backbone with a diffusion or flow-matching action head, are increasingly capable, yet hard to deploy in the real world. Three gaps separate such a policy from a deployable one: a data gap (adapting to a new task still demands task-specific teleoperation data), an inference gap (the policy [...]

Abstract: Humanoid robots operating in human environments must manipulate articulated objects under contact and kinematic constraints that human demonstrations do not satisfy. That mismatch makes the human--humanoid embodiment gap the central bottleneck for learning from human data: robot demonstrations are expensive and sparse, while human demonstrations inhabit a different state-action space and often violate robot [...]

Abstract: Curiosity allows animals and humans to learn through interaction without explicit instruction. This thesis asks how principles of natural curiosity can be translated into embodied agents, and what kinds of world models are needed as bodies, action spaces, and environments become more complex. We study this progression in simulation, using animal behavior and neural dynamics [...]

Abstract: Autonomous vehicles (AVs) are being deployed at scale today, with companies like Waymo achieving upward of 500,000 passenger rides per week. Two of the largest remaining problems in the field are 1) building a system that generalizes across the long-tail of edge cases that are represented few or no times within the training data [...]

Abstract: Forests present compounding challenges for mobile mapping systems. Dense canopy degrades GNSS, uneven terrain demands deployment across diverse platforms, and no single sensing platform can capture the full vertical structure of a forest — from the canopy above to the understory below. Yet precise, georeferenced maps of individual trees are exactly what ecologists and [...]

Abstract: Reinforcement learning agents in complex tasks often require extensive exploration of large state spaces before useful structure emerges. Instead of pure exploration for learning tasks, it is possible to leverage high-level semantic knowledge such as recipes, instructions, or labels, and adapt to new tasks by grounding that prior knowledge in the environment. This talk [...]

Abstract: Anticipating the near-future actions of multiple people is central to embodied systems that plan and coordinate in shared environments, yet most research targets a single agent and ignores the inter-agent dependencies that shape group behavior. This thesis presents InteractFormer, a unified model that treats inter-agent interaction as a first-class signal: operating directly on fine-grained [...]

Abstract: As AI systems become more capable, further progress may depend not only on scaling models and training data, but also on enabling systems to evaluate and improve their own behavior and development. This raises a dual challenge: how can we make self-improvement more effective while ensuring increasingly autonomous systems remain trustworthy? This thesis investigates [...]

Abstract: Language modeling progress is largely evidenced by steadily rising scores on benchmarks of increasing apparent difficulty. From this, the field infers increasingly general capabilities, many of which presuppose robust world modeling. Interpreting a score, however, requires understanding both the task's computational requirements and how the test-taker generalizes from them. Unlike humans, who demonstrably generalize [...]

  Abstract: The United States Green Industry faces a persistent labor shortage that motivates the adoption of agricultural automation.  However, existing systems are not designed for the unstructured, densely planted environment of a tree nursery. This thesis presents a robotic system intended to alleviate this shortage while remaining usable by non-technical farmers, built around a [...]

Abstract: This talk examines the physical interplay between light and heat: what is absent in the visible image—light not reflected by the scene—is absorbed as heat and manifests in the thermal spectrum. This process intrinsically couples illumination, material properties, and heat transport. By exploiting this coupling, a single visible–thermal image pair provides complementary information that enables [...]

Abstract: A robot that manipulates one object well may still fail on the next. Generalizing across diverse objects and tasks is hard because such objects vary widely in geometry, articulation, and interaction dynamics. Hierarchical policies offer a powerful approach: a high-level policy predicts sub-goal end-effector poses, and a low-level policy generates the actions to reach [...]

Abstract: Autonomous agents and robots are taking on increasingly collaborative roles alongside people, from self-driving vehicles that navigate roads alongside human drivers, to language-model agents that understand user intentions and execute tasks independently. In each of these settings, success is determined not only by an agent's individual task competence, but by its ability to work [...]

Abstract: Manual diagrams are a rich and common knowledge source humans use to learn new skills, but their use for robot learning is still underexplored. A challenge with instruction diagrams is that they communicate task progression in a sparse, qualitative visual format, relying on the learner's prior physical understanding to fill in unmentioned execution details. [...]

Abstract: World models have shown promise in robotics by imagining future outcomes of robot actions directly from high-dimensional sensor observations. However, learning visuomotor policies with world models still faces significant reliability challenges, as imagined futures from world models often diverge from the outcomes that may actually occur. This unreliability arises from uncertainty in learned world [...]

Abstract: World models have shown promise in robotics by imagining future outcomes of robot actions directly from high-dimensional sensor observations. However, learning visuomotor policies with world models still faces significant reliability challenges, as imagined futures from world models often diverge from the outcomes that may actually occur. This unreliability arises from uncertainty in learned world [...]

Abstract: High-speed autonomy in the real world requires accurate control, which often relies on dynamics models that capture the complex interaction between a robot and its environment. In off-road regimes, this terrain interaction dominates the robot’s dynamics, driven by formidable characteristics such as diverse surface properties, complex geometries, environment diversity, and high-speed instability. This thesis investigates how [...]

Abstract: United States apple growers lose more than $100 million a year to fire blight, a bacterial disease caused by Erwinia amylovora that is detrimental to pome fruit trees such as apple and pear.This bacterium infects blossoms, shoots, and branches during the bloom season causing tissue to die. Therefore, the detection and removal of infected tissues over [...]

Abstract: Learning robot manipulation policies typically requires large amounts of high-quality demonstration data, which is expensive to collect on real robots. While large robot and human datasets exist, embodiment gaps make transferring knowledge between platforms challenging. This thesis investigates cross-embodiment transfer—how robots can learn from experience collected on different robots and humans to reduce the [...]

Abstract: Modern language models’ in-context learning and instruction-following abilities make it possible to adapt agent behavior without updating model weights. This thesis studies online self-improvement, where an agent encounters each task only once and must accumulate reusable experience across a sequence of tasks, reflecting real-world deployments in which actions have persistent consequences and retries may [...]

Abstract:  Real-time robot motion in dynamic and uncertain environments requires algorithms that can make effective decisions under strict computational constraints. High-fidelity dynamics, large search spaces, uncertainty over future outcomes, and the pursuit of optimal motion each impose substantial computational costs. These challenges become especially pronounced in partially observable and rapidly changing environments, where a robot [...]

Abstract:   Retail and warehouse perception systems work against a catalog that never stops changing: a segmenter deployed today will eventually be asked to find products that did not exist when it was trained. A closed-set segmenter can only grow its vocabulary by widening its classification head and fine-tuning it, which risks the classes it [...]

Abstract: In wildfire scenarios, deploying autonomous drones requires safely anticipating the dynamic behavior of dense smoke to coordinate effectively. Unlike traditional rigid obstacles, smoke represents a fast-moving, complex fluid hazard that impairs visual navigation and onboard sensors. In this thesis, we propose a novel risk-aware, multi-agent path planning framework that treats dynamic smoke as a [...]

Date: Thursday, July 30, 2026 Time: 3:30 PM - 4:30 PM Location / ZOOM Link: (GHC 6115 / https://cmu.zoom.us/j/96096959582 ) Abstract: Many problems in robotics require reasoning over a mix of continuous dynamics and discrete events, such as making and breaking contact in manipulation and locomotion. These problems are locally well modeled by quadratic programs [...]

Abstract: Many problems in robotics require reasoning over a mix of continuous dynamics and discrete events, such as making and breaking contact in manipulation and locomotion. These problems are locally well modeled by quadratic programs with complementarity constraints (QPCCs). While very expressive, QPCCs are non-convex problems, and few solvers exist for computing fast, local solutions [...]