Efficient Warping for Visual Perception
Abstract: Salient foreground regions (e.g., vehicles, faces) occupy only ~10% of pixels, while less informative backgrounds (e.g., sky, trees) dominate ~90%. This imbalance fundamentally limits visual perception: (1) 2D discriminative tasks (e.g., domain-adaptive detection and segmentation) rely heavily on large background regions with high cross-domain variation, making domain adaptation difficult. (2) 2D generative tasks (e.g., [...]
Integrating Reinforcement Learning and Model Predictive Control for Autonomous Off-road Driving
Abstract: Safe and effective autonomous traversal of off-road terrain is challenging due to both terrain properties, such as low traction in sand or deformability of mud, and terrain geometries, including steep slopes, ditches, and uneven surfaces that can induce unsafe vehicle behaviors like excessive pitch and roll. Model Predictive Path Integral (MPPI) control provides a powerful [...]
Leveraging Deformations in Soft Objects and Robots for Contact-Rich Dexterity
Abstract: Robust dexterity depends on the ability to cause and reason about deformation. Robots may routinely need to interact with cables, dough, textiles, and hair in our homes, where deformation is the object state. Soft robots, in turn, are built from compliant materials in which deformation is not a by-product of interaction but its primary [...]
Longitudinal Human–Robot Interaction: Adaptive Personalization Across Repeated Encounters
Abstract: As robots increasingly move into homes, healthcare settings, and public environments, many are expected to support people not through single encounters, but through repeated interaction over time. In these settings, successful human--robot interaction depends not only on immediate task performance, but also on how users adapt to robotic systems, how expectations change with repeated [...]
Precise and Generalizable Robot Manipulation
Abstract: Robots in factories are still largely limited to structured environments with known object models. How can we bring robots into the more diverse, unstructured settings of our daily lives, where objects may vary widely in shape and appearance, while maintaining reliable performance? A popular direction today is to train generalist robot policies on large-scale internet [...]
WinkTPG: An Execution Framework for Multi-Agent Path Finding Using Temporal Reasoning
Abstract: Planning collision-free paths for a large group of agents is a challenging problem in many real-world applications. While recent advances in Multi-Agent Path Finding (MAPF) have shown promising progress, standard MAPF planners continue to rely on simplified kinodynamic models, preventing agents from directly following the generated MAPF plan. To bridge this gap, we propose [...]
Leveraging Local Models for Planning and Control with Contact
Abstract: Many planning and control approaches in robotics have converged on optimization-based formulations, with recent advances achieved by leveraging significant data and compute to attempt to tackle these nonlinear and non-convex problems. In this thesis, we instead focus on local models and demonstrate their benefits and surprising effectiveness. In the case of smooth optimization, the local [...]
Forecasting at Scale with Efficient Deep Learning Architectures
Abstract: Time Series Foundation Models (TSFMs) have scaled rapidly, with publicly reported pretraining corpora growing from 1.23 billion to 1 trillion data points between 2024 and 2026, an approximately 800× increase in two years. Recent work has further supplemented real-world data with synthetic data to expose models to broader time series patterns. Yet, this data-centric [...]
Think Globally, Solve Locally: Non-sequential Planning for Robotic Manipulation
Abstract: Robotic manipulation requires reasoning that bridges local competence over fine-grained dynamics with the construction of valid long-horizon plans. This mirrors human reasoning, where fast, automatic processes propose locally plausible actions from experience, while slower deliberation integrates them into a whole. Furthermore, evidence from cognitive science suggests that humans do not reason sequentially from start [...]
Leveraging Tactile Sensing to Resolve Uncertainty in Contact-Rich Manipulation
Abstract: Manipulation in agricultural and unstructured environments often involves contact-rich interactions with occluded objects. Most deployed systems treat contact as a hazard and rely on vision alone, which limits deployment from real-world field settings. This thesis adopts a different perspective: contact is a strategy for obtaining information where vision cannot provide. Touch can reveal additional [...]
Diffusion Temperature Sampling and Projective Ray Positional Encoding for Multi-view Attention
Abstract: Inductive biases have proven effective and often essential in the design of performant deep learning systems. This thesis presents two contributions that target distinct facets of how inductive biases can improve modern deep models. In Part 1, we focus on controlling the diversity--likelihood trade-off at inference-time of generative models. In Part 2, we study the [...]
Learning Bayesian Experimental Design Policies Efficiently
Abstract: Bayesian Experimental Design (BED) provides a principled framework for informative data collection, and is applied across domains as varied as adaptive clinical trials, ecological monitoring, hyperparameter optimization, and robotic search. Despite this broad applicability, BED methods remain difficult to deploy in practice: high-quality decision-making is computationally expensive, calibrated uncertainty estimation in sequential decision problems [...]
Quanta Perception as Probabilistic Events
Abstract: Autonomous systems ultimately rely on extracting information from light, yet remain brittle in extreme environments, from nighttime navigation to high-speed robotics. This limitation stems from a classical imaging abstraction: conventional sensors integrate photon flux over fixed exposure windows, imposing trade-offs between sensitivity, dynamic range, and temporal resolution that degrade perception when photons are scarce [...]
Robot Learning and Wearable Interfaces in Pursuit of Robotic Caregivers
Abstract: Designing safe and reliable robotic assistance for caregiving is a grand challenge in robotics. A sixth of the United States population is over the age of 65 and more than 1 in 4 (over 70 million) adults in the United States reported having a disability in 2022. Robotic caregivers could positively benefit society; yet, physical [...]
Deep Abstraction Learning for Neuro-Symbolic World Modeling
Abstract: Modern foundation models have achieved remarkable progress by learning broad physical and semantic common sense from large-scale data. However, robots operating in open-ended environments require more than general knowledge alone: they must continually specialize in new tasks, environments, and experiences encountered during deployment. Given only limited deployment-time data, how can robots learn to solve substantially more—and [...]
Designing A Learning-Enabled Non-Anthropomorphic Robotic Hand Framework for Dexterous Manipulation
Abstract: As robots transition into unstructured human environments, achieving dexterous manipulation across diverse tasks becomes increasingly crucial yet remains challenging. This objective underscores the value of high-degree-of-freedom (DoF) robotic hands capable of versatile grasping and in-hand manipulation. Complementing the hardware, success in these settings also depends on intelligent and autonomous hand control strategies that provide [...]
Contact-Rich Manipulation with Vibro-Tactile Sensing
Abstract: Contact-rich manipulation requires reasoning about complex physical interactions. While vision and force sensing provide useful information, many interaction dynamics remain difficult to observe directly, especially under occlusion, rapid contact transitions, or distributed contact. This thesis explores how vibro-tactile sensing, also known as structure-born acoustic sensing, can serve as a practical and information-rich modality for [...]
RI PhD Speaking Qualifier / MSR Talk – Yuemin Mao
TBD
RI PhD Thesis Defense – Junyu Nan
Date: Tuesday May 19, 2026 Time: 1:00 - 3:00PM (EST) Location: NSH (Newell Simon Hall) 3305 Zoom Link Type: PhD Thesis Defense Title: Learning Geometric, Physical, and Semantic Priors for Embodied Planning and Control Abstract: Embodied intelligence requires perceiving, predicting, and acting in environments with an understanding of the geometric, semantic, and physical structure of the world. Recently, [...]
RI PhD Thesis Proposal – Renos Zabounidis
Date: May 20th, 2026 Time: 3:00 - 4:30 pm Location: NSH 4305 Zoom Link Type: RI PhD Thesis Proposal Who: Renos Zabounidis Title: Enforcing Neuro-Symbolic Structure in Deep Reinforcement Learning Abstract: Monolithic deep reinforcement learning trains a single network to learn vision, physics, planning, and control from reward alone. The result is poor sample efficiency, [...]
Scalable Imitation Learning for Robust Manipulation and Physical Human-Robot Interaction
Abstract: Robots in everyday human environments are expected to perform robust manipulation across cluttered, constrained, and physically interactive settings. This thesis studies how scalable simulation-based data generation can train vision-based imitation learning policies for robust zero-shot transfer to the real world. In the first part of this thesis, we study robotic manipulation in cluttered shelf [...]
Generalizable Neural Dynamics Modeling for Complex Deformable Object Manipulation
Abstract: Hair care is an essential daily activity, yet it remains inaccessible to individuals with limited mobility and challenging for autonomous robot systems due to the fine-grained physical structure and complex dynamics of hair. This thesis presents DYMO-Hair, a model-based robot hair care system. We introduce a novel dynamics learning paradigm that is suited for [...]
RI PhD Thesis Proposal – Anurag Ghosh
Date: May 21st, 2026 Time: 3:30 - 5:00 pm Room: NSH Room 4305 Zoom: https://cmu.zoom.us/j/98318417145 Type: RI PhD Thesis Proposal Who: Anurag Ghosh Title: Scaling Long-Tailed Driving Perception and Planning with In-the-Wild Videos Abstract: Closed-loop driving, where methods produce actions a simulator reacts to, remains largely tied to driving logs from instrumented fleets. Thus, reliably driving in rare-but-critical scenarios is [...]
RI PhD Thesis Defense – Kenneth Shaw
Date: Friday May 22 Time: 3:30PM - 4:30PM (EST) Location: NSH (Newell Simon Hall) 3305 ZOOM Link Title: Building Robot Hands and Teaching Dexterity Abstract: Our human hands are masterpieces of power and precision, capable of typing, hammering, or delicately using chopsticks. Yet most robots today still rely on simple two-finger grippers in controlled settings because dexterous hands are [...]
Toward Real-World Autonomous Off-Road Driving with Reinforcement Learning
Abstract: Off-road autonomous driving presents significant challenges such as navigating unmapped variable environments, traversing difficult terrain geometries such as steep slopes and ditches, and managing complex terrain dynamics. Addressing these challenges requires effective low-level adaptable control and long-horizon planning. Most existing methods utilize Model Predictive Control (MPC) methods such as Model Predictive Path Integral (MPPI), [...]
Explore and Exploit: Learning Policies for Efficient and Coordinated Active Search
Abstract: Robotic search is becoming a central capability in domains where the world is large, uncertain, and costly to inspect directly: search and rescue, environmental monitoring, surveillance, and infrastructure inspection. In these settings, the hard problem is not perception alone but the online sensing decision: where to look next as evidence arrives, while every motion [...]