BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Robotics Institute Carnegie Mellon University - ECPv6.15.12.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Robotics Institute Carnegie Mellon University
X-ORIGINAL-URL:https://www.ri.cmu.edu
X-WR-CALDESC:Events for Robotics Institute Carnegie Mellon University
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250804T100000
DTEND;TZID=America/New_York:20250804T120000
DTSTAMP:20260922T104238
CREATED:20250725T135938Z
LAST-MODIFIED:20250725T135938Z
UID:147679-1754301600-1754308800@www.ri.cmu.edu
SUMMARY:3D Video Models through Point Tracking\, Reconstructing\, and Forecasting
DESCRIPTION:Abstract: \nThis thesis advances 3D video understanding by bridging reconstruction and dynamics forecasting from monocular video\, with applications in robotics\, autonomy\, and immersive environments. We introduce a novel pipeline that translates 2D video into 4D scenes by combining object-centric tracking\, learned 2D view synthesis priors\, and Gaussian splatting\, enabling accurate geometry and motion recovery even under occlusions. We introduce a multi-object joint optimization of 3D dynamic reconstructions that captures cross-object interactions and object-centric generative priors\, yielding state-of-the-art 4D reconstructions with consistent instance masks and temporally coherent appearance. Complementing this\, we propose a scalable transformer-based diffusion model for forecasting 3D particle dynamics\, enabling fast\, action-conditioned planning via score-based guidance. Trained purely in simulation\, the model generalizes to real-world objects reconstructed from monocular video\, outperforming traditional planners in both accuracy and efficiency. Together\, these contributions unify 3D reconstruction and generative dynamics modeling into an interactive framework for reasoning and control in complex physical scenes. \n\n\nThesis Committee Members: \nKaterina Fragkiadaki\, Chair\nKris Kitani\nShubham Tulsiani\nKosta Derpanis\, York University\n\nDraft of the Thesis Defense Document
URL:https://www.ri.cmu.edu/event/3d-video-models-through-point-tracking-reconstructing-and-forecasting-2/
LOCATION:NSH 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250805T110000
DTEND;TZID=America/New_York:20250805T123000
DTSTAMP:20260922T104238
CREATED:20250729T140930Z
LAST-MODIFIED:20250729T140930Z
UID:148063-1754391600-1754397000@www.ri.cmu.edu
SUMMARY:Unified 3D Perception and Generative Control for Generalist Robots
DESCRIPTION:Abstract:\nTo build robot generalists\, we need models that can operate across diverse tasks\, scenes\, and embodiments. While recent efforts scale data and model capacity and incorporate expressive generative objectives\, most still rely on 2D inputs to predict inherently 3D actions—introducing a mismatch between perception and control. In my thesis\, I explore how unifying 3D spatial representations with generative models enables policies that are expressive\, multimodal\, and grounded in the physical world.\n\nMy early work established foundations in 3D vision and policy learning\, leading to the development of 3D Diffuser Actor (3DDA)—the first 3D diffusion policy for general robotic manipulation. 3DDA demonstrated that combining 2D foundational representations with 3D-aware attention in a generative framework enables multimodal behavior and strong performance across tasks.\n\nIn this talk\, I will introduce 3D Flow Actor (3DFA)\, a versatile generalization of 3DDA that supports single-arm\, bimanual and dexterous manipulation\, while offering up to 20× faster training and inference by integrating recent advances in generative modeling. 3DFA achieves state-of-the-art results across simulation benchmarks and demonstrates robust real-world performance on the bimanual ALOHA platform—outperforming contemporary policies with 1000x more parameters.\n\nNext\, I will present key design choices for effectively scaling 3D policies in size. Our 3DFA-VLA model integrates a vision-language backbone with carefully designed feature upsampling layers to construct a billion-parameter policy that preserves explicit 3D token grounding. This results in strong performance and improved data efficiency compared to 2D vision-language-action models.\n\nI will conclude by outlining future directions enabled by our work\, including large-scale real-world training on automatically calibrated RGB-D data.\n——————\n\nThesis committee:\nKaterina Fragkiadaki (chair)\nYonatan Bisk\nShubham Tulsiani\nAbhishek Gupta (Univ. of Washington)\n\nLink to thesis draft:\nhttps://drive.google.com/drive/u/0/folders/1GFiBjbLWHmpIscfh3PBY0Mk_kBqzAGV1\n\nMeeting ID: 607 608 9211\nPasscode: 689168
URL:https://www.ri.cmu.edu/event/unified-3d-perception-and-generative-control-for-generalist-robots/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250820T110000
DTEND;TZID=America/New_York:20250820T123000
DTSTAMP:20260922T104238
CREATED:20250812T132808Z
LAST-MODIFIED:20250812T132808Z
UID:148251-1755687600-1755693000@www.ri.cmu.edu
SUMMARY:Reachable Sets for Control and Planning: from Reactive Safety to Contact-Rich Manipulation
DESCRIPTION:Abstract: \nRobots are increasingly deployed in settings where safety\, performance\, or both are mission-critical—from agile aerial vehicles avoiding collisions at high speed to manipulators executing intricate\, contact-rich tasks. In my thesis\, I present a unifying approach to these seemingly disparate challenges through the lens of reachable sets\, a versatile but underutilized computational primitive in robotics. \nIn the first part\, I focus on reactive safe control for agile systems\, such as drones\, extending reachability-based methods to handle uncertain\, high-dimensional\, real-world systems. I will describe two approaches: a robust-adaptive controller synthesized via sum-of-squares programming\, which guarantees safety while minimizing interference with performance\, and a neural-network-based control barrier function learned adversarially for scalability without sacrificing safety. \nIn the second part\, I turn to motion planning for contact-rich manipulation\, where the challenge lies in the hybrid nature of the problem—combinatorial contact modes coupled with continuous\, nonconvex motion. Here\, I introduce a hierarchical planner that uses reachable sets as motion primitives\, constructed in object space to capture kinematic and dynamic feasibility by design. This enables efficient global planning for complex bimanual systems\, producing significantly higher-quality trajectories than state-of-the-art methods. Together\, these contributions demonstrate the breadth of problems reachable sets can address\, from low-level reactive safety to long-horizon planning. \nCommittee Members: \nChangliu Liu\, Chair\nJohn Dolan\, Chair\nMaxim Likhachev\nPang Tao\, RAI
URL:https://www.ri.cmu.edu/event/reachable-sets-for-control-and-planning-from-reactive-safety-to-contact-rich-manipulation/
LOCATION:GHC 4215
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250826T123000
DTEND;TZID=America/New_York:20250826T133000
DTSTAMP:20260922T104238
CREATED:20250820T135228Z
LAST-MODIFIED:20250820T135228Z
UID:148306-1756211400-1756215000@www.ri.cmu.edu
SUMMARY:Toward Personalized Assistive Systems: Leveraging Large Language Models for Prediction and Intervention
DESCRIPTION:Abstract:\n\nMany older adults\, particularly those with Mild Cognitive Impairments (MCI) struggle with complex\, sequential tasks such as meal preparation. In this thesis\, we present a framework for personalized sequence prediction and assistance detection during meal preparation to support older adults\, particularly those with Mild Cognitive Impairments (MCI). By leveraging the reasoning capabilities of large language models (LLMs)\, our system anticipates user actions and identifies moments when assistance may be needed. We introduce two methods for preference-based sequence prediction\, called Independent Context and Shared context\, using either a participant’s own prior actions or sequences from others as context. Evaluated on two meal preparation datasets\, these approaches outperform baseline models by up to 33.8%\, demonstrating both user-specific adaptation and generalization across cooking domains. To inform assistance strategies\, we conducted a meal preparation data-collection study with older adults at two independent living facilities. Insights from this study revealed common errors\, such as forgotten items or visits to irrelevant locations. We used these findings to develop a second approach that detects such mistakes and prompts assistance using LLM reasoning. This approach was validated on both synthetic and real-world data\, showing strong performance in identifying when users may need help. These contributions form the basis of a personalized assistive system that supports users while preserving their independence in daily meal preparation tasks.\n\nCommittee:\nReid Simmons\, co-chair\nZackory Erickson\, co-chair\nIllah Nourbakhsh\nPatrick Callaghan
URL:https://www.ri.cmu.edu/event/toward-personalized-assistive-systems-leveraging-large-language-models-for-prediction-and-intervention/
LOCATION:GHC 6501
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250826T140000
DTEND;TZID=America/New_York:20250826T150000
DTSTAMP:20260922T104238
CREATED:20250820T135404Z
LAST-MODIFIED:20250820T135404Z
UID:148308-1756216800-1756220400@www.ri.cmu.edu
SUMMARY:Towards Generalizable Robotic Policies via Data-Driven Learning at Scale
DESCRIPTION:Abstract: \nTo enable robots to operate seamlessly in complex\, real-world environments\, they must master fine-grained manipulation skills and exhibit robust\, adaptive behavior across diverse environments. This thesis explores a data-driven approach to learning generalizable and reactive manipulation policies by leveraging efficient data generation pipelines and expressive neural models. We first introduce BiDex\, a low-cost teleoperation system for collecting high-quality demonstrations on dexterous bimanual tasks\, addressing the challenge of acquiring real-world data. To overcome the limitations of scale and diversity in physical data collection\, we present Neural MP\, a simulation-based framework for autonomous data generation. Building on this foundation\, we propose DRP\, a learning paradigm that augments offline imitation learning with online fine-tuning and reactive components\, enabling policies to perform reliably in dynamic and partially observable settings. Together\, these contributions provide a practical recipe for developing robust robotic manipulation systems at scale. \nCommittee: \nDeepak Pathak (chair) \nRuslan Salakhutdinov \nKenneth Shaw
URL:https://www.ri.cmu.edu/event/towards-generalizable-robotic-policies-via-data-driven-learning-at-scale/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250828T130000
DTEND;TZID=America/New_York:20250828T143000
DTSTAMP:20260922T104238
CREATED:20250818T182435Z
LAST-MODIFIED:20250819T122821Z
UID:148285-1756386000-1756391400@www.ri.cmu.edu
SUMMARY:Dataset-Driven and Generative Approaches to Domain Generalization in Human-Centric Vision
DESCRIPTION:Abstract:\n\n\n\n\n\n\n\n\n\nHuman-centered computer vision technology relies heavily on large\, diverse datasets\, yet even the largest collections cannot fully capture the variability of human appearance\, motion\, and viewpoint. At the same time\, collecting data from human subjects is time-consuming\, labor-intensive\, and raises privacy concerns. To overcome these challenges while maintaining efficiency\, researchers increasingly turn to two complementary directions: dataset-driven approaches (synthetic data generation\, domain adaptation\, hybrid training\, targeted fine-tuning) and generative refinement methods that improve model outputs when lightweight architectures cannot fully generalize. \nThis thesis explores both directions by presenting new data for human-based vision models and addressing their limitations through synthetic augmentation\, curriculum-based training\, and single-step generative refinement. Together\, these methods compensate for gaps in training diversity\, improve generalization to unseen domains\, and reduce the computational cost of training and inference. The approaches are applied to two applications: activity recognition (classifying human actions from sequences of frames) and 3D avatar generation (creating 3D avatars from a few subject images). \nIn the first part\, we introduce REMAG\, a dataset suite comprising both real and synthetic data across eleven action classes\, captured from ground and drone cameras (Chapter 2). The synthetic portion is generated using four distinct methods\, combining either traditional computer graphics (CG) or neural rendering with motion sources from either marker-based motion capture or 2D video-tracked motions. Through extensive experiments\, we demonstrate that a two-step fine-tuning strategy—pre-training on high-quality synthetic data followed by fine-tuning on a small amount of real data—can match or even surpass the performance of models trained on a substantially larger real dataset\, while also reducing training time. However\, in real-world scenarios\, collecting synchronized ground- and aerial-view data is often impractical due to the significant effort and resources required. To address this limitation\, in Chapter 3 we investigate a domain adaptation setting in which no real aerial-view training data is available\, examining how models can efficiently generalize to the real aerial-view domain when trained only with real ground-view and synthetic aerial-view data. \nIn the second part\, we focus on training a 3D avatar generation model called Universal Avatars (UA) using a large-scale multi-view dataset of human heads called Ava-256 (Chapter 4). While this dataset allows the model to generate complete 3D avatars from input images and drive them with expression signals\, lightweight architectures such as GP-Avatar\, which enable real-time avatar synthesis from a single appearance image\, struggle to preserve identity and expression consistency across diverse viewpoints. To address this tradeoff between efficiency and fidelity\, Chapter 5 introduces TurboPortrait3D\, a single-step diffusion method that refines coarse novel views generated by GP-Avatar. Unlike existing 3D-aware generative methods that rely on multi-step optimization\, TurboPortrait3D produces sharper\, identity-faithful\, and 3D-consistent novel views in real time\, demonstrating that generative refinement can effectively complement lightweight dataset-driven models. \n\n\n\n\n\n\n\n\n\n\n\nCommittee members: Jessica Hodgins (Chair)\, Fernando de la Torre\, Jun-Yan Zhu\, Julieta Martinez (Meta)
URL:https://www.ri.cmu.edu/event/dataset-driven-approaches-to-domain-generalization-in-human-centric-vision/
LOCATION:GHC 6121
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250829T090000
DTEND;TZID=America/New_York:20250829T103000
DTSTAMP:20260922T104238
CREATED:20250820T134828Z
LAST-MODIFIED:20250820T134828Z
UID:148304-1756458000-1756463400@www.ri.cmu.edu
SUMMARY:Influence-Aware Safety for Human-Robot Interaction
DESCRIPTION:Abstract:\nIn recent years\, we have seen how influential (and potentially harmful) algorithms can be in our lives through recommender systems and language models; sometimes creating polarization and conspiracies that lead to unsafe behavior. Now that robots are also growing more common in the real world\, we must be very careful to ensure that AI-driven systems are aware of the influence they have on people\, especially when it comes to real-world behavior. In this thesis\, we focus on the problem of influence-aware safe control for human-robot interaction in hopes of enabling robots to intentionally and positively influence people to make their interactions with robots more safer and more efficient. We first study this problem from the safe control perspective by introducing a novel method for dealing with the multimodality of the robot’s uncertainty over a human’s intention inside a robust safe controller. Next\, we explore different methods for generating influence-aware robot behavior from different levels of abstraction: action-directed\, goal-directed\, and strategy-directed. We ultimately find useful tools for designing robot behaviors that can proactively influence human collaborators towards positive outcomes. We then introduce a method to solve the influence-aware safe control problem that uses reach-avoid dynamic games to incorporate a prediction model of a human into the synthesis of a safe controller. This ultimately allows for more efficient interactions without sacrificing safety. Finally\, we extend this problem formulation to more general interactions between a human and a language-based AI assistant. We demonstrate that safety-critical reinforcement learning can enable us to automatically learn guardrails for the AI assistant in order to steer the human towards safer outcomes. Ultimately\, we hope that the work done in this thesis will help researchers in robotics (and beyond) to understand the importance of modeling the influence that AI-driven agents have on people and how to use this understanding to keep people safe.\n\n\nThesis Committee Members: \nChangliu Liu\, Co-chair \nAndrea Bajcsy\, Co-chair \nAditi Raghunathan \nGuy Rosman\, Toyota Research Institute
URL:https://www.ri.cmu.edu/event/influence-aware-safety-for-human-robot-interaction/
LOCATION:Newell-Simon Hall 3305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250829T110000
DTEND;TZID=America/New_York:20250829T130000
DTSTAMP:20260922T104238
CREATED:20250819T195025Z
LAST-MODIFIED:20250819T195955Z
UID:148297-1756465200-1756472400@www.ri.cmu.edu
SUMMARY:Scalable Sim-to-Real Learning for General-Purpose Humanoid Skills
DESCRIPTION:Abstract:\n \nHumanoids represent the most versatile robotic platform\, capable of walking\, manipulating\, and collaborating with people in human-centered environments. Yet\, despite recent advances\, building humanoids that can operate reliably in the real world remains a fundamental challenge. Progress has been hindered by difficulties in whole-body control\, robust perceptive reasoning\, and bridging the sim-to-real gap.\n\nThis thesis proposal explores how scalable simulation and learning can systematically overcome these barriers. I will present a research trajectory that advances humanoid capabilities along three dimensions:\n\n1. Sim-to-Real Motor Control (Chapter 2–5): From real-time teleoperation (H2O) to dexterous loco-manipulation (OmniH2O)\, to a versatile generalist controller (HOVER)\, and agile transfer via dynamics alignment (ASAP)\, these works demonstrate increasingly dexterous and adaptable control.\n2. Sim-to-Real Perceptive Control (Chapter 6): With ABS\, we show that robust real-world locomotion requires tightly coupling exteroceptive and proprioceptive sensing—shifting sim-to-real learning from blind skill execution to perception-driven control.\n3. Future Directions (Chapter 7): I will outline next steps in (1) Perceptive Loco-Manipulation—end-to-end visuomotor policies unifying perception\, locomotion\, and manipulation—and (2) Real-to-Sim Evaluation—using high-fidelity simulators environments to provide consistent evaluation protocols and benchmarks for real-world policy evaluation.\n\nTaken together\, these works explore how far scalable sim-to-real learning can advance humanoid capabilities across control and perception. While sim-to-real is not the only path toward reliable humanoids\, this thesis aims to rigorously test its limits—probing how scaling simulation and learning can push humanoids closer to functioning as capable partners in real-world environments.\n\nThesis Committee Members:\nGuanya Shi (co-chair)\nChangliu Liu (co-chair)\nKris Kitani\n\n\nMarco Hutter (ETH Zurich)\nPieter Abbeel (UC Berkeley)
URL:https://www.ri.cmu.edu/event/scalable-sim-to-real-learning-for-general-purpose-humanoid-skills/
LOCATION:GHC 4405
CATEGORIES:PhD Thesis Proposal,Student Talks
END:VEVENT
END:VCALENDAR