BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Robotics Institute Carnegie Mellon University - ECPv6.15.12.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Robotics Institute Carnegie Mellon University
X-ORIGINAL-URL:https://www.ri.cmu.edu
X-WR-CALDESC:Events for Robotics Institute Carnegie Mellon University
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20270314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20271107T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260601T143000
DTEND;TZID=America/New_York:20260601T153000
DTSTAMP:20260707T131935
CREATED:20260526T132047Z
LAST-MODIFIED:20260526T132047Z
UID:151385-1780324200-1780327800@www.ri.cmu.edu
SUMMARY:Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation
DESCRIPTION:Abstract:\nImitation learning has emerged as a leading paradigm for teaching manipulation skills to robots\, but its success depends on the costly endeavour of collecting robot demonstrations through teleoperation. Generalizing to novel objects\, environments\, and task variations typically requires massive datasets that are expensive to scale. This thesis investigates an alternative lever: hierarchy—explicitly factorizing manipulation policies into high-level reasoning about where to go and low-level control about how to get there—and asks whether such a factorization can both improve data efficiency from robot demonstrations alone and enable the use of\ncheaper sources of data such as human video.\n\nWe present GHOST\, a framework for learning visuomotor manipulation policies that generalize beyond the training distribution. GHOST factorizes control into (i) a high-level policy that predicts the next sub-goal as a distribution over 3D end-effector poses from multi-view RGB-D observations\, and (ii) a low-level goal-conditioned controller that executes embodiment-specific actions. To condition image-based policies on 3D goals\, we introduce a simple spatial interface that projects predicted goals into the image plane and represents them as end-effector heatmaps. Across a suite of manipulation tasks\, this hierarchical factorization consistently improves performance and robustness compared to a flat Diffusion Policy. We further show that this hierarchical interface makes it easy to incorporate human demonstrations without relying on (noisy) action retargeting. Because sub-goals are largely embodiment-agnostic\, we train\nthe high-level policy on human video to specify how learned skills should be applied and composed\, while keeping the low-level policy trained purely on robot data. This hierarchy enables adaptation to novel objects and task variations using only a small number of human demonstrations.\n\nWe evaluate GHOST on a suite of manipulation tasks spanning pick-and-place\, cloth folding\, and tool-use\, demonstrating both improved in-distribution performance and meaningful cross-embodiment generalization from human video. We conduct ablations to further isolate the contributions of the hierarchical factorization\, and identify the visual domain gap between human and robot observations as a significant bottleneck for long-horizon generalization. \nCommittee:\nDavid Held (advisor)\nShubham Tulsiani\nAviral Kumar\nYilin Wu
URL:https://www.ri.cmu.edu/event/hierarchical-sub-goal-policies-for-generalizing-robot-manipulation/
LOCATION:Gates Hillman Center 4405
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260604T110000
DTEND;TZID=America/New_York:20260604T123000
DTSTAMP:20260707T131935
CREATED:20260529T154504Z
LAST-MODIFIED:20260529T154504Z
UID:151398-1780570800-1780576200@www.ri.cmu.edu
SUMMARY:User Intent-Driven and Context-Aware Personalization for Assistive Exoskeletons
DESCRIPTION:Abstract:\nPersonalizing exoskeleton control to individual preferences is crucial for real world deployment. Data-driven approaches have enabled user-generalizable controllers\, yet conventional personalization methods optimize biomechanical cost functions over user preferences. Prior work shows that users can perceive and report their preferred parameters\, yet no lightweight method maps user intent to quantitative control parameter changes in real time. \nTo address this gap\, we present a vision-language model (VLM) guided human-exoskeleton interface that translates natural user feedback and egocentric visual context into parameter updates for a hip exoskeleton controller. Our framework uses an off-the-shelf VLM that performs high-level intent parsing\, while low-level action selection is offloaded to a contextual bandit. This bandit explores and exploits over two personalization parameters\, the magnitude of assistance and the offset timing\, while learning individual preferences over time. We use human feedback (Likert scale) ratings as rewards for the bandit\, iterating parameter updates until the user reports satisfaction (Likert ≥ 5). \nWe evaluated this framework across a five-task locomotion track\, spanning level ground\, ramps\, and stairs\, across two laps\, against a single-shot interpreter and a one-shot-feedback baseline. Our contextual bandit pipeline improved from default initializations within a single interaction\, raising satisfaction by an average of 3 Likert points in multi-rating loops. No-improvement personalization instances were eliminated by the second lap\, and user satisfaction interactions increased from 67% to 95% across laps. We demonstrate VLMs paired with contextual bandits as an effective framework for online\, context-aware\, user preference learning in real-time exoskeleton personalization. \n\nCommittee:\nDr. Inseung Kang (chair)\nDr. Hartmut Geyer\nMichelle Zhao
URL:https://www.ri.cmu.edu/event/user-intent-driven-and-context-aware-personalization-for-assistive-exoskeletons/
LOCATION:Newell-Simon Hall 3305
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260608T123000
DTEND;TZID=America/New_York:20260608T140000
DTSTAMP:20260707T131935
CREATED:20260602T163009Z
LAST-MODIFIED:20260602T163009Z
UID:151481-1780921800-1780927200@www.ri.cmu.edu
SUMMARY:Design and Evaluation of Low-Cost\, Open-Source Haptic Interfaces for Diverse Learning Applications
DESCRIPTION:Abstract: Touch is a powerful yet underused channel for learning. Prior research shows that haptic interaction can support both sensorimotor skill acquisition and the understanding of abstract concepts by grounding learning in bodily experience. However\, most haptic devices remain expensive\, technically complex\, and difficult to reproduce\, which keeps them largely confined to specialized laboratories. This limits their use in education and rehabilitation and has slowed progress toward scalable\, low-cost\, open-source solutions\, as well as toward a systematic understanding of how affordable haptic devices should be designed to reliably produce learning benefits. As a result\, the broader learning potential of haptics remains underexplored\, especially across diverse domains and beyond measures of immediate task success.\nThis thesis examines the design and evaluation of haptic systems for learning across three distinct domains. The first system\, HaptiClay\, explores how haptics and gesture can support mathematics learning by helping students construct concrete representations of terms in polynomial functions. The thesis traces the iterative design of the device and reports interventions with students that use haptics to encourage gestural movements while molding polynomial functions and relate those gestures to specific terms in the polynomials. We then analyze learning outcomes to understand the effectiveness of the intervention. The second system\, DexKit\, enables students to experience dexterity concepts in dexterous teleoperation through touch\, including robotic manipulation control\, object interaction\, and stiffness variation. It introduces a dexterous manipulation platform that combines a soft robotic hand with a three-finger haptic interface\, including a novel two-degree-of-freedom mechanism for the index and middle fingers and a soft delta mechanism for the thumb. The third system\, VibroGait\, is a wearable haptic device for gait correction that helps users learn improved walking patterns through vibrotactile feedback. The thesis presents the design of a flexible skin-interfacing device\, the gait prediction algorithms and their implementation\, and studies comparing multiple haptic feedback patterns for gait correction. \nAcross these case studies\, the thesis investigates how effective\, low-cost learning tools can be designed\, which design principles generalize across domains\, how haptics influence learning beyond task success\, and how haptic systems for learning can be evaluated rigorously. By bringing together mathematics learning\, robotic teleoperation\, and gait correction\, this work expands the evidence base for accessible haptic learning technologies and contributes practical design knowledge for future low-cost\, open-source haptic systems. \n\nCommittee\nMelisa Orta Martinez (chair)\nJames McCann\nEni Halilaj\nKylie Peppler (University of California\, Irvine)\n\n\nThesis Proposal Draft
URL:https://www.ri.cmu.edu/event/design-and-evaluation-of-low-cost-open-source-haptic-interfaces-for-diverse-learning-applications/
LOCATION:3305 Newell-Simon Hall
CATEGORIES:PhD Thesis Proposal,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260609T140000
DTEND;TZID=America/New_York:20260609T153000
DTSTAMP:20260707T131935
CREATED:20260601T172413Z
LAST-MODIFIED:20260601T172413Z
UID:151404-1781013600-1781019000@www.ri.cmu.edu
SUMMARY:Scaling Sim2Real Learning for Robot Manipulation
DESCRIPTION:Abstract: \nRecent progress in robot learning has led to impressively capable manipulation systems. Much of the progress has come from scaling up human demonstrations; however\, collecting such data through manual teleoperation is slow\, costly\, and hard to scale.\n\nPhysics-based Simulation offers a scalable\, safe\, and efficient alternative for generating large demonstration datasets. However\, some core challenges limit the full potential of this approach: the heavy manual effort required to design simulation tasks and rewards\, the gap between simulation and reality\, and the difficulty of learning policies that generalize well when trained on large\, diverse simulation datasets.\n\nIn this thesis\, we tackle these challenges through scalable\, generalizable\, and adaptive robot learning. First\, I will show how structured policy representations can enable simulation trained policies to achieve broad generalization in the real world and serve as a strong prior for downstream fine-tuning. Second\, I will introduce the Generative Simulation pipeline for automatic generation of large-scale simulation datasets with minimal human efforts. Finally\, I will discuss some novel algorithms for efficient adaptation of simulation-trained policies to the real world. Together\, these efforts move us toward robots that can learn broadly\, adapt quickly\, and assist people in real homes and workplaces.\n\nCommittee:\nZackory Erickson (co-chair)\nDavid Held (co-chair)\nKaterina Fragkiadaki\nChuang Gan (UMass Amherst and MIT-IBM Watson AI Lab)\nDieter Fox (University of Washington and Ai2)\nLink to draft thesis
URL:https://www.ri.cmu.edu/event/scaling-sim2real-learning-for-robot-manipulation/
LOCATION:1305 Newell Simon Hall
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260610T100000
DTEND;TZID=America/New_York:20260610T113000
DTSTAMP:20260707T131935
CREATED:20260601T184629Z
LAST-MODIFIED:20260601T184629Z
UID:151406-1781085600-1781091000@www.ri.cmu.edu
SUMMARY:Annotation-Free Learning for Mobile Robot Navigation in Unstructured Environments
DESCRIPTION:Abstract: Navigation in unstructured environments is a capability critical to many robotics applications such as forestry\, construction\, disaster response and defense. In these domains\, robots have the potential to eliminate much of the dull\, dirty and/or dangerous work that is currently performed by humans. Unfortunately\, these environments pose a unique set of challenges for navigation not present in controlled settings such as warehouses and highway driving.\n\nWe argue that the key challenge in deploying robots in unstructured environments is the difficulty of interpreting high-dimensional streams of multi-modal perception data\, and translating them into robust\, expressive representations for downstream path planning and control. This problem is exacerbated by the fact that unstructured environments are highly uncertain and lack a clean mapping from semantics to navigation decisions (i.e. traversability). \nWhile there have been notable successes in deploying robots in unstructured environments\, such systems are highly tuned to a given robot platform and environment\, and are the result of months to years of dedicated effort from a highly skilled team of engineers. As such\, deployment of these systems to new platforms or environments requires extensive tuning of perception-planning interfaces\, labeling of semantic images\, etc. Unfortunately\, this makes engineering expertise the bottleneck in the widespread adoption of mobile robots. \nI will present my work in designing a learning-based perception and traversability system for high-speed off-road driving from camera and LiDAR data. Importantly\, this approach is annotation-free\, enabling performance matching or exceeding current SoTA in off-road driving from a relatively small amount of tele-operated demonstrations. \n\nCommittee:\nSebastian Scherer\, chair\nAaron Johnson\nDrew Bagnell\nWenshan Wang\nDavid D. Fan (Field AI)\n \n\nThesis Draft
URL:https://www.ri.cmu.edu/event/annotation-free-learning-for-mobile-robot-navigation-in-unstructured-environments/
LOCATION:3305 Newell-Simon Hall
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260611T100000
DTEND;TZID=America/New_York:20260611T113000
DTSTAMP:20260707T131935
CREATED:20260528T204739Z
LAST-MODIFIED:20260529T163943Z
UID:151395-1781172000-1781177400@www.ri.cmu.edu
SUMMARY:Autonomous Crop Manipulation: From Model-Based Reasoning to Learned Interaction
DESCRIPTION:Abstract: Robots that manipulate crops must contend with plants that occlude themselves\, deform under contact\, and make the manipulator contact structures it neither targets nor sees in advance. This thesis argues that autonomous manipulation of crops in unstructured agricultural environments is best advanced not by choosing between model-based and learned approaches\, but by integrating them where each is strongest: model-based components provide explicit geometric priors and safety constraints for contact-rich interaction\, while learned components capture plant dynamics and visuomotor behaviors that resist hand-engineering. \nFour technical contributions develop this argument. The first is a model-based skeletonization method that recovers tree branch structure from RGB-D views using an occlusion-likelihood field and a min-cost path search to reason explicitly about unobserved geometry. The second is a graph neural network trained on mass-spring-damper simulations that predicts how a tree deforms under push. The third is an end-to-end diffusion policy trained on demonstrations collected with a handheld shear-gripper that drives pepper harvesting in an unprotected outdoor field. The fourth is a reactive safety layer that wraps a learned visuomotor policy with a control barrier function quadratic program whose constraints are drawn from an occupancy map updated online from contact estimates\, enabling safe reaching through cluttered canopies. \nTaken together\, the four contributions trace a path from offline geometric reasoning about static plants to safe contact-rich interaction in clutter\, showing that robust agricultural manipulation emerges not from end-to-end learning alone\, but from a synthesis in which model-based safety enables reliable learned interaction. \nCommittee:\nGeorge Kantor (chair)\nOliver Kroemer\nDave Held\nMaren Bennewitz (University of Bonn) \nThesis Draft
URL:https://www.ri.cmu.edu/event/autonomous-crop-manipulation-from-model-based-reasoning-to-learned-interaction/
LOCATION:3305 Newell-Simon Hall
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260611T123000
DTEND;TZID=America/New_York:20260611T133000
DTSTAMP:20260707T131935
CREATED:20260604T185218Z
LAST-MODIFIED:20260604T185218Z
UID:151483-1781181000-1781184600@www.ri.cmu.edu
SUMMARY:Red and Blue Teaming for Robust Manipulation under Geometric Variations
DESCRIPTION:Abstract:\nRobotic manipulation policies are typically evaluated on curated\, in-distribution test sets\, which offer limited insight into how these policies behave under plausible variation. One important source of this variation is geometric in nature\, arising from small changes in object geometry that quietly alter grasp affordances and contact dynamics. Rather than treating robustness as a property to be measured passively\, this thesis develops methods that use object geometry as a controllable axis along which a policy can be actively stress-tested and subsequently improved.\n\nWe first introduce Geometric Red-Teaming (GRT)\, a framework that automatically discovers CrashShapes: structurally valid\, user-constrained mesh deformations that induce catastrophic failures in pre-trained manipulation policies. GRT couples a Jacobian-field deformation model with a gradient-free\, simulator-in-the-loop optimization strategy\, allowing it to search over plausible object geometries while treating the policy being tested as a black box. Across insertion\, articulation\, and grasping tasks\, GRT consistently finds deformations that collapse policy performance\, exposing brittle failure modes that static benchmarks miss. We further show that fine-tuning on a small set of CrashShapes together with the nominal object\, a process we call blue-teaming\, improves task success by up to 60 percentage points on those shapes while preserving performance on the original object. We validate both red-teaming and blue-teaming on a real robot arm\, where simulated CrashShapes reduce task success from 90% to as low as 22.5%\, and blue-teaming recovers performance to as high as 90% on the corresponding real-world geometry.\n\nWhile GRT provides a direct evaluation tool for geometric robustness\, its simulator-in-the-loop search takes hours per object and therefore cannot supply CrashShapes for large-scale blue-teaming. To address this limitation\, we introduce CrashGen\, a point-cloud-conditioned diffusion model that amortizes this search by generating failure-inducing deformation controls in seconds rather than hours. A deterministic Jacobian-field operator then maps these controls to physically plausible deformations of the original mesh. Trained once on controls discovered by running GRT offline\, CrashGen generates failure-inducing variants for over 80% of held-out objects under simulation validation. Such variants can in turn serve as targeted training data: blue-teaming Contact-GraspNet on a mixture of original meshes and CrashGen variants improves grasp success on held-out objects by up to 21% in simulation and 30% on a real robot\, relative to a baseline fine-tuned only on the original meshes.\n\nTogether\, GRT and CrashGen show that object geometry can serve two complementary roles in making manipulation policies more reliable under geometric variation. GRT provides an active evaluation tool for identifying policy-critical geometric perturbations\, while CrashGen provides a constructive mechanism for turning those perturbations into scalable training data. Once an expensive diagnostic\, geometric failure discovery becomes a scalable source of targeted supervision for the very policies it stress-tests.\n\nCommittee:\nZackory Erickson (co-chair)\nDavid Held (co-chair)\nAndrea Bajcsy\nYufei Wang
URL:https://www.ri.cmu.edu/event/red-and-blue-teaming-for-robust-manipulation-under-geometric-variations/
LOCATION:NSH 4305
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260615T110000
DTEND;TZID=America/New_York:20260615T123000
DTSTAMP:20260707T131935
CREATED:20260604T191624Z
LAST-MODIFIED:20260604T191624Z
UID:151486-1781521200-1781526600@www.ri.cmu.edu
SUMMARY:Integrating Structured Knowledge for State and Geometry Estimation
DESCRIPTION:Abstract:\n\nReliable state and geometry estimation from limited observations is a fundamental challenge in robotics and perception. Observations are often noisy\, partial\, or ambiguous\, making estimation ill-posed without additional structure. This thesis argues that robust estimation in these regimes is enabled by integrating structured knowledge into the inference process. \nEstimation can be viewed as inferring latent state or geometry by combining three complementary forms of structure: constraints that restrict the feasible solution space\, physical models that describe how observations are generated\, and priors that regularize underdetermined solutions. \nFirst\, constraints provide structure in the solution space. We develop methods for imposing hard constraints in real-time estimation through incremental optimization\, and for learning well-conditioned measurement noise models that shape the optimization landscape. We also study constraints learned from demonstrations and analyze when they can serve as general state constraints. \nSecond\, physical models provide structure in the observation process. We study settings where sensing physics fundamentally limits observability\, and show how incorporating physics-based forward models into neural rendering enables accurate 3D reconstruction and resolves ambiguities in acoustic and multimodal sensing. \nFinally\, priors provide structure when estimation remains underdetermined even after modeling constraints and physics. We demonstrate how priors learned from data-rich visual domains can be transferred to data-scarce sensing modalities\, enabling pose-free acoustic 3D reconstruction and tactile 3D reconstruction from sparse robot touches. \nTogether\, these contributions provide a unified perspective on estimation as the integration of constraints\, physical models\, and priors. \nThesis Committee Members:\n\nMichael Kaess (Chair)  \nIoannis Gkioulekas\nShubham Tulsiani\nNikolay Atanasov (University of California – San Diego)\n\n \nURL link to thesis: Thesis Document
URL:https://www.ri.cmu.edu/event/integrating-structured-knowledge-for-state-and-geometry-estimation/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260615T130000
DTEND;TZID=America/New_York:20260615T143000
DTSTAMP:20260707T131935
CREATED:20260608T193629Z
LAST-MODIFIED:20260608T193629Z
UID:151519-1781528400-1781533800@www.ri.cmu.edu
SUMMARY:Physical Process-Informed Mapping for Robotic Exploration
DESCRIPTION:Abstract:\nMobile robots used for information gathering tasks rely on dense\, predictive mapping of large-scale regions to determine where to take measurements. Current approaches to mapping commonly rely on Gaussian process regression to spatially correlate data\, extrapolate from sparse samples\, and estimate uncertainty. However\, these approaches do not incorporate meaningful information about physical processes that contribute to the scientific observations being recorded\, instead focusing on only predicting measurements or relying on black box expert-based systems to incorporate physically meaningful information.\n\nIn this work\, we introduce physical process-informed mapping\, a framework for integrating scientific knowledge about physical processes into mapping for mobile exploration robots. This approach provides two advantages for exploration. First\, the hierarchical statistical model underlying the mapping framework is capable of providing predictions and uncertainty estimates for both the sensor measurements and the latent natural phenomena driving these measurements in a dense\, probabilistic manner. Second\, various components of the mapping approach provide methods to measure and adjust the importance of models of scientific knowledge\, enabling learning about the conditions under which these models are more or less relevant to the latent phenomenon. Taken together\, these methods improve the interpretability of mapping for autonomous scientific discovery\, enabling future robotic explorers to take advantage of a more in-depth representation of information\, and can provide increases in the predictive accuracy of mapping.\n\nThesis Committee Members:\nDavid Wettergreen (Chair)\nWennie Tabib\nMikael Kuusela\nTerrence Fong (NASA Ames Research Center)\n\n\nThesis Draft
URL:https://www.ri.cmu.edu/event/physical-process-informed-mapping-for-robotic-exploration-2/
LOCATION:3305 Newell-Simon Hall
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260616T100000
DTEND;TZID=America/New_York:20260616T113000
DTSTAMP:20260707T131935
CREATED:20260609T175151Z
LAST-MODIFIED:20260609T175151Z
UID:151545-1781604000-1781609400@www.ri.cmu.edu
SUMMARY:Aligning Observations Across Viewpoint\, Time\, and Embodiment for Agricultural Perception and Manipulation
DESCRIPTION:Abstract:\n\nAgricultural specialists are actively turning to robotic and computer vision-based systems to reduce the manual labor required to inspect and manipulate crops. These tasks require robots to perceive and interact with plants from partial\, localized observations\, often in dense and cluttered environments. For perception\, a central challenge is that crops are small\, are easily occluded\, and may change in appearance and position over time. For manipulation\, the ability to learn visuomotor policies is limited by the lack of available datasets and the difficulty of collecting robot demonstrations in the field. This thesis addresses these challenges by aligning and associating partial observations across viewpoint and time for agricultural perception\, and across viewpoint and embodiment for learning wrist-camera manipulation policies from human demonstrations. \nIn the first part of this thesis\, we develop perception-based methods for visually inspecting small crops in agriculture from limited observations. We present a 3D reconstruction pipeline for non-destructive seed counting of sorghum panicles\, a next-best-view planning approach for autonomously imaging and sizing apple fruitlets\, and a transformer-based method for spatio-temporally associating apple fruitlets across days and viewpoints. \nThe second part of this thesis shifts towards robot manipulation and learning from human demonstrations. We present a method that transforms monocular egocentric human demonstrations into wrist-camera observations and robot actions for training visuomotor policies\, without requiring depth sensors\, multi-view camera setups\, or custom data collection hardware. Building on this work\, we propose to align egocentric and wrist-camera observations and actions in latent space\, reducing reliance on explicit object tracking and image-space rendering. We further propose to incorporate visuo-tactile sensing for grape cluster inspection and harvesting. Together\, these efforts investigate how aligning observations can support agricultural robots that reason from limited visual information and learn manipulation policies when robot data is difficult to collect. \n\nThesis Committee Members:\nGeorge Kantor (Chair)\nDavid Held\nJeffrey Ichnowski\nSoumik Sarkar (Iowa State University)\n \nThesis Proposal Draft
URL:https://www.ri.cmu.edu/event/aligning-observations-across-viewpoint-time-and-embodiment-for-agricultural-perception-and-manipulation/
LOCATION:1305 Newell Simon Hall
CATEGORIES:PhD Thesis Proposal,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260616T140000
DTEND;TZID=America/New_York:20260616T153000
DTSTAMP:20260707T131935
CREATED:20260610T142430Z
LAST-MODIFIED:20260610T142621Z
UID:151566-1781618400-1781623800@www.ri.cmu.edu
SUMMARY:Scalable Oversight Across Generative Visual AI: Toward Visual Storytelling for Everyone
DESCRIPTION:Abstract: \nGenerative visual AI has advanced by scaling data and compute\, but its next bottleneck is oversight: the expert signals that evaluate\, reward\, and teach models what “good” looks like. Providing such oversight is increasingly difficult because foundation vision-language models now match or surpass most humans at the skills being judged. \nThis thesis develops scalable oversight for generative visual AI\, turning limited expert judgment into abundant and reliable evaluation\, reward\, and training signals. First\, I present VQAScore and GenAI-Bench\, evaluation standards for text-to-visual generation adopted by 100+ frontier labs such as DeepMind\, and show that a stronger judge directly improves generation through inference-time scaling. Second\, I extend oversight to video\, where precise language must be built rather than collected: CameraBench (NeurIPS’25 Spotlight) defines a structured cinematic vocabulary with professional creators\, and CHAI (CVPR’26 Highlight) introduces critique-based human-AI oversight in which experts critique model drafts rather than write from scratch. One recipe of specification\, oversight\, and post-training simultaneously improves captioning\, reward modeling\, and video generation\, enabling a small open model to surpass proprietary models like GPT and Gemini. Finally\, I present Moodio\, a deployed AI film studio with hundreds of daily active users\, whose cinematic video retrieval outperforms SOTA embedding models and measurably helps creators generate more compelling videos. I conclude with ongoing work on personalized oversight for end-to-end creative agents\, toward visual storytelling for everyone. \nThesis Committee Members: \n\nDeva Ramanan (Chair)\nDeepak Pathak\nGraham Neubig\nAli Farhadi (University of Washington and Microsoft)
URL:https://www.ri.cmu.edu/event/scalable-oversight-across-generative-visual-ai-toward-visual-storytelling-for-everyone/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260618T100000
DTEND;TZID=America/New_York:20260618T110000
DTSTAMP:20260707T131935
CREATED:20260615T135404Z
LAST-MODIFIED:20260615T135404Z
UID:151596-1781776800-1781780400@www.ri.cmu.edu
SUMMARY:Towards Fine-Grained Diagnosis of GUI Agents
DESCRIPTION:Abstract: \nGraphical User Interface (GUI) agents need strong planning—what to do next—and grounding—where to click next—to solve user tasks. Yet these agents remain unreliable\, and standard metrics such as task success or next-action accuracy often obscure why they fail. In this talk\, I argue that reliable GUI agents require fine-grained diagnosis of core agentic capabilities\, studied through two complementary directions. \nFirst\, I examine planning failures in GUI agents. Existing work can often identify planning errors\, but rarely pinpoints which component failed. This is largely because there is no structured breakdown of the skills an agent must master to plan effectively. This thesis identifies several such skills and proposes synthetic data pipelines to diagnose failures across them. Surprisingly\, even the strongest agents often fail on basic skills\, such as temporal reasoning over webpage sequences. I further show that performance on these fine-grained planning skills correlates with downstream task success\, suggesting a cheaper way to predict agent performance without expensive evaluations. \nSecond\, I diagnose failures in GUI grounding. Traditional accuracy-based metrics do not reliably expose hidden failure modes\, and I argue that sensitivity-based metrics are needed to compare models. In particular\, I find that no model consistently clicks the same UI element\, such as a calendar button\, across diverse user scenarios\, apps\, and operating systems. This raises concerns about whether these models truly understand what they click. To operationalize this diagnosis\, I propose a novel data generation pipeline and a GUI grounding diagnosis agent that automatically generates diverse instructions\, uses model feedback\, and iteratively identifies failure cases across state-of-the-art grounding models. Results show a significant mismatch between model rankings under accuracy-based metrics and our sensitivity-based metrics\, highlighting the need for more reliable agent performance measures. \nTogether\, this thesis provides diagnostics that can guide model development and advance more robust\, trustworthy interactive AI systems. Our future work delineates how each of these diagnostics can serve as a simple intervention to improve existing GUI agent pipelines. \nCommittee: \nFernando De La Torre\, co-chair\nAndrea Bajcsy\, co-chair\nLouis-Philippe Morency\nYinong (Oliver) Wang
URL:https://www.ri.cmu.edu/event/towards-fine-grained-diagnosis-of-gui-agents/
LOCATION:3305 Newell-Simon Hall
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260618T100000
DTEND;TZID=America/New_York:20260618T110000
DTSTAMP:20260707T131935
CREATED:20260615T135538Z
LAST-MODIFIED:20260615T135538Z
UID:151598-1781776800-1781780400@www.ri.cmu.edu
SUMMARY:Embodied Design\, Modeling\, and Optimization for Bio-Inspired Aquatic Robots
DESCRIPTION:Abstract:\nBio-inspired aquatic robots offer a promising route to agile and efficient locomotion in fluid environments\, where conventional rigid systems remain limited. In bio-inspired aquatic systems\, locomotion is not determined by actuation or control alone\, but instead emerges from tightly coupled interactions among body morphology\, distributed compliance\, actuation\, onboard sensing\, and the surrounding flow\, making analysis\, design\, and optimization fundamentally challenging. \nThis thesis studies these coupled problems through the development of embodied aquatic robot platforms\, variational dynamical models\, and integrated sensing strategies\, with particular emphasis on undulatory and flapping swimming\, variable-stiffness design\, strongly coupled fluid–robot interaction\, and flow sensing and feedback based on local pressure measurements. The goal is to understand how morphology\, dynamics\, and sensing can be jointly exploited within a continuous-time dynamical framework to realize robotic swimmers that are more efficient\, maneuverable\, and adaptive\, while also enabling reliable optimization of physical design\, sensing configuration\, and control through optimization-based fluid dynamic formulations and high-fidelity gradients. These ideas are further examined and validated through experiments on physical robotic platforms. \nCommittee:\nProf. Carmel Majidi\nProf. Zachary Manchester\nProf. Victoria Webster-Wood\nDr. Jeong Hun Lee
URL:https://www.ri.cmu.edu/event/embodied-design-modeling-and-optimization-for-bio-inspired-aquatic-robots/
LOCATION:Gates Hillman Center 4405
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260618T110000
DTEND;TZID=America/New_York:20260618T123000
DTSTAMP:20260707T131935
CREATED:20260608T191139Z
LAST-MODIFIED:20260608T191612Z
UID:151508-1781780400-1781785800@www.ri.cmu.edu
SUMMARY:Integrating Learning and Collaboration for Human-Robot Alignment
DESCRIPTION:Abstract: The alignment problem for robots considers how robots can learn to behave in accordance with human values. Today\, robot learning paradigms enable humans to provide data (e.g.\, preference labels or demonstrations)\, which the robot uses to update its behavior (e.g.\, reward model or policy) to better align with human intentions.  However\, the current paradigm requires the human to constantly supervise\, provide new feedback\, and—more fundamentally—perfectly understand where the robot is misaligned. Even if the robot eventually learns a perfect model of how the human wanted it to behave\, the overall human-robot interaction during alignment could have been demanding\, confusing\, or arduous for the person.\n\nThis dissertation argues that alignment should not be viewed solely as a problem of learning the correct task behavior\, but also as a problem of supporting humans throughout the alignment process itself. We propose that the learning process should be fundamentally collaborative\, in which robots actively participate in alignment through introspection\, communication\, uncertainty estimation\, sharing control\, and interaction planning.Our early works investigate how robots can learn and adapt collaborative strategies from interaction\, including adapting to partner preferences and proactively communicating robot capabilities to improve coordination. We then introduce model-agnostic uncertainty quantification methods for interactive robot learning using online conformal prediction\, enabling robots to calibrate deployment-time uncertainty from intermittent human feedback. Building on these capabilities\, we develop collaborative active learning methods in which robots request assistance when uncertain\, calibrate intervention policies using human feedback\, strategically allocate human effort across multitask learning settings\, and negotiate control handoffs through shared-control interaction mechanisms. Finally\, we study the relationship between uncertainty and human intervention behavior\, showing that robot uncertainty and human judgments of intervention necessity are only partially aligned\, motivating methods that explicitly calibrate robot assistance requests to human preferences. The outcomes are robots that know when they do not know\, communicate uncertainty\, request help strategically\, support human teachers during learning\, and adapt interaction itself as part of the alignment process.Thesis Committee Members: \n\nHenny Admoni (Chair)\nReid Simmons (Chair)\nAndrea Bajcsy\nAnirudha Majumdar (Princeton)\n\n\n\nURL link to thesis
URL:https://www.ri.cmu.edu/event/integrating-learning-and-collaboration-for-human-robot-alignment/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260622T110000
DTEND;TZID=America/New_York:20260622T120000
DTSTAMP:20260707T131935
CREATED:20260616T135241Z
LAST-MODIFIED:20260616T135241Z
UID:151605-1782126000-1782129600@www.ri.cmu.edu
SUMMARY:Towards Generalizable Embodied Navigation with Vision-Language Models
DESCRIPTION:Abstract:\nEmbodied navigation asks an autonomous agent to move through unknown environments and accomplish tasks such as finding objects or following instructions. Reliable performance in real-world settings\, from household assistance to warehouse logistics\, requires the agent to tightly integrate perception\, semantic reasoning\, and long-horizon planning under cluttered layouts\, ambiguous appearances\, and robot-specific constraints. Vision-language models (VLMs) offer rich semantic priors for this task\, but directly inserting them into the navigation loop often leads to inefficient exploration\, unstable behavior\, and limited transfer across platforms. This thesis argues that these failures stem from a multi-level misalignment between how VLMs reason and what navigation demands\, and presents four complementary contributions to address it. STRIVE shows that object navigation improves substantially when the environment is summarized as a structured graph of objects\, viewpoints\, and rooms\, letting the VLM reason at a semantic level while classical algorithms handle local exploration. SysNav extends this into a deployable system by decoupling semantic reasoning\, room-level planning\, and embodiment-specific control for robust cross-platform deployment. IntentNav shifts from prompting to learning\, showing that navigation decisions become more stable when trained with intent-aligned supervision from human demonstrations. Recognizing that object-goal search underutilizes VLM reasoning\, Goal2Pixel moves to instruction-guided navigation where longer\, compositional instructions demand richer language grounding\, and reformulates the task as pixel grounding so the model directly connects instruction understanding to executable motion. Together\, these works trace a progression from structured representation through system integration and learned decision making to instruction-guided navigation\, arguing that effective embodied navigation with VLMs requires aligning reasoning with the right representations\, architectures\, learning objectives\, and task formulations.\n \nCommittee:\nProf. Ji Zhang (Chair)\nProf. Wenshan Wang\nZhixuan Liu
URL:https://www.ri.cmu.edu/event/towards-generalizable-embodied-navigation-with-vision-language-models/
LOCATION:Newell-Simon Hall 3305
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260623T120000
DTEND;TZID=America/New_York:20260623T133000
DTSTAMP:20260707T131935
CREATED:20260610T141042Z
LAST-MODIFIED:20260610T141042Z
UID:151561-1782216000-1782221400@www.ri.cmu.edu
SUMMARY:From Margin to Center: Designing Inclusive & Equitable Service Robots with Disabled Adults
DESCRIPTION:Abstract\n\n\nService robots – autonomous systems that perform personal and professional tasks – have become a common sight in the Global North. In Human-robot interaction (HRI)\, researchers rarely consider the design implications of service robots for people with disabilities (PwDs) beyond controlled assistive contexts\, such as private homes and assisted living facilities. Nevertheless\, the purview of PwDs transcends assistive contexts: PwDs know from lived experience that autonomous innovations can cause real harm to members of the disabled community when deployed. Therefore\, considering the lived experiences of PwDs\, many of whom exist at the social margins of society\, can enable the design of inclusive service robots. Even so\, robotics design practice can be exclusionary for people who are neither roboticists nor robot design practitioners.\n\nParticipatory Design (PD)\, or cooperative design (“co-design”)\, an equitable approach aimed at shifting power relations and integrating community interests in design processes outcomes\, is becoming commonplace in HRI. However\, HRI design practitioners who work with PwDs seldom interrogate whose lived experiences are represented in the design process – commonly privileging the experiences of professionals\, such as technical experts and medical practitioners; this further encodes normative biases about how PwDs may want robots to behave around them or the types of support they may desire. Instead\, HRI design practitioners should center on the experience-as-knowledge of members of the disabled community within co-design processes.\n\nIn this thesis\, I present the lived experiences as a PD methodological framework for designing service robots with disabled adults. I demonstrate the application of this framework by co-designing service robots with disabled adults across three domains: assistive navigation\, last-mile delivery\, and mental well-being. I highlight the principles of the lived experience framework\, which reflects an onto-epistemological commitment to re-position the experience-as-expertise of PwDs\, including shifting power towards PwDs through accessible design communication; highlighting power relations\, embracing tensions\, and fostering meaningful engagements with PwDs with intersecting social marginalization; and encouraging reflexivity in the co-design process. Furthermore\, I offer design guidelines and socio-technical considerations for developers and design practitioners who wish to create inclusive service robots and equitable human-robot experiences. This work is a purposeful demonstration of the meaningful contributions made by democratizing the design of enabling\, yet disruptive\, autonomous systems and a call-to-action for HRI practitioners who seek renewed commitments to PD as a transformative framework.\n\n\nThesis Committee Members:\nAaron Steinfeld\, Chair\nJean Oh\, RI\nPatrick Carrington\, HCII\nCynthia Bennett\, Google\n\nURL link to thesis
URL:https://www.ri.cmu.edu/event/from-margin-to-center-designing-inclusive-equitable-service-robots-with-disabled-adults/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260624T140000
DTEND;TZID=America/New_York:20260624T150000
DTSTAMP:20260707T131935
CREATED:20260622T141205Z
LAST-MODIFIED:20260622T141205Z
UID:151631-1782309600-1782313200@www.ri.cmu.edu
SUMMARY:Towards Socially Intelligent Multi-Agent Systems: Zero-Shot MARL Coordination and Theory-of-Mind Benchmarking of LLM Agents for Strategic Deception
DESCRIPTION:Abstract:\nAn agent that performs well on its own may still struggle when working with others. In multi-agent environments\, success depends not only on understanding the world but also on understanding what other agents know\, intend\, and conceal. Cooperative partners follow hidden conventions\, while adversarial opponents deceive. This work argues that robust multi-agent behavior requires explicit reasoning about these hidden mental states\, and that we must measure this reasoning directly rather than simply looking at task outcomes. \nThese concepts are developed through two complementary projects. The first\, BEACON\, addresses the zero-shot coordination problem: how can an agent coordinate effectively with unfamiliar partners it has never trained with? When agents learn from offline data\, they often lock into dataset-specific conventions that work well with familiar partners but fail with new ones. BEACON is an offline-to-online learning framework that clusters offline trajectories into different conventions\, trains diverse specialists for each convention\, and uses belief-conditioned counterfactual rollouts to adapt online. On 2- and 3-player Hanabi\, BEACON achieves state-of-the-art zero-shot coordination performance while using up to five times fewer training frames than strong online baselines. It also coordinates with human partners comparably to a leading online method. The second project\, AmongUs-X\, asks whether large language model agents genuinely deceive or merely win through other means. Built on the social-deduction game Among Us and spanning 21 model families across more than 8\,700 games\, the benchmark elicits agents’ beliefs at fixed points during meetings. This yields eight Theory-of-Mind metrics measuring detection\, deception\, influence\, and grounding. Win-rate-derived ratings track crewmate detection but miss impostor deception entirely. However\, the elicited beliefs remain well-calibrated\, enabling direct mechanism-level evaluation. \nBoth projects arrive at the same conclusion: high self-play scores can hide poor coordination\, and high win rates can hide absent deception. Modeling other agents’ hidden information and measuring that modeling explicitly is essential for building socially intelligent multi-agent systems and evaluating them reliably. \nCommittee:\nDr. Katia Sycara (chair)\nDr. Jiaoyang Li\nRenos Zabounidis
URL:https://www.ri.cmu.edu/event/towards-socially-intelligent-multi-agent-systems-zero-shot-marl-coordination-and-theory-of-mind-benchmarking-of-llm-agents-for-strategic-deception/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260625T130000
DTEND;TZID=America/New_York:20260625T140000
DTSTAMP:20260707T131935
CREATED:20260622T141019Z
LAST-MODIFIED:20260622T141019Z
UID:151629-1782392400-1782396000@www.ri.cmu.edu
SUMMARY:Data-Driven Representation and Reasoning for Aviation Safety
DESCRIPTION:Abstract:\nAviation safety analysis increasingly benefits from large-scale operational trajectory data\, yet raw motion traces alone are insufficient for understanding safety-critical events on the airport surface. The significance of an aircraft’s motion depends on the structured operational environment in which it occurs\, including runways\, taxiways\, hold-short lines\, and interactions among multiple agents over time.\n\nThis thesis presents a broader framework for AI-enabled aviation safety and develops several components to support it. Amelia-42 dataset provides a large-scale operational data foundation for studying airport surface movement. Trajectory alignment with airport surface graphs recovers snapped routes\, route-transition predictions\, candidate conflict points\, and time-to-node estimates. To support interpretable safety reasoning\, World2Rules introduces a neural-symbolic pipeline that learns human-interpretable runway-incursion rules from incident reports and nominal operational observations. Critical Scenario Identification mines real runway-incursion reports and evaluates whether models can identify the critical agents and timestamps in safety-relevant interactions. Together\, these components demonstrate how trajectory data can be transformed into structured representations\, interpretable safety rules\, and evaluation methodologies that support the identification\, explanation\, and analysis of safety-critical aviation scenarios.\n\nCommittee:\nDr. Sebastian Scherer (Chair)\nDr. Jean Oh\nJunwon Seo
URL:https://www.ri.cmu.edu/event/data-driven-representation-and-reasoning-for-aviation-safety/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260625T133000
DTEND;TZID=America/New_York:20260625T143000
DTSTAMP:20260707T131935
CREATED:20260623T134347Z
LAST-MODIFIED:20260623T134347Z
UID:151646-1782394200-1782397800@www.ri.cmu.edu
SUMMARY:Brain-Aligned Tactile Representations for Dexterous Robot Learning
DESCRIPTION:Abstract: Touch is the essential sensory modality through which animals and robots physically negotiate the world. While much of robotic touch focuses on the capabilities of currently available tactile hardware\, this thesis asks a more general question: what forms of tactile processing and representation could allow robots to approach the dexterity of animals? This thesis argues that simulated force and torque provide a privileged representation of touch: one that is biologically grounded\, computationally scalable\, and effective for dexterous behavior. \n\n\n\n\nFor biological touch\, the rodent vibrissal pathway provides a tractable model system for studying tactile processing.  We train task-optimized temporal networks on realistic whisker force/torque sequences to identify shapes and compare their internal representations against recordings from rats’ barrel cortex. We find that convolutional recurrent models (ConvRNNs) align very closely with neural data\, and contrastive self-supervision with tactile-specific augmentations matches supervised alignment as a label-free proxy.\nIn robotic touch\, we test whether the same class of representation is useful for dexterous behavior. We develop a GPU-parallel tactile sensor simulator that exposes a family of tactile abstractions under one interface\, from binary contact and per-taxel force/torque to marker displacement and temperature\, fast enough to serve as the front-end for dexterous reinforcement learning. For in-hand manipulation tasks\, we find that sensor placement matters more than sensor type or resolution\, that whole-hand coverage closes most of the gap to a privileged teacher. Across 3 dexterous tasks\, per-taxel force/torque emerges as the most useful and robust observation.\nTogether\, these results argue that force and torque are not merely convenient signals to simulate\, but a biologically grounded and behaviorally effective representation of touch. With recurrence and broad spatial coverage\, simulated force and torque provide a common representational substrate for understanding tactile computation in brains and building tactile intelligence in robots.\n\nCommittee:\nProf. Aran Nayebi (Chair)\nProf. Katerina Fragkiadaki\nAkash Sharma
URL:https://www.ri.cmu.edu/event/brain-aligned-tactile-representations-for-dexterous-robot-learning/
LOCATION:Gates Hillman Center 4405
CATEGORIES:MSR Thesis Presentation,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260626T100000
DTEND;TZID=America/New_York:20260626T113000
DTSTAMP:20260707T131935
CREATED:20260617T153533Z
LAST-MODIFIED:20260617T170801Z
UID:151617-1782468000-1782473400@www.ri.cmu.edu
SUMMARY:RI PhD Thesis Defense - Angela Chen
DESCRIPTION:Date: June 26\, 2026\nTime: 10:00 – 11:30 AM\nLocation: GHC – Room 4405\nZoom Link\nType: Ph.D. Thesis Defense\nWho: Angela Chen\nTitle: Behavioral Modeling of Interpersonal Dynamics as Controllable Agentic Systems: Empirically-grounded Adaptive Virtual Patients for Psychotherapy Training \nAbstract: The need for mental health care continues to outpace the supply of trained psychotherapists. Training is itself a bottleneck because supervision time is limited and trainees get few chances to practice repeatedly on realistic cases with feedback. Standardized patients played by actors are expensive and hard to scale\, and the hardest clinical moments are not ones a trainee can safely rehearse with real clients. Large language models (LLMs) have made interactive virtual patients practical\, yet most current systems are not empirically grounded: they carry little psychologically meaningful state and focus almost exclusively on one-on-one therapy\, leaving multi-party interaction such as couples therapy unsupported. \nThis thesis treats the design of an LLM-based virtual patient (VP)\, an interactive agent whose behavior must remain coherent over a long interaction\, as a control system. Rather than being steered by a fixed prompt\, the VP’s behavior is governed by a dynamics controller\, so its behavioral trajectory can be inspected and adjusted. The thesis develops this idea in three studies. The first builds LLM-based measures of therapist behavior and client behavior\, applies them to a large corpus of clinical transcripts\, and uses structural equation modeling to estimate how therapist micro-skills affect a client’s disclosure and emotion. The second turns these estimates into an adaptive virtual patient: detectors read the therapist’s empathy and exploration\, a dynamics controller updates the patient’s disclosure state using the estimated coefficients\, and the language model generates a reply at that level of disclosure. The third study extends to a multimodal\, multi-agent interaction system in which two VPs and the therapist interact through a stage controller built around the demand–withdraw pattern common in couples therapy. \nMethodologically\, this thesis shows how to build human–AI interaction systems for a specialized domain that requires explicit behavioral modeling. Concretely\, it delivers a validated pipeline for measuring therapy process\, two training systems for individual and couples therapy\, and a corpus of therapist–agent interactions. Together these broaden access to deliberate practice for trainees\, using AI to strengthen therapists’ preparation rather than substitute for it. \nCommittee:\nHaiyi Zhu (Chair)\, Carnegie Mellon University\nSherry Tongshuang Wu\, Carnegie Mellon University\nAaron Steinfeld\, Carnegie Mellon University\nHolly Swartz\, University of Pittsburgh Medical Center \nLINK TO THESIS DRAFT DOCUMENT
URL:https://www.ri.cmu.edu/event/ri-phd-thesis-defense-angela-chen/
LOCATION:Gates Hillman Center 4405
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260630T093000
DTEND;TZID=America/New_York:20260630T110000
DTSTAMP:20260707T131935
CREATED:20260622T175815Z
LAST-MODIFIED:20260622T175815Z
UID:151640-1782811800-1782817200@www.ri.cmu.edu
SUMMARY:Spatial Reasoning and Semantic Representations for Autonomous Exploration and Object Search
DESCRIPTION:Abstract: Autonomous robot exploration and object search in unknown environments are fundamental capabilities in robotics\, with applications ranging from search and rescue to structural inspection. A central challenge in both tasks is that robots often must make decisions based on information they have not yet directly observed–reasoning about unexplored space\, predicting future information gain\, or inferring where a target object is likely to be found. This thesis argues that autonomous robots can achieve efficient exploration and long-horizon object search by leveraging spatial and semantic representations of increasing expressiveness that enable reasoning beyond immediate sensor observations. \nWe develop this argument through a progression of representations. We begin by extracting geometric cues–such as rooms\, doorways\, and spatial connectivity–directly from sensor data\, and show how a team of robots can use these compact structural representations to coordinate room-based exploration in indoor environments. We then move beyond what has already been observed: by learning to predict unobserved portions of an environment\, a robot can estimate the information it would gain from exploring a candidate location or path before actually visiting it\, and a team of robots can use these predictions to coordinate exploration and manage information sharing under communication constraints. Finally\, we extend reasoning beyond the geometric altogether\, introducing a persistent 3D semantic representation that grounds open-vocabulary vision-language understanding into a structure capable of perceiving objects well beyond the range of onboard depth sensors. This representation enables a single robot to perform long-horizon object search in large outdoor environments\, and allows a team of robots to coordinate search behavior by sharing a sparse\, communication-efficient form of this representation rather than dense maps. \nWe demonstrate this progression through six contributions spanning indoor and outdoor environments and single- and multi-robot deployments\, showing that as representations grow richer–from geometric structure\, to predicted space\, to persistent semantic memory–robots become capable of more efficient exploration\, more effective object search\, and more naturally coordinated teamwork. \n\nThesis Committee:\nSebastian Scherer (Chair)\nYonatan Bisk\nWenshan Wang\nGraeme Best (University of Technology Sydney)\nMicah Corah (Colorado School of Mines)\n\nThesis draft
URL:https://www.ri.cmu.edu/event/spatial-reasoning-and-semantic-representations-for-autonomous-exploration-and-object-search/
LOCATION:Newell-Simon Hall 4305
CATEGORIES:PhD Thesis Defense,Student Talks
END:VEVENT
END:VCALENDAR