Student Talks
Towards Robotic Convoying in Unstructured Environments
Abstract: Multi-agent robotic teaming is the only realistic solution to many large-scale autonomous operations. Conventionally, operations are modeled as a set of tasks that are largely decoupled from each other and the environment at execution time. However, this operational model fails when the successful execution of a task requires multiple agents to synchronize their actions [...]
Learning to Create 3D Content
Abstract: With the popularity of Virtual Reality (VR), Augmented Reality (AR), and other 3D applications, developing methods that let everyday users capture and create their own 3D content has become increasingly essential. However, current 3D creation pipelines often require either tedious manual effort or specialized capture setups. Additionally, resulting assets often suffer from baked-in lighting, [...]
Learning From People: Assistive Robotics and Optimization from Preferences
Abstract: Robotic algorithms rarely come perfectly pre-configured, and when choosing parameters, tradeoffs must often be made: between performance and robustness; efficiency and safety; the comfort of the user and the comfort of bystanders. While engineers can tune parameters by hand or carefully design reward functions to optimize over, this is not always a straightforward task. [...]
Towards Practical Vision-and-Language Navigation Systems Through 3D Referential Grounding
Abstract: As robots transition toward practical deployment as collaborative agents in human environments, it becomes essential to improve language-conditioned environmental understanding. A vision-and-language navigation (VLN) system must adapt to both the types of language used and the actions expected by a human collaborator. Often, a single sentence containing spatial relations and semantic attributes---e.g., “fetch the [...]
Object-Centric Goal Prediction Towards Precise, Generalizable Placement
Abstract: Recent advances in robotic manipulation have established the effectiveness of learning from demonstration. However, while end-to-end policies excel in expressivity and flexibility for complex skill learning, they struggle to efficiently adapt to task variations in geometry and configuration. Conversely, alternative approaches relying on object-centric goal prediction offer improved generalizability, but rely on restrictive assumptions [...]
Advancing Multimodal Sensing and Robotic Interfaces for Chronic Care
Abstract: The healthcare system prioritizes reactive care for acute illnesses, often overlooking the ongoing needs of individuals with chronic conditions that require long-term management and personalized care. Addressing this gap through technology can empower patients to better manage their conditions, greatly enhancing quality of life and independence. Multimodal sensing, incorporating inertial, acoustic, and vision-based sensors, [...]
Towards Automating Accessibility in Digital Authoring Workflows
Abstract: Most digital content today remains inaccessible to people with disabilities, who make up 16% of the global population. A long-standing challenge in accessible computing is ensuring digital authors consistently provide the metadata required to make their content accessible through assistive technologies. Despite numerous specialized accessibility tools, authors often lack the time, training, or incentive [...]
Vision-based Human Motion Modeling and Analysis
Abstract: Modern computer vision has achieved remarkable success in tasks such as detecting, segmenting, and estimating human pose in images and videos—often reaching or even surpassing human-level performance. However, significant challenges remain in predicting and analyzing future human motion. This thesis explores how vision-based methods can improve the fidelity and accuracy of human motion modeling [...]
ReferEverything: Towards Segmenting Everything We Can Speak of in Videos Using Text-to-Video Diffusion Models
Abstract: In this thesis we present REM, a framework for segmenting a wide range of concepts in video that can be described through natural language. Our method unlocks the universal visual-language mapping learned by video diffusion models on Internet-scale data by fine-tuning them on small-scale Referring Object Segmentation datasets. Our key insight is preserving the [...]
Building richer 3D maps: utilizing a hybrid geometry representation and auxiliary inputs in neural surface reconstruction
Abstract: As robots are increasingly deployed in real-world environments, their perception systems face growing demands. Tasks such as tracking and manipulation require maps with both high spatial fidelity and detailed object-level organization, which must be delivered faster to support timely decision-making and control. Concurrently, advances in vision foundation models allow us to build powerful prediction [...]