RI PhD Thesis Proposal - Yunus Seker - Robotics Institute Carnegie Mellon University
Loading Events

PhD Thesis Proposal

November

23
Mon
Muhammet Yunus Seker PhD Student Robotics Institute,
Carnegie Mellon University
Monday, November 23
1:00 pm to 3:00 pm
Newell-Simon Hall 4305
RI PhD Thesis Proposal – Yunus Seker
Date: November 23, 2026
Time: 1:00–3:00 PM ET  
Type: RI PhD Thesis Proposal
Zoom Link: LINK
Location: NSH 4305  
Title: Grounding Task-Specialized Models in Object Priors for Generalizable Robot Manipulation
Who: Muhammet Yunus Seker
Abstract:Generalizable robot manipulation requires robots to perform diverse tasks involving unfamiliar objects and physical interactions. General-purpose models aim to provide broad task coverage, while specialized models and skills can achieve high performance within narrower operating conditions. However, achieving reliable generalization without extensive training or independently developing specialized solutions for each new task remains challenging. This thesis investigates how exploiting object- and interaction-centric structural priors can enable the construction, selection, and reuse of modular, task-specialized models and manipulation skills.The completed research explores this principle through two complementary directions. In predictive modeling, Sum-GP-UCB exploits dependencies between individual observations and the parameters of the objects involved in each interaction to decompose system identification into specialized reward models, supporting efficient optimization and incremental incorporation of new observations. Model Preconditions learns the applicability of predictive models from the physical context of interactions, enabling efficient model selection and data-efficient adaptation from simulation to real-world conditions.

In manipulation skills, GTA-1 exploits task-relevant object keypoints and axes to transfer previously specified skills to novel objects without any retraining. GTA-2 extends this representation through four VLM agents that autonomously decompose task instructions, compose reusable controllers into task-specialized skills, assign execution parameters, and ground task-relevant geometric features in the observed scene. By separating abstract skill synthesis from scene-specific grounding, GTA-2 enables generated skills to be reused across different scene configurations while supporting targeted refinement of individual components. Real-robot evaluations on 14 manipulation tasks demonstrate zero-shot skill generation and generalization across different scene configurations.

Building on these contributions, the proposed research investigates how robots can accumulate and improve their capabilities through continued interaction. It explores agentic orchestration for progressively developing stable, reusable manipulation programs; persistent object-centric scene understanding that retains accumulated knowledge and uses interactive perception and exploration to reduce uncertainty; and compositional observation mechanisms for monitoring execution and supporting adaptation. The broader objective is to enable robots to progressively develop specialized capabilities around recurring objects and interactions while retaining the flexibility to handle unfamiliar situations.

 

COMMITTEE MEMBERS

Oliver Kroemer (Chair)
Jonathan Francis
Yonatan Bisk
Leslie Pack Kaelbling (Massachusetts Institute of Technology)