Home/Jean Hyaejin Oh

Jean Hyaejin Oh

Systems Scientist
Email: hyaejino@andrew.cmu.edu
Office: NSH 4521
Administrative Assistant: Stephanie Matvey

My research interests lie in the area of AI in robotics including autonomy, language understanding, multimodal perception, path planning, and machine learning. I am interested in creating persistent robots that can co-exist with humans in shared environments, learning to improve themselves over time through continuous training, exploration, and interactions. Toward this general goal, I am currently focusing on the translation of information among vision, language, and planning so that robots can 1) perceive better by fusing with verbal information provided by humans, 2) describe what they have observed in natural language, 3) generate semantic navigation plans by following complex directions, 4) learn to navigate in a socially compliant way, and 5) explain their past, current, and future actions and plans. As for the communication modalities for robots, I am primarily interested in verbal (text or speech) communication in natural language, but I am also interested in alternative ways for robots to express what goes on in their internal states, e.g., communication modalities studied in the arts and design community such as media arts.

Currently, I am leading the perception and the learning tasks for the DARPA Aircrew-Labor In-Cockpit Automation System (ALIAS) program (co-PI) that aims to bring intelligence to cockpits through semantic perception and learning new skills from observing experienced pilots. I am also a PI on the intelligence architecture subtask of the ARL RCTA program where my team’s work on language understanding on robot navigation won the Best Cognitive Robotics Paper Award at the IEEE International Conference on Robotics and Automation (ICRA) in 2015. My newest project is the US DoD – Korea MOTIE (co-PI) collaboration on robotics technologies for disaster response, where I will be leading the semantic map construction by leveraging text data from social media and crowdsourcing.

Publications

Displaying 15 Publications
Modeling Cooperative Navigation in Dense Human Crowds
Anirudh Vemula, Katharina Muelling and Jean Hyaejin Oh

Conference Paper, Proceedings of the International Conference on Robotics and Automation (ICRA) 2017, February, 2017
Integrated intelligence for human-robot teams
Jean Hyaejin Oh, Menglong Zhu, Sangdon Park, Thomas M. Howard, Matthew R. Walter, Daniel Barber, Oscar Romero, Arne Suppe, Luis Ernesto Navarro-Serment, Felix Duvallet, Abdeslam Boularias, Jerry Vinokurov, Terence Keegan, Robert Dean, Craig Lennon, Barry Bodt, Marshall Childers, Jianbo Shi, Kostas Daniilidis, Nick Roy, Christian Lebiere, Martial Hebert and Anthony (Tony) Stentz

Conference Paper, The 2016 International Symposium on Experimental Robotics (ISER 2016), October, 2016
Path Planning in Dynamic Environments with Adaptive Dimensionality
Anirudh Vemula, Katharina Muelling and Jean Hyaejin Oh

Conference Paper, Proceedings of the Ninth International Symposium on Combinatorial Search (SoCS-2016), July, 2016
Inferring door locations from a teammate’s trajectory in stealth human-robot team operations
Jean Hyaejin Oh, Luis Ernesto Navarro-Serment, Arne Suppe, Anthony (Tony) Stentz and Martial Hebert

Conference Paper, Carnegie Mellon University, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2015
Grounding Spatial Relations for Outdoor Robot Navigation
Abdeslam Boularias, Felix Duvallet, Jean Hyaejin Oh and Anthony (Tony) Stentz

Conference Paper, Carnegie Mellon University, 2015 IEEE International Conference on Robotics and Automation (ICRA), May, 2015
Toward Mobile Robots Reasoning Like Humans
Jean Hyaejin Oh, Arne Suppe, Felix Duvallet, Abdeslam Boularias, Jerry Vinokurov, Luis Ernesto Navarro-Serment, Oscar Romero, Robert Dean, Christian Lebiere, Martial Hebert and Anthony (Tony) Stentz

Conference Paper, Carnegie Mellon University, Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), January, 2015
Inferring Maps and Behaviors from Natural Language Instructions
Felix Duvallet, Matthew Walter, Thomas Howard, Sachithra Hemachandra, Jean Hyaejin Oh, Seth Teller, Nicholas Roy and Anthony (Tony) Stentz

Conference Paper, Carnegie Mellon University, International Symposium on Experimental Robotics, June, 2014
Chapter 11: Probabilistic Plan Recognition for Proactive Assistant Agents
Jean Hyaejin Oh, Felipe Meneguzzi and Katia Sycara

Book Section/Chapter, Carnegie Mellon University, Plan, Activity, and Intent Recognition: Theory and Practice, February, 2014
Probabilistic plan recognition for proactive assistant agents
Jean Hyaejin Oh, Felipe Meneguzzi and Katia Sycara

Journal Article, Elsevier, Waltham MA, January, 2014
Enhancing robot perception using human teammates
Jean Hyaejin Oh, Arne Suppe, Anthony (Tony) Stentz and Martial Hebert

Conference Paper, Carnegie Mellon University, Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2013), May, 2013
Prognostic normative reasoning
Jean Hyaejin Oh, Felipe Meneguzzi, Katia Sycara and Timothy Norman

Journal Article, Carnegie Mellon University, Engineering Applications of Artificial Intelligence, Vol. 26, No. 2, pp. 863-872, March, 2013
An agent architecture for prognostic reasoning assistance
Jean Hyaejin Oh, Felipe Meneguzzi, Katia Sycara and Norman S Kerman

Conference Paper, Carnegie Mellon University, In Proc. 22nd Int. Joint Conf. on Artificial Intelligence, (IJCAI2011), Barcelona, Spain, pages 2513-2518, July, 2011
A few good agents: Multi-agent social learning
Jean Hyaejin Oh and Stephen Smith

Conference Paper, Carnegie Mellon University, Proc. of the Seventh International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 339-346, May, 2008
Learning from Main Streets: A machine learning approach identifying neighborhood commercial districts
Jean Hyaejin Oh, Jie-Eun Hwang, Stephen Smith and Kimberle Koile

Book Section/Chapter, Carnegie Mellon University, Innovations in Design & Decision Support Systems in Architecture and Urban Planning, pp. 325 - 340, June, 2006
Learning user preferences in distributed calendar scheduling
Jean Hyaejin Oh and Stephen Smith

Book Section/Chapter, Carnegie Mellon University, Practice and Theory of Automated Timetabling V, LNCS, Vol. 3616, March, 2005
Mailing Address:

Carnegie Mellon University
Robotics Institute
5000 Forbes Avenue
Pittsburgh, PA 15213

2017-11-22T09:54:49+00:00