Home/Sebastian Scherer

Sebastian Scherer

Systems Scientist
Email: basti@andrew.cmu.edu
Office: NSH 2113
Phone: (412) 589-9581
Personal Homepage

The availability and advantages of distributed electric propulsion have led to a new design space for flying vehicles. The rise of small multi-copter vehicles and more recently the concept of on-demand urban air transport gives autonomous vehicles an advantage because pilots negate the potential cost savings, limit vehicle designs, and corresponding market.

For example, multi-copters have the potential to rapidly reach a large set of relevant viewpoints for inspection, cinematography, and mapping. However, to reach this potential these vehicles need to be aware and react to their limitations, adapt and learn what “relevant” means, and need to respond to the changing world in a safe manner.

In my research, I use the terms model and adaptation broadly: Models can be the robot dynamics, error dynamics of state estimation, wind model, planning abstractions, or perception representations for example. Adaptation based on “error signals” can be an integrator, self-supervised learning, or offline training of CNNs.

Current approaches to improve operations of these vehicles fall short. Either they can reactively keep the vehicles safe, have high performance in the nominal case, or are able to adapt but not perform relevant missions. These shortcomings are because either approaches rely on static models and do not utilize the feedback available or they only rely on training of policies.

I am interested in overcoming these limitations to robot autonomy by increasing the amount of feedback signals utilized and automating adaptation to these signals in a safe way. This motivates me to answer these two fundamental research questions:

  1. How can one improve the safety of adaptive autonomy systems without sacrificing performance and respecting real-time constraints?
  2. What are appropriate adaptation mechanism and signals that can improve the capability for real-world (flying) robots while maintaining the ability to give safety guarantees?

These research questions address fundamental limitations of today’s autonomous systems where the complex interactions between different modules lead to fragile systems and unexpected behavior and the desired behavior is difficult to capture.

I am studying these questions by researching and applying machine learning to the traditional perception/state estimation/and planning components of an autonomy system that relies on strong models as well as blending the approach with model-free approaches where appropriate.

Over the last several years, we have found that adaptive model-based approaches are powerful and high-performance when we applied them to autonomous flight. Models are powerful because they enable the abstraction of irrelevant parts, the ability to predict and reason based on these predictions. However, we have also found that models are fragile and will fail if the modeling error is too large.

Any useful model will have some simplifying assumptions and if these assumptions are valid the overall system will work well. Modelling errors are typically easy to find; however, how to adapt to these errors is not obvious.

On the other hand, model-free approaches have the advantage of not placing an artificial constraint on the complexity of the model or where model complexity needs to be expressed; however, the performance outside of the observed boundaries is unknown and validating the correctness is difficult.

Over the next couple of years, I see us achieving a new level of safety and capability for flying systems. My team and I will achieve this by a shift and adaptation of current strong model-based paradigms to a mixed (model/model-free) paradigm that is able to adapt in real time at multiple frequencies and levels. This will lead to new applications beyond what is possible now such as on-demand urban air transport. The methods we develop to address these challenges will lead to a new research agenda of safe real-time adaptation algorithms. In the longer term, these adaptive systems will lead to the next set of questions on how to achieve systems that behave according to the designers with minimal external inputs.

Publications

Displaying 74 Publications
Learning Heuristic Search via Imitation
Mohak Bhardwaj, Sanjiban Choudhury and Sebastian Scherer

Proceedings of the 1st Annual Conference on Robot Learning, Vol. 78, pp. 271-280, October, 2017
Looking Forward: A Semantic Mapping System for Scouting with Micro-Aerial Vehicles
Daniel Maturana, Sankalp Arora and Sebastian Scherer

Carnegie Mellon University, International Conference on Intelligent Robots and Systems (IROS), September, 2017
Real-time Semantic Mapping for Autonomous Off-Road Navigation
Daniel Maturana, Po-Wei Chou, Masashi Uenoyama and Sebastian Scherer

Conference Paper, Carnegie Mellon University, Field and Service Robotics (FSR), September, 2017
Real-Time Semantic Mapping for Autonomous Off-Road Navigation
Daniel Maturana, Po-Wei Chou, Masashi Uenoyama and Sebastian Scherer

Conference Paper, Carnegie Mellon University, Field and Service Robotics, September, 2017
Season-Invariant Semantic Segmentation with A Deep Multimodal Network
Dong-Ki Kim, Daniel Maturana, Masashi Uenoyama and Sebastian Scherer

Conference Paper, Carnegie Mellon University, Field and Service Robotics, September, 2017
DROAN – Disparity-space Representation for Obstacle AvoidaNce
Geetesh Dubey, Sankalp Arora, Sebastian Scherer

Conference Paper, September, 2017
Robust Crack Detection in Concrete Structures Images using Multi-Scale Enhancement and Visual Features
Xiangzeng Liu, Yunfeng Ai and Sebastian Scherer

Conference Paper, 2017 IEEE International Conference on Image Processing (ICIP 2017), September, 2017
Wire Detection using Synthetic Data and Dilated Convolutional Networks for Unmanned Aerial Vehicles
Ratnesh Madaan, Daniel Maturana and Sebastian Scherer

Conference Paper, IEEE/RSJ International Conference on Intelligent Robots and Systems, September, 2017
Randomized Algorithm for Informative Path Planning with Budget Constraints
Sankalp Arora and Sebastian Scherer

Conference Paper, International Conference on Robotics and Automation (ICRA), June, 2017, May, 2017
Robust Localization and Localizability Estimation with a Rotating Laser Scanner
Weikun Zhen, Sam Zeng and Sebastian Scherer

Conference Paper, IEEE International Conference on Robotics and Automation 2017, May, 2017
Smooth Trajectory Optimization in Wind: First Results on a Full-Scale Helicopter
Vishal Dugar, Sanjiban Choudhury and Sebastian Scherer

Conference Paper, AHS Forum, Vol 73, 2017, March, 2017
A KITE in the Wind: Smooth Trajectory Optimization in a Moving Reference Frame
Vishal Dugar, Sanjiban Choudhury and Sebastian Scherer

Conference Paper, 2017 IEEE International Conference on Robotics and Automation (ICRA), February, 2017
A Multi-Sensor Fusion MAV State Estimation from Long-Range Stereo, IMU, GPS and Barometric Sensors
Yu Song, Stephen T. Nuske and Sebastian Scherer

Journal Article, Sensors, Vol. 17, No. 11, December, 2016
Detecting cars in aerial photographs with a hierarchy of deconvolution nets
Satyaki Chakraborty, Daniel Maturana and Sebastian Scherer

Tech. Report, CMU-RI-TR-16-60, Robotics Institute, Carnegie Mellon University, November, 2016
Nonholonomic motion planning in partially unknown environments using vector fields and optimal planners
Guilherme Augusto Silva Pereira, Sanjiban Choudhury and Sebastian Scherer

Conference Paper, Congresso Brasileiro de Automatica (CBA), October, 2016
Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments
Shichao Yang, Yu Song, Michael Kaess and Sebastian Scherer

Conference Paper, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2016
Robust Autonomous Flight in Constrained and Visually Degraded Shipboard Environments
Zheng Fang, Shichao Yang, Sezal Jain, Geetesh Dubey, Stephan Roth, Silvio Mano Maeta, Stephen T. Nuske, Yuzhang Wu and Sebastian Scherer

Journal Article, Carnegie Mellon University, Journal of Field Robotics, September, 2016
Robust Autonomous Flight in Constrained and Visually Degraded Shipboard Environments
Zheng Fang, Shichao Yang, Sezal Jain, Geetesh Dubey, Stephan Roth, Silvio Mano Maeta, Stephen T. Nuske, Yuzhang Wu and Sebastian Scherer

Journal Article, Journal of Field Robotics, September, 2016
Modeling and Control of Coaxial UAV with Swashplate Controlled Lower Propeller
Richard Lee, Koushil Sreenath and Sebastian Scherer

Tech. Report, CMU-RI-TR-16-52, Robotics Institute, Carnegie Mellon University, August, 2016
Kinodynamic Motion Planning on Vector Fields using RRT*
Guilherme Augusto Silva Pereira, Sanjiban Choudhury and Sebastian Scherer

Tech. Report, CMU-RI-TR-16-35, Robotics Institute, Carnegie Mellon University, July, 2016
A Framework for Optimal Repairing of Vector Field-based Motion Plans
Guilherme Augusto Silva Pereira, Sanjiban Choudhury and Sebastian Scherer

Conference Paper, Proceedings of the 2016 International Conference of Unmanned Aircraft Systems (ICUAS), pp. 261-266, June, 2016
Constrained CHOMP using Dual Projected Newton Method
Sanjiban Choudhury and Sebastian Scherer

Tech. Report, CMU-RI-TR-16-17, Robotics Institute, Carnegie Mellon University, May, 2016
List Prediction Applied To Motion Planning
Abhijeet Tallavajhula, Sanjiban Choudhury, Sebastian Scherer and Alonzo Kelly

Conference Paper, 2016 IEEE International Conference on Robotics and Automation (ICRA), May, 2016
Real-time 3D Scene Layout from a Single Image Using Convolutional Neural Networks
Shichao Yang, Daniel Maturana and Sebastian Scherer

Conference Paper, International Conference on Robotics and Automation (ICRA), May, 2016
Regionally Accelerated Batch Informed Trees (RABIT*): A Framework to Integrate Local Information into Optimal Path Planning
Sanjiban Choudhury, Jonathan D. Gammell, Timothy D. Barfoot, Siddhartha Srinivasa and Sebastian Scherer

Conference Paper, 2016 IEEE International Conference on Robotics and Automation (ICRA), May, 2016
Mixed-Initiative Control of a Roadable Air Vehicle for Non-Pilots
Michael C. Dorneich, Emmanuel Letsu-Dake, Sanjiv Singh, Sebastian Scherer, Lyle J. Chamberlain and Marcel Bergerman

Journal Article, Carnegie Mellon University, Journal of Human-Robot Interaction, Vol. 4, No. 3, pp. 38-61, December, 2015
Multi-Scale Convolutional Architecture for Semantic Segmentation
Aman Raj, Daniel Maturana and Sebastian Scherer

Tech. Report, CMU-RI-TR-15-21, Robotics Institute, Carnegie Mellon University, pp. 14, October, 2015
Online Safety Verification of Trajectories for Unmanned Flight with Offline Computed Robust Invariant Sets
Daniel Althoff, Matthias Althoff and Sebastian Scherer

Conference Paper, Carnegie Mellon University, IEEE/RSJ International Conference on Intelligent Robots and Systems, September, 2015
Recognition of Human Group Activity for Video Analytics
Jaeyong Ju, Cheoljong Yang, Sebastian Scherer and Hanseok Ko

Conference Paper, Carnegie Mellon University, Advances in Multimedia Information Processing, pp. 735, September, 2015
VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition
Daniel Maturana and Sebastian Scherer

Conference Paper, Carnegie Mellon University, IEEE/RSJ International Conference on Intelligent Robots and Systems, September, 2015
Theoretical Limits of Speed and Resolution for Kinodynamic Planning in a Poisson Forest
Sanjiban Choudhury, Sebastian Scherer and J. Andrew (Drew) Bagnell

Conference Paper, Carnegie Mellon University, Robotics Science and Systems, July, 2015
Autonomous Exploration and Motion Planning for an Unmanned Aerial Vehicle Navigating Rivers
Stephen T. Nuske, Sanjiban Choudhury, Sezal Jain, Andrew D. Chambers, Luke Yoder, Sebastian Scherer, Lyle J. Chamberlain, Hugh Cover and Sanjiv Singh

Journal Article, Carnegie Mellon University, Journal of Field Robotics, June, 2015
Autonomous Exploration for Infrastructure Modeling with a Micro Aerial Vehicle
Luke Yoder and Sebastian Scherer

Conference Paper, Carnegie Mellon University, Field and Service Robotics, June, 2015
Learning a Context-Dependent Switching Strategy for Robust Visual Odometry
Kristen Holtz, Daniel Maturana and Sebastian Scherer

Conference Paper, Carnegie Mellon University, Field and Service Robotics, June, 2015
Robust Autonomous Flight in Constrained and Visually Degraded Environments
Zheng Fang, Shichao Yang, Sezal Jain, Geetesh Dubey, Silvio Mano Maeta, Stephan Roth, Sebastian Scherer, Yu Zhang and Stephen T. Nuske

Conference Paper, Carnegie Mellon University, Field and Service Robotics, June, 2015
Emergency Maneuver Library – Ensuring Safe Navigation in Partially Known Environments
Sankalp Arora, Sanjiban Choudhury, Daniel Althoff and Sebastian Scherer

Conference Paper, Carnegie Mellon University, 2015 IEEE International Conference on Robotics and Automation, May, 2015
PASP: Policy Based Approach for Sensor Planning
Sankalp Arora and Sebastian Scherer

Conference Paper, Carnegie Mellon University, 2015 IEEE International Conference on Robotics and Automation, May, 2015
Real-time Onboard 6DoF Localization of an Indoor MAV in Degraded Visual Environments Using a RGB-D Camera
Zheng Fang and Sebastian Scherer

Conference Paper, Carnegie Mellon University, 2015 IEEE International Conference on Robotics and Automation, May, 2015
The Dynamics Projection Filter (DPF) – Real-Time Nonlinear Trajectory Optimization Using Projection Operators
Sanjiban Choudhury and Sebastian Scherer

Conference Paper, Carnegie Mellon University, IEEE International Conference on Robotics and Automation, May, 2015
The Planner Ensemble: Motion Planning by Executing Diverse Algorithms
Sanjiban Choudhury, Sankalp Arora and Sebastian Scherer

Conference Paper, Carnegie Mellon University, IEEE Conference on Robotics and Automation, May, 2015
3D Convolutional Neural Networks for Landing Zone Detection from LiDAR
Daniel Maturana and Sebastian Scherer

Conference Paper, Carnegie Mellon University, International Conference on Robotics and Automation, March, 2015
Connected Invariant Sets for High-Speed Motion Planning in Partially-Known Environments
Daniel Althoff and Sebastian Scherer

Conference Paper, Carnegie Mellon University, 2015 IEEE International Conference on Robotics and Automation, March, 2015
High-precision Autonomous Flight in Constrained Shipboard Environments
Shichao Yang, Zheng Fang, Sezal Jain, Geetesh Dubey, Silvio Mano Maeta, Stephan Roth, Sebastian Scherer, Yu Zhang and Stephen T. Nuske

Tech. Report, CMU-RI-TR-15-06, Robotics Institute, Carnegie Mellon University, February, 2015
Experimental Study of Odometry Estimation Methods using RGB-D Cameras
Zheng Fang and Sebastian Scherer

Conference Paper, Carnegie Mellon University, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September, 2014
Learning Motion Planning Assumptions
Anirudh Vemula, Sanjiban Choudhury and Sebastian Scherer

Tech. Report, CMU-RI-TR-14-14, Robotics Institute, Carnegie Mellon University, August, 2014
Visual Odometry in Smoke Occluded Environments
Aditya Agarwal, Daniel Maturana and Sebastian Scherer

PhD Thesis, CMU-RI-TR-15-07, Robotics Institute, Carnegie Mellon University, July, 2014
Robust Multi-Sensor Fusion for Micro Aerial Vehicle Navigation in GPS-Degraded/Denied Environments
Andrew D. Chambers, Sebastian Scherer, Luke Yoder, Sezal Jain, Stephen T. Nuske and Sanjiv Singh

Conference Paper, Carnegie Mellon University, In Proceedings of American Control Conference, Portland, OR, June, 2014
A Principled Approach to Enable Safe and High Performance Maneuvers for Autonomous Rotorcraft
Sankalp Arora, Sanjiban Choudhury, Sebastian Scherer and Daniel Althoff

Conference Paper, Carnegie Mellon University, AHS 70th Annual Forum, Montre ́al, Que ́bec, Canada, May 20-22, May, 2014
The Planner Ensemble and Trajectory Executive: A High Performance Motion Planning System with Guaranteed Safety
Sanjiban Choudhury, Sankalp Arora and Sebastian Scherer

Conference Paper, Carnegie Mellon University, AHS 70th Annual Forum, Montre ́al, Que ́bec, Canada, May, 2014
Autonomous River Exploration
Sezal Jain, Stephen T. Nuske, Andrew D. Chambers, Luke Yoder, Hugh Cover, Lyle J. Chamberlain, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, Field and Service Robotics, Brisbane, December, 2013
Robocopters to the Rescue
Lyle Chamberlain and Sebastian Scherer

Magazine Article, Carnegie Mellon University, September, 2013
Autonomous Emergency Landing of a Helicopter: Motion Planning with Hard Time-Constraints
Sanjiban Choudhury, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, AHS Forum 69, May, 2013
First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles
Tomoyuki Mori and Sebastian Scherer

Conference Paper, Carnegie Mellon University, International Conference on Robotics and Automation, May, 2013
Infrastructure-free Shipdeck Tracking for Autonomous Landing
Sankalp Arora, Sezal Jain, Sebastian Scherer, Stephen T. Nuske, Lyle J. Chamberlain and Sanjiv Singh

Conference Paper, Carnegie Mellon University, International Conference on Robotics and Automation, May, 2013
RRT*-AR: Sampling-Based Alternate Routes Planning with Applications to Autonomous Emergency Landing of a Helicopter
Sanjiban Choudhury, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, International Conference on Robotics and Automation, May, 2013
Sparse Tangential Network (SPARTAN): Motion Planning for Micro Aerial Vehicles
Hugh Cover, Sanjiban Choudhury, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, International Conference on Robotics and Automation, May, 2013
Realtime Alternate Routes Planning:The RRT*-AR Algorithm
Sanjiban Choudhury, Sebastian Scherer and Sanjiv Singh

Tech. Report, CMU-RI-TR-12-27, Robotics Institute, Carnegie Mellon University, December, 2012
Autonomous landing at unprepared sites by a full-scale helicopter
Sebastian Scherer, Lyle J. Chamberlain and Sanjiv Singh

Journal Article, Carnegie Mellon University, Robotics and Autonomous Systems, September, 2012
First Results in Autonomous Landing and Obstacle Avoidance by a Full-Scale Helicopter
Sebastian Scherer, Lyle J. Chamberlain and Sanjiv Singh

Conference Paper, Carnegie Mellon University, ICRA, May, 2012
River mapping from a flying robot: state estimation, river detection, and obstacle mapping
Sebastian Scherer, Joern Rehder, Supreeth Achar, Hugh Cover, Andrew D. Chambers, Stephen T. Nuske and Sanjiv Singh

Journal Article, Carnegie Mellon University, Autonomous Robots, Vol. 32, No. 5, May, 2012
Multiple-Objective Motion Planning for Unmanned Aerial Vehicles
Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '11), September, 2011
Perception for a River Mapping Robot
Andrew D. Chambers, Supreeth Achar, Stephen T. Nuske, Joern Rehder, Bernd Manfred Kitt, Lyle J. Chamberlain, Justin Haines, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '11), September, 2011
Navigation and Control for Micro Aerial Vehicles in GPS-Denied Environments
Rudolph Molero Fernandez, Sebastian Scherer, Lyle J. Chamberlain and Sanjiv Singh

Tech. Report, CMU-RI-TR-10-08, Robotics Institute, Carnegie Mellon University, June, 2011
Low-Altitude Operation of Unmanned Rotorcraft
Sebastian Scherer

PhD Thesis, CMU-RI-TR-11-03, Robotics Institute, Carnegie Mellon University, May, 2011
Self-Supervised Segmentation of River Scenes
Supreeth Achar, Bharath Sankaran, Stephen T. Nuske, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, ICRA 2011, May, 2011
Self-Aware Helicopters: Full-Scale Automated Landing and Obstacle Avoidance in Unmapped Environments
Lyle J. Chamberlain, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, AHS Forum 67, March, 2011
Online Assessment of Landing Sites
Sebastian Scherer, Lyle J. Chamberlain and Sanjiv Singh

Conference Paper, Carnegie Mellon University, AIAA Infotech@Aerospace 2010, April, 2010
Efficient C-Space and Cost Function Updates in 3D for Unmanned Aerial Vehicles
Sebastian Scherer, David Ferguson and Sanjiv Singh

Conference Paper, Carnegie Mellon University, Proceedings International Conference on Robotics and Automation, May, 2009
Flying Fast and Low Among Obstacles: Methodology and Experiments
Sebastian Scherer, Sanjiv Singh, Lyle J. Chamberlain and Mike Elgersma

Journal Article, Carnegie Mellon University, The International Journal of Robotics Research, Vol. 27, No. 5, pp. 549-574, May, 2008
Flying Fast and Low Among Obstacles
Sebastian Scherer, Sanjiv Singh, Lyle J. Chamberlain and Srikanth Saripalli

Conference Paper, Carnegie Mellon University, Proceedings International Conference on Robotics and Automation, April, 2007
Tartan Racing: A Multi-Modal Approach to the DARPA Urban Challenge
Christopher Urmson, Joshua Anhalt, J. Andrew (Drew) Bagnell, Christopher R. Baker, Robert E. Bittner, John M. Dolan, David Duggins, David Ferguson, Tugrul Galatali, Hartmut Geyer, Michele Gittleman, Sam Harbaugh, Martial Hebert, Thomas Howard, Alonzo Kelly, David Kohanbash, Maxim Likhachev, Nick Miller, Kevin Peterson, Raj Rajkumar, Paul Rybski, Bryan Salesky, Sebastian Scherer, Young-Woo Seo, Reid Simmons, Sanjiv Singh, Jarrod M. Snider, Anthony (Tony) Stentz, William (Red) L. Whittaker and Jason Ziglar

Tech. Report, Robotics Institute, Carnegie Mellon University, DARPA Grand Challenge Tech Report, April, 2007
Learning Obstacle Avoidance Parameters from Operator Behavior
Bradley Hamner, Sanjiv Singh and Sebastian Scherer

Journal Article, Carnegie Mellon University, Special Issue on Machine Learning Based Robotics in Unstructured Environments, Journal of Field Robotics, Vol. 23, No. 12-Nov, pp. 1037-1058, December, 2006
Learning to Drive Among Obstacles
Bradley Hamner, Sebastian Scherer and Sanjiv Singh

Conference Paper, Carnegie Mellon University, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2663 - 2669, October, 2006
Model Checking of Robotic Control Systems
Sebastian Scherer, Flavio Lerda and Edmund M. Clarke

Conference Paper, Carnegie Mellon University, Proceedings of the 8th International Symposium on Artificial Intelligence, Robotics and Automation in Space (iSAIRAS), September, 2005
Mailing Address:

Carnegie Mellon University
Robotics Institute
5000 Forbes Avenue
Pittsburgh, PA 15213

2017-11-10T15:09:47+00:00