Carnegie Mellon University Robotics Institute Research Guide

Machine Learning

Machine learning (ML) permeates the Institute and plays an integral role in many robotics sub-fields as well as in many on-going projects. Several of our faculty (Atkeson, Bagnell, Dubrawski, Gordon, Guestrin and Schneider) specialize in the foundations of learning. Each focuses on questions about learning within autonomous control and sensing. These horizontal technologies are used throughout the institute and include:

• Reinforcement Learning and Adaptive Control (Atkeson, Bagnell, Gordon, and Schneider),
• Imitation Learning (Atkeson, Bagnell, Simmons, Veloso),
• Mining of massive high dimensional data streams (Dubrawski, Guestrin, Schneider),
• Autonomous Science (Dubrawski, Schneider, and Wettergreen), and
• Multi-agent Learning (Gordon, Sycara, Veloso, Stentz).

We review below just a few of the areas within Robotics for which machine learning has become a core technology:

• Learning has become a core part of research efforts in computer vision. Data-driven techniques are being used to tackle problems here that have proven very difficult to model, but where large quantities of data are readily available. The ultimate goal is to use the ever-growing amount of stored visual information (digital photo albums, webcams, movies, etc.) - where possible annotated by humans - to learn, understand, and re-synthesize the visual world around us. Data-driven techniques include the "Visual Memex" (Efros), which aims to utilize the huge amount of existing visual data to discover links and connections between visual elements. The goal is to "explain" novel visual signals not in terms of hard categories ("car", "chair", "city"), but rather in terms of what has been seen before. Collaborations throughout the Institute include developing structured prediction learning techniques that are appropriate for computer vision (Bagnell, Hebert, Kanade), applying learning to visual behavior analysis (De la Torre), to identifying and sorting (Bagnell, Bares, Fromme) strawberry plant crops.
• In response to frustration with problems for which it has proven difficult to engineer and manually program solutions, machine learning has in recent years become an integral part of field robotics. Imitation learning techniques, including inverse optimal control methods, (Atkeson, Bagnell, Simmons, Stentz, Veloso) have played an increasingly important role in achieving a vision of “programming-by-demonstration” where behaviors that have proven difficult to craft by hand are instead created by expert demonstration. These tools have been applied to a variety of platforms from humanoid robotics to legged locomotion to off-road rough-terrain mobile robot navigation.
• Supervised learning techniques have become standard within field robotics and are used for tasks as diverse as detecting and rejecting dust and snow (Stentz and Bagnell), to identifying vegetation and obstacles (Hebert, Bagnell, Stentz) in rough terrain. These techniques are relied upon now in many applications of 3D scene analysis. Self-supervised learning, where a robot is able to generate training examples to improve its own performance, have played a key role. This includes work that enables robots to effectively teach themselves to interpret long-range ambiguous sensor data by a-priori training using easier-to-interpret data captured at close range (Bagnell, Stentz, Singh). It also includes recent efforts at modeling vehicle dynamics by machine learning techniques (Kelly).
• Machine learning is playing a key role in assistive and medical technologies. This ranges from the development of large scale cell tracking over time-lapse microscope imagery (Kanade), to the development of learning-based driver assistance tools (Bagnell, Steinfeld, Kanade). Machine learning plays a particularly large role addressing the assistive technology within the Quality of Life Technology Center (NSF ERC) (Atkeson, Bagnell, De la Torre, Guestrin, Hebert, Kanade) both for detection and recognition and recommendation tasks.

Autonomous science

Much of the recent effort within the Institute can be categorized as Assisted or Autonomous Science where robots serve as helpers and co-discoverers of knowledge. This includes the development of active learning methods that attempt to efficiently find cosmological parameters of the universe via testing observational and simulated data (Schneider). RI (Schneider) has successfully commercialized a system that automatically recognizes psycho-active drugs by their effects on mice and is able to autonomously run experiments to identify substances with potential therapeutic effect. Learning approaches to modeling spatial phenomena have been applied to algal bloom detection and monitoring (Dolan, Guestrin) and autonomous robotic exploration of geologic properties to guide robots (Wettergreen) toward informative areas for exploration. Field experiments demonstrate that these systems are able to successfully operate on multi square-kilometer scales.

Multi-agent Learning

Machine learning techniques are playing an increasing role in the analysis and synthesis of multi-robot and multi-agent behavior. Learning methods have been developed (Gordon, Sycara) that enable multi-agents to adapt to the shifting landscape of other agents and collectively achieve equilibrium strategies. No-regret learning tools have been developed as a mechanism for multi-agent planning (Gordon), and close relationships have been explored with learning in market-based distributed control systems. (Dias, Stentz).

Outlook

Learning is part of the research of a large and growing share of the Robotics faculty. Dozens of graduate students include machine learning in their thesis research. The dissemination throughout the Institute of research in machine learning has led to many collaborations:

• assistive technology (Atkeson, Bagnell, Guestrin, Kanade, Steinfeld),
• massive databases of movement primitives (Pollard and Faloutsos),
• learning helicopter control (Amidi, Bagnell, Kanade and Schneider),
• multi-agent coordination and negotiation (Gordon, Sycara, Veloso), and
• learning for field mobile robots (Stentz, Bagnell, Hebert, Kelly).

In recent years, sponsors and partners have become increasingly savvy about the potential of machine learning technology and have funded a variety of programs that either explicitly or implicitly call for research in this area. Trends suggest an increase in these efforts and some sponsors (e.g. DARPA, ARL) have indicated machine learning is a long-term priority. Machine Learning within RI is tightly integrated with machine learning research activities through SCS and Carnegie Mellon. For instance, many of our faculty (e.g. Atkeson, Bagnell, Mostow, Schneider, Dubrawski, Gordon, Guestrin, Sycara) are affiliated with SCS's Machine Learning Department (MLD).

Given the unique challenges in robotics and the ubiquity of learning-based solutions to robotics problems, the future prospects for machine learning with the Institute seems assured. There remains, however, an important question of how the Robotics Institute should position itself with respect to sibling organizations within Carnegie Mellon. Two paths immediately present themselves:

• Robotics could cede core learning research and depend instead on collaborations with the Machine Learning and Statistics departments within the university. This presents significant risks, however, in assuring the research niches crucial for the Institute are addressed.
• A more modest approach would be to continue to focus on both basic and applied learning research that remains focused on ML problems.

In the same way that the Institute actively recruits faculty, staff and students with expertise in electrical and mechanical engineering and computer science, RI can treat learning as a core competency and a priority to grow and develop. While we do not plan to specialize in all areas of machine learning, specific research areas exist for which the Institute is particularly well-suited. For example, we are poised to become world leaders in the study of learning for decision-making and control as well as learning in real-time systems. This may be further encouraged by faculty and special faculty hires in these areas.

At the same time we can greatly benefit from interaction with our institutional siblings; it is likely, for example, that an increasing number of Robotics faculty will become affiliated with the ML department. Additionally, the tremendous scale of data developed within the field of robotics will provide natural opportunities for collaboration with faculty interested in large scale and distributed computation.

Education

The institute currently offers a variety of graduate classes that focus explicitly on machine learning. Statistical Techniques in Robotics provides a foundation in both Bayesian probabilistic modeling techniques as well on developing and using online learning algorithms for robotic applications. Learning-based Methods in Vision, explores how data driven techniques and machine learning can enhance machine understanding of the visual world. Adaptive Control and Reinforcement Learning focuses on the role Machine Learning plays in the making decisions and developing strategies and controllers. Additionally, RI faculty regularly teach core classes within the Machine Learning department.

Continue Reading: Manipulation & Control