My research focuses entirely on applying machine learning to control. I want to find out just how autonomous we can make robots, factories and other complex control systems.
A vast range of complex systems are becoming amenable to computer decision making. Most such systems have sensors, and so through their lifetime they experience a stream of sensor data. The machine analysis of this data stream can lead to the system improving its own internal knowledge of its behavior. And in turn it can use the improved knowledge to design improved control laws for itself. How can we do this mining of the data stream to extract as much predictive accuracy as possible, how can we do it quickly, and how can we then make the system develop good control laws with what it has learned?
I work with these questions by developing a number of experimental algorithms, described shortly, and be testing them with a somewhat large variety of real-world problems. The large variety is to find out whether the algorithms are sufficiently autonomous that I can get away without being an expert in the application domains. Some completed, ongoing, and future applications include a billiards robot, robot juggling, candy bar manufacture, RC helicopters, decision making in economics, textile manufacture, RC toy cars, pinball, electricity pricing, car engine emission control and computer games.
The kind of algorithms I work with lie at the borders between artificial intelligence, statistics, control theory and heuristic search. I make heavy use of a class of techniques called memory-based algorithms which have some useful properties. Predictions are made with interpolations from local previous experiences, and can be performed very quickly using geometric techniques, can circumvent the problems of a one-to-many inverse in redundant systems and avoid the necessity of retraining behaviors which have not been used recently.
Another key to fast learning is exploration. Both the learning effort and the collection of data should be concentrated on the critical parts of the problem. Furthermore, real-world tasks can impose severe demands on learning controllers. I am developing the general memory-based learning (GMBL) system to tackle many of these difficulties. This technique centers on efficiently harnessing an amazingly powerful, but normally hopelessly expensive, statistical technique called leave-one-out cross-validation. This can help the systems automatically select the most appropriate function approximator, the best trade-off between fitting the data and filtering noise, and can be made to detect irrelevant, coupled or partially relevant sensors and actuators. Inventing good techniques to perform large-scale cross-validation searches is an exciting area with ongoing research opportunities.
To generate improved control laws autonomously, the system must be able to make long-term predictions, and plan to achieve long-term goals. This planning must happen whilst simultaneously learning the environment. Weak-but-general search methods such as Dynamic Programming and A* search can prove helpful here, while retaining wide applicability to many domains. Some recent research involves scheduling search control during on-line planning to minimize wasted computations, and there are further interesting possibilities there. Ongoing research extends this to the PartiGame algorithm, in which exploration and planning are scheduled in a multi-resolution manner. A recursive partitioning of state-space adapts itself in real time to yield fine detail in the critical regions, while remaining at a coarse granularity elsewhere.
I am also interested in working on several new developments of the research described above, ranging from (1) developing a quickly re-configurable meta-robot system with which we can produce, test and learn new control tasks with a turnover period of days instead of months, (2) developing formal theories of classes of control tasks, with associated proofs of convergence of appropriate machine learning algorithms, (3) developing new learning algorithms for certain classes of harder problems, and (4) tailoring efficient versions of certain optimization techniques, including genetic algorithms, to the control domain.
|Research Interest Keywords|
|artificial intelligence, machine learning|
|The Robotics Institute is part of the School of Computer Science, Carnegie Mellon University.|
Contact Us | Update Instructions