Graphics enhanced version of this site
Faculty Research Guide
Jump to: Faculty by name | Faculty by interests | Full text version (for printing)
My goal is to build vision-guided autonomous flying robots. Vision allows such robots to serve as intelligent eyes-in-the-sky suitable for numerous applications including law enforcement, search and rescue, aerial mapping and inspection, and movie making. Vision for flight control encompasses a broad range of research areas. In particular, I am interested in vision-based object detection and tracking, optical position estimation, inertial navigation, GPS, and non-linear system modeling. I am the co-founder (with Takeo Kanade) of the Autonomous Helicopter Laboratory where we have developed a number of autonomous helicopters for our ongoing research in such areas. We have developed two six degree-of-freedom testbeds for indoor helicopter control experiments. In addition, we have built two fully autonomous helicopters for outdoor vision-guided flight research. These prototypes are mid-sized (14 ft. long) crop dusters outfitted with custom-designed vision systems and an array of sensors including accelerometers, angular sensors, and GPS receivers. Each can fly autonomously using an on-board visual odometer which estimates motion by locking-on to and visually tracking ground objects. In addition, we have integrated inertial and angular sensors and a GPS receiver with vision for high quality position estimation allowing autonomous landing, takeoff, and smooth trajectory following. Currently, we are developing a number of new flight systems. In particular, we are developing real-time object trackers for high speed aerial pursuit, compact laser scanners for obstacle detection and 3D mapping, and versatile control systems for transitioning to high-speed flight maneuvers.
Labs: Helicopter Lab
Projects: Autonomous Helicopter, PerceptOR (NREC), and Reconfigurable Vision Machine
Research Interests: biological vision
My primary goal is to enable robotics as a technology of choice for hazardous applications and in inhospitable environments. I pursue this goal through persistent robotic systems research and development, technology advancements, field testing, and breakthrough demonstrations. I focus on the science, research, and engineering of mobile robots for outdoor environments and especially on issues of robotic mobility. I am also interested in the methods and practice for successful robot prototyping. The fundamental questions of my research are how to design better robots for mobility, robustness, and reliability, and what are the practical methods and tools to predict, quantify, and verify robot performance.
My objective is to create and deliver significant robotic systems in a sustained pace. A “significant” system is one that introduces a new way to solve a problem or leads to measurable improvements over the existing state-of-art. Naturally, such a system would also entail some innovation. Another measure of significance is the degree to which others adapt and use a system’s technologies and development lessons. Those would include mobility, control, configuration design, and hardware architecture among others. The ultimate measure of significance is a system’s acceptability and use by the customer. By virtue of meeting these success metrics I aspire to make a measurable impact on robotics.
Two major themes distinguish my research: robotic mobility and robot prototyping. Mobility is the cornerstone of performance of any field robot. Robotic mobility goes far beyond traditional locomotion as defined by the ability to drive, steer, and climb over obstacles. Robotic mobility relates to the ability to enable real-time locomotion control; position the robot and precisely follow navigation paths; smooth motions induced to onboard sensors and computers; and effectively deploy and position payloads or work tools. I investigate methods for analyzing and quantifying robotic mobility, and the essential relationships between robot mechanics and control. I pursue this line of work for wheeled, legged and unconventional robotic locomotion systems such as multi-pod hoppers and hybrid locomotors.
Robot prototyping captures the complete process from robot concept creation to field validation. At the highest level, I am interested in the systematization of the configuration (i.e. geometric or preliminary design) and the design processes which are common to any system prototyping effort. I develop practical methodologies for creating and evaluating robot configuration topologies, and the methods for analytical predictions of ground performance. Of key significance are relationships between classical mobility and robotic functionalities such as sensing and control as they affect robot design. I also research methods for improving the design engineering of robotic systems to achieve higher robustness and reliability. Lastly, I research metrics that would improve any phase of robot prototyping and I leverage on development and testing lessons to extrapolate on the applicability of such metrics in the design of future robotic systems.
Projects: Tactical Unmanned Ground Vehicle (TUGV), Sun Synchronous Navigation, Robotic Antarctic Meteorite Search, Lunar Rover Initiative, LORAX: Life on Ice, Robotic Antarctic Explorer, Life in the Atacama, Inflatable Rover Technologies, Autonomous Rover Technologies, Atacama Desert Trek, and Gladiator
Research Interests: teleoperation, systems engineering, space robotics, sensors, mobile robots, mechatronics, mechanisms, legged locomotion, human-computer interaction, hazardous environments, field robotics, design, control, and artificial intelligence
I am an experimentalist in the field of robot learning, specializing in the learning of challenging dynamic tasks such as juggling. I am also working on applying ideas and techniques from robotics to intelligent environments. I combine designing learning algorithms with exploring their behavior in implementations on actual robots and in intelligent environments. My research interests include nonparametric learning, reinforcement learning, learning from demonstration, and modeling human behavior. For more detail please see my personal home page.
Labs: Robotics Education Lab
Projects: Navigation Among Movable Obstacles, Informedia Digital Video Library, Dynamic Biped, Learning Locomotion, and Quality of Life Technology Center
Research Interests: quality-of-life technology, planning, motion planning, manipulation, machine learning, legged locomotion, intelligent environments, humanoid robotics, human-computer interaction, entertainment robotics, control, and artificial intelligence
I am interested in "closing the loop" on complex systems; that is, I am interested in designing algorithms that allow systems to observe their own operation and improve performance. My belief is that the border land between planning, control and computational learning is particularly rich with research challenges and potential to make real, immediate impact on applications. I'm particularly interested in systems for which we can obtain at best a partial model. To this end, I'm excited about extending research tools that come from information theory, statistics, control theory, statistical physics and optimization.
At the moment, I am particularly focused on two areas in machine learning. First I am working on applications of learning and decision making applied to mobile robotics. Second, I am interested in developing rich, structured probabilistic models that are appropriate for both making and learning decisions.
Labs: Auton Lab
Projects: Urban Challenge, UGCV PerceptOR Integrated, Tartan Racing, Learning Locomotion, Federation of Intelligent Robotic Explorers Project, Autonomous Helicopter, Quality of Life Technology Center, and Auton Project
Research Interests: planning, mobile robots, machine learning, control, and artificial intelligence
I seek to enable the widespread adoption of roboticized mobile equipment. The theme of my work is the synthesis of basic robotics technologies to create robust systems and the identification of appropriate target applications. Within a decade, automated machines contending with complex tasks and contingencies will operate for long duration without human input. I foresee automated machines succeeding in the challenging markets of mining, agriculture, construction and hazardous operations.
My research focus is the full cycle conception to testing of intelligent machines for mining, construction, utility and agricultural applications. My objective is to design, build and test large forceful robotic systems that will evolve to be used in industry to solve tasks in complex, dynamic (i.e., changing) and dangerous environments. Specific research topics are robot system configuration, mechanism design, reliability and perception devices for outdoor environments. I have specific interest in robotic system design and reliability and improving the processes of design, production and testing of prototype robotic systems to enable predictable levels of reliability. I have led research efforts in automated volcanic exploration, automated surface and underground mining and the development of a variety of ranging sensors.
My dual role is the Director of the National Robotics Engineering Center (NREC). This mission of the Center is to adapt terrestrial robotics technology from research centers and labs to enable and create viable robotics products within U.S. industry. As Director, I am in an ideal role to identify and solve the problems that limit the commercial practicality and adoption of mobile robotics.
Projects: Unmanned Ground Combat Vehicle (UGCV), Terregator, Robotic All Terrain Lunar Exploration Rover, Motion Free Scanning Radar, UPI Teleoperation System, M2000: A Semi-Autonomous System for High-Speed Paint Removal, Demeter, Dante II, Autonomous Loading System, Underground Mining Operator Assist, Asbestos Pipe-Insulation Removal Robot System, Ambler, UGCV PerceptOR Integrated, and Gladiator
Research Interests: range finders, mobile robots, mechanisms, legged locomotion, hazardous environments, field robotics, construction, actuators, active perception, and 3-D perception
My research interests include the following:
Labs: Helicopter Lab and Tele-Supervised Autonomous Robotics
Projects: Wide Area Prospecting Using Supervised Autonomous Robots and Autonomous Helicopter
Research Interests: space robotics, planning, motion planning, mobile robots, manipulation, field robotics, entertainment robotics, education, control, and active perception
My research is focused on building intelligent systems for automated manufacturing. The ultimate goal is to design an artifact and send the design to an automated production facility that would produce it. I am currently investigating several research issues in the design area:
After a design is complete, it is passed to a production facility. Here, a setup and process plan is prepared along with descriptions of machine setups. The setup descriptions and the stock material are sent to the manufacturing workstations that automatically assemble the part into the fixtures. The manufacturing step is completed for that setup and for all subsequent setups, until the final part is correct and complete.
And finally, we are building the hardware environment that will sufficiently structure the environment to permit the automated production of one-off production quality parts.
Labs: Rapid Manufacturing Lab
Projects: Tooling Planner, Stacking Planner, Product Decomposition, Motion Planner, Grasping Planner, Fine Motion Planner, Bend Sequence Planner, BendCad Modeler, and Backgage Hardware
Research Interests: manufacturing
My research interests focus on multi-robot systems that operate in adversarial environments. My main interests relate to
I am also involved in the following SCS project:
Labs: MultiRobot Lab and TechBridgeWorld
Projects: Mobile Autonomous Robot Software, Treasure Hunt: Pickup Teams, Robotic Soccer, and Camera Assisted Meeting Event Observer
Research Interests: robot soccer, multi-agent systems, mobile robots, machine learning, entertainment robotics, and artificial intelligence
My education and research interests straddle the border between computational theory and mechatronic engineering, makes mathematical principles accessible to engineering, and reaches out to practioners in the chosen application fields. In my group’s research, rigorous mathematical results enable engineering advancements while the practical aspects of implementation drive theoretical pursuit. My program centers on two foci: highly articulated systems and coverage tasks. These foci touch upon fundamentals in robotics including: topological methods, control of mechanical systems, design, mapping, and differential geometry. This work is directly tied into search and rescue, de-mining, auto-body painting, and medical surgery. These endeavors require the interaction between people and technology and thus, I seek to exploit its benefits and understand the barriers of this interaction.
My research group has constructed a variety of snake robots which can exploit their many internal degrees of freedom to thread through tightly packed volumes accessing locations that people and conventional machinery otherwise cannot. Three challenges facing snake robot research are design, path planning and locomotion. Since we are interested in search and rescue (I am an Associate Director for the Center for Robotic Assisted Search and Rescue), we designed our robots to maneuver in three-dimensions and posses a small cross-sectional diameter.
Once the snake robot is built, it still requires control. Simple engineering hacks alone are not sufficient to coordinate the internal degrees of freedom to allow for purposeful motion. Essentially, the robot must plan in a non-Euclidean multi-dimensional space. Our approach uses a topological map of the space, which reduces planning from a multi-dimensional search problem to a one-dimensional search. In 1997, I received the NSF Career award to develop a topological map based on a retract-like structure for rod-shaped and convex-body robots operating in a non-Euclidean configuration space. In collaboration with the Johnson Space Center, we have applied this approach to AERCam, a free-flying robot.
Our topological mapping techniques have the added benefit that they induce well-defined sensor-based control laws that can direct a robot to explore an unknown space with provable guarantees. However, one of the critical challenges in exploring unknown spaces is localization while mapping, or the so-called simultaneous localization and mapping (SLAM) problem. We developed a hierarchical SLAM technique that works well in large spaces. Specifically, we use a topological map to divide the free space into regions where high-resolution maps, corresponding to each edge of the topological map, can be created. This approach scales well because we never create a large high-resolution map, but rather represent a large space with a collection of small high-resolution maps tied together by a topological map. Scott Thayer’s group in the FRC used this approach to map underground mines.
The topological approaches do not apply to “snake robots,” per say, but to elephant trunk robots which generally have a fixed base and move around in three dimensions. Snake robots, on the other hand, must coordinate their internal degrees of freedom and interact with their environment to propel themselves in a desired direction. Here, we analyze a broad spectrum of mechanical systems, those governed by both conservation of momentum and non-holonomic constraints. Specifically, we take recourse to the fundamental mechanics to develop gait evaluation techniques that are based on a geometric/kinematic contribution and a dynamic contribution. These evaluation techniques are then used to synthesize kinematic, dynamic, and kinodynamic gaits for systems whose Lagrangian is invariant under group actions. The principle contribution here is that we determined the “right” space inside of which one can intuitively design and optimize gaits for mechanical systems such as snake robots.
The symbiosis of applied math and engineering has already had an impact on the robotic search for mines. My group has developed provably complete techniques for coverage path planning, a method that determines a path for a robot to follow so that the robot passes over every point in a target region. The mathematical guarantee is critical in mine-sweeping where missing one mine makes the mission a failure. In 1999, the Office of Naval Research awarded me its Young Investigator Program award to further this work. Our approach uses a cell decomposition, a representation where the environment is divided into cells and a graph is formed encoding the adjacent relationships (topology) among the cells. Coverage in each cell is “simple,” and thus complete coverage is achieved by visiting each cell in the decomposition. In many situations time may not permit covering an environment completely. However, if the planner has a probabilistic a priori understanding of how mines are laid, it can opportunistically guide the robot. For patterned mine fields, we developed a Bayesian method of efficiently decoding the parameters that describe the minefield. Once these are known, the robot can cover a fraction of the target region and locate most of the mines. This work is done with Prof. Mark Schervish in Statistics.
In collaboration with Prof. Rizzi at Carnegie Mellon, we applied similar coverage technology to the application of auto-body painting with the Ford Motor Company to expedite the paint operation while minimizing hazardous waste. This is a coverage problem in three-dimensions, but must also respect the dynamics of the paint applicator and effects due to curvature of the surface. Already, we have demonstrated utility of this work on car body parts painted at Ford.
Also, in collaboration with Prof. Rizzi, we combined the above described cell decomposition concept with sequential composition to develop a new type of hybrid control. Recall that hybrid controls is a combination of discrete and continuous planning where an arbitrator selects which continuous feedback control law to invoke. Here, we define policies in cells and as the robot travels through its free space, our method uses the adjacency relationships among the cells to select which policy to invoke. In a sense, instead of using path planning to determine a “thin” path, we now have a procedure which determines a “fat” path or vector field along which the robot “flows” from a start location to a goal.
In the above research endeavors, my group has brought the realities and uncertainties of mechanical systems to the precision of applied math and computer science. This philosophy of using construction and implementation to reinforce theory permeates my courses and advising. My graduate students participate in rigorous reading groups on basic mathematic theory (e.g., see http://differentialgeometry.org) and they construct mechanical artifacts. Moreover, my students have first-hand experience with the applications: we have participated in mock search and rescue scenarios, we have fielded our de-mining robot, we have performed experiments at Ford, etc. My four Ph.D. graduates all have a strong theoretical contribution as well as a thorough experiment.
In my undergraduate robotics course, students use LEGO robotics lab modules, developed by my students and myself, to reinforce the theoretical materials presented in class. Via construction of an artifact, the lab experiences seriously motivate students to synthesize lessons, critically explore beyond them, and then think creatively with meta-lessons. The course strikes a balance between conventional one-way lectures and modern constructionism. This course sits as the centerpiece for the robotics minor, which I developed and currently direct at Carnegie Mellon
I am the lead author of a motion planning textbook which makes the deep fundamental underpinnings of robotics accessible to the novice. At the same time, the book focuses on implementation issues and ties together low-level concepts with theoretical concerns. I use this book in a newly designed course on motion planning (http://voronoi.sbp.ri.cmu.edu/~motion).
My near-term research goals include multi-robot systems, hybrid controls, and medical devices. In terms of hybrid controls, we are using conventional motion planning algorithms to develop synthesis tools for hybrid controls of systems possessing non-trivial dynamics and non-holonomic constraints. With medical devices, we already have developed a snake robot for minimally invasive surgery, where the device can reach deeper into the body without a need for large incisions. We have tested our prototype several times in a live pig with Dr. Zenati at UPMC. With Dr. Wolf, we have developed a mechanism for bone shaving; this draws from our paint work where instead of depositing we are removing material.
Understanding motion is important. We will apply the rigorous fundamentals of robotics to modeling biological motion. Already, we have begun using our kinematic analysis to model the knee as a parallel mechanism. I also find the social aspects of human-technology interaction to be interesting and important. My future work will consider: how technology can be used to teach basic principles and what are the barriers for technology acceptance. Already, the LEGO-based robotics course has given us insight on how technology can teach some basic math. For the auto body painting work, the medical robotics, and the urban search and rescue experiences, I have seen how “social” issues impact the decision to use technology. For example, we never would have performed so many experiments and developed a keen medical robotics perspective without Dr. Zenati and his willingness to “push” this technology, and introduce it to his colleagues. My future work seeks to model the non-economic aspects of technology acceptance and its barriers.
Labs: Biorobotics and Biomedical Robotics Lab
Projects: Visual Localization, Vacuum Cleaning Robots, The Arc Transversal Median, Snake Robot Design, Simultaneous Localization and Mapping, Sensor Based Coverage of Unknown Planar Environments, Search and Rescue, Robotic Demining, Retract-like structures for SE(2) and SE(3), Retract-like structures for Euclidian Spaces, Probabilistic Coverage, Motion Planning for Snake Robots, Modular Distributed Manipulator System, Mobile Robot Platform Design, Mini Bone-Attached Robotic System, Mechatronics, LSTAT/Snake Robot, Localization with Mobile Robots, Lego Educational Robotics, Highly-Articulated Robotic Probe, Coverage Path Planning in the Plane: Exact Cellular Decompositions, Controlled Coverage for Auto-Body Painting, Constrained Controlled Coverage, Bridge Inspection with Serpentine Robots, and Autonomous Extra-vehicular Robotic Camera
Research Interests: space robotics, snake robots, multirobot systems, motion planning, mobile robots, mechatronics, mechanisms, inspection, field robotics, factory and warehouse automation, education, control, and artificial intelligence
My research interests include:
Computer Vision
Pattern Recognition
Visual Tracking
Machine Learning
Statistical Signal Processing
Labs: Human Identification at a Distance and Face Group
Projects: Spatio-Temporal Facial Expression Segmentation, Face Recognition, Multimodal Diaries, Deception Detection, Facial Expression Analysis, Machine learning approaches to invert the Radiative Transfer Equation, Reflective Agents with Distributed Adaptive Reasoning, Learning Kernel Expansions for Image Classification, Indoor People Localization, Hot Flash Detection, Forecasting the Anterior Cruciate Ligament Rupture Patterns, Face Model Building and Fitting, Component Analysis for Data Analysis, Camera Assisted Meeting Event Observer, Bird Classification, Automatic Segmentation of Proteomic Images, Quality of Life Technology Center, Intelligent Diabetes Assistant, Depression Assessment, Facial Feature Detection, Multimodal Data Collection, and Temporal Segmentation of Human Motion
Research Interests: artificial intelligence, computer vision, data mining, data visualization, gesture recognition, image compression, image processing, information fusion, machine learning, machine understanding of video and human
behavior, neural networks, pattern recognition, quality-of-life technology, sensor fusion, and statistics
I am a practicing orthopaedic surgeon:
Clinic office:
Renaissance Orthopaedics, PC
Mellon Pavilion, Suite 252
4815 Liberty Avenue
Pittsburgh, PA 15224
vox: (412) 683 7272
fax: (412) 683 0341
I have developed a series of educational conferences to train practicing orthopaedic surgeons. One example is the MIS meets CAOS Symposium Series and is the first ever conference that merges the concepts of less and minimally invasive surgery and computer assisted orthopaedic surgery. This conference series was specifically developed to educate surgeons on these new techniques and technologies and focuses on the development of less and minimally invasive surgery for partial and total joint replacement. I have also recently developed a new Patient and Family Centered Care (PFCC) conference series which brings together all constituencies focusing on technology and re-engineering of processes coupled with unique training and educational programs in order to develop innovative solutions to support PFCC. The goal of this conference series is to ultimately improve every phase of healthcare delivery with the focus on the patients' experience. In addition, I am conducting Arthritis Fairs geared towards patient education. The goal of the Arthritis Fair is to provide participants with information that will improve communication and knowledge so that physicians and patients can make informed choices regarding conservative and operative treatments of arthritis.
Labs: Medical Robotics and Computer Assisted Surgery and Medical Instrumentation Lab
Projects: XAlign, Mini Bone-Attached Robotic System, Knee Navigation Systems (KneeNav TKR/ACL), Joint Replacement Biomechanics, Hip Range of Motion, Hip Navigation System, High Performance Computing for Robot-Assisted Surgery, and 3D Image Overlay
Research Interests: medical applications
My principal research objective is to define and advance the science of technology for developing communities (TFDC); that is, technology relevant to communities where monetary resources are scarce, and the accessible infrastructure and indigenous skills are very different from the norms prevalent in the technologically developed world. My goal is to build technology that empowers these underserved communities in a manner that is culturally relevant and locally sustainable. To this end, I founded TechBridgeWorld (www.techbridgeworld.org) at Carnegie Mellon University to provide the necessary infrastructure for collaborative work between the university and underserved communities around the world. TechBridgeWorld extends the benefits of technology to developing communities, thus promoting a novel field of research that uniquely enhances the world we live in.
A second important research goal is to advance the state of the art in autonomous team coordination. Much of my work to date with team coordination deals with market-based systems where team members conduct and participate in auctions to allocate tasks and resources. My dissertation work laid the foundation for the TraderBots coordination framework that is now a licensed tool used by several groups for research and development in team coordination. My goal for autonomous team coordination is to advance the understanding and the science of market-based coordination mechanisms. I am primarily interested in applications of team coordination in uncertain and dynamic conditions, and in enabling robust, intelligent, and effective coordination of limited resources under these conditions using market-based approaches. An important aspect of this work is to understand and enable effective human-robot teams engaged in complex tasks. My work in team coordination is relevant to TFDC applications such as disaster relief and some of this work is evolving to explicitly address needs in disaster response.
Teaching and research at the Carnegie Mellon University, Qatar campus
Founding member and graduate faculty advisor for women@SCS
Labs: TechBridgeWorld and rCommerce
Projects: Sun Synchronous Navigation, Federation of Intelligent Robotic Explorers Project, Cognitive Colonies, and Treasure Hunt: Pickup Teams
Research Interests: technology for developing communities, space robotics, planning, multi-agent systems, motion planning, mobile robots, human-computer interaction, hazardous environments, field robotics, education, and artificial intelligence
My research goal is to create systems and methodologies that allow groups of robots and humans to collaborate with one another to perform useful tasks. My research is motivated by an interest in the modeling and control of physical systems on the one hand, and in the creation of effective human-machine interfaces on the other. Given continuing technological advances in computing, sensing, actuation, and miniaturization, I believe that the nexus of these two interests is an increasingly exciting area to explore.
The pursuit of this research involves two complementary thrusts. On the one hand, robot systems should be endowed with maximal autonomy. Perception, actuation, planning, and even tasking should be performed with the smallest amount of human intervention possible. Human attention and intelligence are then freed for supervisory and remedial actions that transcend the current competence of the robot or group of robots being controlled. On the other hand, human users should have increasingly effective means of tasking, controlling, and communicating with robots. Human control of robots then becomes less burdensome and more intuitive, and users are able to oversee more complex tasks with greater numbers of robots.
Telesupervisory human-robot systems. I want to build systems that allow maximally intuitive human input in controlling large numbers of machines with variable autonomy. The focus of my current NASA project ?Wide-Area Prospecting Using Supervised Autonomous Robots? is to create a telesupervision system that allows a single astronaut in a ?shirtsleeve? environment to control multiple rovers performing a prospecting task, increasing astronaut safety and productivity. This proposal presents an exciting opportunity to have a large impact in current NASA plans to search for water ice and important minerals on the moon, as outlined in the NASA Lunar Exploration Plan. At the same time, it provides a chance to build a telesupervision architecture applicable to a wide variety of tasks, to include exploration, space assembly, inspection, and maintenance. One of the project?s goals is to lay the groundwork for reversing the current rover control situation, in which multiple rover drivers and planners are dedicated to one rover. We will seek to make advances in rover hazard and assistance detection and high-fidelity telepresence and teleoperation, and will make a direct performance comparison to a single astronaut operating on a planetary surface.
Multi-modal human control. As robots enter the human environment and come in contact with inexperienced users, they need to be able to interact with users in an intuitive fashion - keyboard and mouse are no longer acceptable as the only input modalities. Humans should be able to communicate with robots using methods as similar as possible to the concise, rich, and diverse means they use to communicate with one another. Cooperation among humans is multi-modal and often intuitive, whereas current methods of cooperation among robots and between robots and humans, whether for programming or control, are generally highly specified and inflexible. Interesting research issues in this area include the selection, development, combination, and interpretation of appropriate input modalites (gesture, voice, touch, etc.) and the creation of usable, intuitive multi-modal programming and control systems.
Robot reliability. Mobile robots are typically unreliable. NASA has expressed interest in using modular self-repairable robotic teams for the exploration and colonization of Mars. The NASA Technology Plan 2001 states the need for ?vehicle systems technology developments that are extremely reliable, modular, offer long service life, are self-diagnosing, self-reconfiguring, self-maintaining, and self-repairing.? The use of modular, self-repairing robot teams adds new complexity to the mission design process for robotic exploration. Decisions must be made about how to divide tasks among multiple robots and how to configure the robots and teams to accomplish both individual tasks and overall mission goals. In order to do this, one needs methods to quantify the impact of mission design decisions on mission success. I am interested in using systems engineering reliability principles to answer questions like: "What is the lowest-cost configuration of robots that will accomplish a given set of mission tasks with a given probability of success?" This analysis allows comparison of teams of repairable vs. non-repairable robots, teams where the robots use components with different reliabilities, and teams with different numbers of robots and different numbers of spare parts.
Labs: Advanced Mechatronics Lab and Tele-Supervised Autonomous Robotics
Projects: Urban Challenge, Transformer Winding Automation, Tartan Racing, Self-Mobile Space Manipulator, Real and Virtual Environment for Multiple Robots, Wide Area Prospecting Using Supervised Autonomous Robots, Interactive Robot Programming, Distributed Surveillance & Sensing, Distributed Robotics Systems - CyberScout, CyberATV, Reliability of Mobile Robot Teams, Telesupervised Adaptive Ocean Sensor Fleet, Autonomous Helicopter, and Asynchronous Teams of Autonomous Agents
Research Interests: control, mechatronics, teleoperation, human-computer interaction, space robotics, mobile robots, and factory and warehouse automation
I am interested in autonomous systems that work, are useful and make economic sense, and in finding ways to effectively build and deploy them.
Labs: Auton Lab
Research Interests: artificial intelligence, data visualization, and machine learning
My research is in the area of computer vision and computer graphics, especially at the intersection of the two. I am particularly interested in using data-driven techniques to tackle problems which are very hard to model parametrically but where large quantities of data are readily available. The ultimate goal is to use the ever-growing amount of stored visual information (digital photo albums, webcams, movies, etc.) to learn, understand, and resynthesize the visual world around us.
In very broad strokes, here are the main current themes of my research:
Labs: Computer Graphics Lab
Projects: Geometrically Coherent Image Interpretation
Research Interests: object recognition, machine learning, image processing, computer vision, and computer graphics
I am interested in making robots act purposefully and successfully in a world in which most everything is uncertain. Sensors are noisy, actions are imprecise, and objects are often in the wrong location. Despite such obstacles to purposeful action, there are many tasks that can be accomplished successfully. Humans, animals, and some machines are proof. Providing robots with the ability to operate autonomously and purposefully requires an understanding of how different tasks may be accomplished by different repertoires of actions. Grasping, hitting, and dropping are some actions that are useful in a robot's repertoire. More exotic actions include shaking, twirling, and other actions that randomize an object's state. Recently I constructed a "two-palm robot". The robot consisted of two manipulator arms cooperating to manipulate objects without the need for full kinematic constraint. The arms "programmed themselves", that is, they invoked an automatic planner to find sequences of motions for reorienting objects in their palms. The planner built a geometric graph based on a critical event analysis of the underlying mechanics.
My work is motivated by several desires. First, I would like to program robots more easily than is currently possible. Second, I would like to understand the scope and limitations of autonomous systems, whether biological or artificial. Third, I would like to reduce the complexity of design and planning by codifying the design parameters required to achieve a given level of automation. An underlying goal of my research is to understand the relationship between sensing, action, and prediction. In the past, I have explored various extreme points in this space. With Matt Mason I explored sensorless strategies, for my thesis work I looked at randomized strategies, and most recently I investigated fast-action minimal-sensing strategies. My research draws on tools from geometry, mechanics, planning, and stochastic processes.
I am interested in sensing strategies that acquire object shape and configuration concurrently during manipulation, a research direction pioneered by my former students Yan-Bin Jia and Mark Moll. Currently, Matt Mason, Sidd Srinivasa, and I are working on a related project to develop a theory of task-level kinesthetic perception.
I am also interested in protein homology, in particular determining structural homology from sparse NMR data. I am exploring techniques from knot theory to model protein structures.
Labs: Manipulation Lab
Projects: Desktop Robotics
Research Interests: shape sensing, protein structure, protein knot theory, motion planning, mobile manipulation, and manipulation mechanics
As information systems have evolved from isolated computational engines to distributed networks, the autonomous ability to gather and act on information is becoming increasingly important. My research is in the interdisciplinary area of MicroElectroMechanical Systems (MEMS): sensor and actuator systems with performance derived from integration of electronics and mechanical structures with features measured in microns to millimeters. Fabrication of the batch-fabricated electromechanical devices and the development of related processes leverage the enormous investment in mature Very-Large-Scale Integrated (VLSI) circuit manufacturing. Benefits of this approach include much lower manufacturing cost, greater miniaturization, greater integration, and in many cases higher performance than can be achieved with conventional methods used to build systems requiring sensors and actuators. My research focus on integrated MEMS will eventually lead to the manufacture of low-cost sensor-and-actuator Application-Specific Integrated Circuits (ASICs). Integrated MEMS technology will be pervasive in future embedded systems.
Our research group designs, fabricates, and tests microdevices that are primarily made using a process in high conventional foundry CMOS is followed by simple micromachining steps. This process provides us with high-performance electronics integrated on chip with electrostatically actuated microstructures, capacitive and piezoresistive sensors, and polysilicon thermal heaters. Projects include micromechanisms for magnetic probe-based data storage, accelerometers and gyroscopes for inertial sensing, and ciliary sensors for tactile and acoustic imaging. Of particular interest is how large arrays of these sensors and actuators may improve overall system-level performance. Issues include system design and integration, distributed control and communication, and interfacing to the environment.
MEMS are coupled multi-domain systems and, therefore, are difficult to design without expertise in a diverse set of fields. To address this problem in our lab, MEMS designers and CAD developers work closely together in a synergetic research environment. We are developing a multi-domain hierarchical design methodology to speed up the design cycle. A MEMS schematic is being developed in which mechanical, electromechanical, and electronic elements are graphically interconnected, resulting in rapid simulation and evaluation of designs. We are also modeling topologies for common MEMS applications, such as accelerometry, to codify design constraints for use in automated synthesis tools.
Labs: Microelectromechanical Systems Laboratory
Projects: Ultra-High-Density Data Cache for Low-Power Communications, Tactile Displays Using MEMS Actuator Arrays, Silicon Micro-Disk Arrays for Data Storage, Schematic Design for MEMS, Resonator Synthesis, Integrated MEMS Inertial Measurement Unit, Integrated MEMS for Space Applications, High-Aspect-Ratio CMOS Micromachining Process, Foundation for MEMS Synthesis, Circuit Extraction from MEMS Layout, Biologically Inspired Micro Robotics, and Application Specific Integrated MEMS Process Service
Research Interests: VLSI, sensors, microrobotics, MEMS, and mechatronics
As information systems increasingly leave fixed locations and appear in our pockets and palms, they are getting closer to the physical world, creating new opportunities for perceiving and controlling our machines, structures and environments. To exploit these opportunities, information systems will need to sense and act as well as compute. Investing engineered systems with the ability to sense and act is the focus for my research activities in microelectromechanical systems (MEMS).
Building on and using techniques in signal processing, modeling, robotics, and micromachined device and fabrication, we are developing MEMS components and arrays with initial applications in biomedical and analytical instruments, human-machine interfaces, and optical/radio-frequency switching and signal processing. MEMS components and arrays will enable highly functional and reliable analytical instruments with new assays and protocols, new drug delivery systems and new kinds of prosthetic devices. Electromechanical switching, filtering and signal processing of optical and radio frequency signals are emerging MEMS application areas showing promise as a means of achieving superior performance with size, cost and power consumption orders of magnitude smaller than those of conventional methods. Richer human-machine interfaces will be possible through the development of displays and sensors that tap into the full sensory capabilities and modalities of humans. Specifically, MEMS makes possible the construction of affordable, high-resolution tactile and haptic stimulators and displays to complement widely-available and well-developed auditory and visual interfaces, creating new dimensions not yet possible to exploit in human-machine interfaces and virtual reality systems.
The future and real promise of MEMS will be in our ability to design systems of components with thousands to millions of electromechanical parts integrated with electronics to create MEMS arrays with a systems function greater than the sum of the individual parts. This next stage in the evolution and maturity of MEMS will be driven less by captive fabrication facilities and process development and more by innovative, aggressive electromechanical systems design. MEMS is poised to take full advantage of advances in information technology and couple them to advances in robotics and control theory to drive a fundamentally new approach to electromechanical system design and fabrication. For the first time, approaches akin to VLSI electronics can be taken to usher in an equally exciting and productive era of VLSI electromechanics. By merging sensing and actuation with computation, MEMS will not only invest existing systems with enhanced capabilities and reliability, but will make possible radically new devices and systems designs that exploit the miniaturization, multiplicity and microelectronics of MEMS.
Labs: Tissue Engineering and Microelectromechanical Systems Laboratory
Projects: Application Specific Integrated MEMS Process Service, Silicon/Neuron Interface, and Biologically Inspired Micro Robotics
Research Interests: tissue engineering, microrobotics, MEMS, human-computer interaction, haptics, display devices, and bioengineering
3-D Recognition and Model Construction
My group is investigating a number of algorithms and applications in the area of 3-D data manipulation and interpretation. This includes developing techniques for the automatic construction of three-dimensional models from large sets of views with applications to object model generation, interior mapping, and terrain map registration; and the recognition of 3-D objects in complex scenes from large databases of models.
Object Recognition in Images
We are also developing algorithms for recognizing objects in images. The approaches include nearest neighbor techniques for recognition of objects in general position in images, edge-based decision tree techniques specialized for recognizing wiry objects, and classification techniques for the identification of large classes of objects and for image segmentation in the context of natural scenes. All the approaches are based on learning classifiers from training exemplars; we are also exploring techniques for weakly-labeled training in order to reduce training overhead.
Mobile Robots
Labs: Vision and Mobile Robotics Lab, NavLab, and 3D Computer Vision Group
Projects: Video Verification of Identity, Urban Challenge, Unmanned Ground Vehicles, Terrain Classification, Tartan Racing, Tactical Mobile Robotics, Sonar Mapping for Underwater Vehicles, Robotic All Terrain Lunar Exploration Rover, Position Estimation, Perception for Rock Sampling, Model Building, Medical Imaging, MARS2020, Lunar Rover Initiative, Humanoid Vision, High Speed Laser Scanner, Exploitation of 3-D Data, CTA Robotics, Cognitive Colonies, Autonomous Navigation System, Automatic 3D Modeling from Range Images, Ambler, Advanced Sensor Based Defect Management at Construction Sites, A Spherical Representation for Recognition of 3-D Curved Objects, A Reactive System for Off-Road Autonomous Driving, 3D Vision for Autonomous Navigation, 3D Terrain Mapping, Geometrically Coherent Image Interpretation, 3D Object Recognition, 2D Recognition, Quality of Life Technology Center, and Event Detection in Videos
Research Interests: mobile robots and computer vision
My research focuses on the coordination and control of dynamic physical systems, both natural and human-made and explores techniques that may someday allow robots and animated creatures to plan and control their actions in complex and unpredictable environments while interacting with humans. To make progress toward this goal, I have active projects in computer graphics, human-robot interaction and humanoid robotics.
My current computer graphics research focuses on generating human motion for computer animation. We have explored a number of techniques: using control algorithms in combination with physically realistic simulation, re-sequencing and interpolating motion capture clips, capturing skin and muscle deformations, and most recently data-driven simulations. In human-robot interaction, I am interested in determining which elements of human motion and behavior must be mimicked to facilitate natural interactions. In humanoid robotics, we are exploring control techniques for our Sarcos humanoid robot with the goal of producing natural looking motion.
Labs: Human-Robot Interaction Group and Computer Graphics Lab
Research Interests: quality-of-life technology, legged locomotion, human motion simulation, entertainment robotics, computer graphics, and animation
My current research effort focuses on three main areas, all involving the creation of innovative new hardware, software, and systems. The first area concerns distributed agent-based cooperative high-precision manipulation. The principal application is agile assembly of small high-precision electromechanical products such as computer storage devices, medical devices, communication devices, and other high-density mechatronic equipment. The goal is to revolutionize the assembly of these kinds of products by drastically reducing the time it takes to design, program, and deploy automated assembly systems, while increasing their precision by several orders of magnitude and reducing their physical size. The second area concerns human-computer interaction, especially through haptic interaction with computed or remote environments. Here a goal is to enable truly transparent and high-fidelity interaction with eventual application to medicine, computer-augmented design, and telemanipulation, including scaled manipulation of microscopic and nanoscopic objects. The third area concerns intelligent mobile robots which are dynamically stable, including both rolling and walking machines. If such robots are to operate successfully in peopled environments, they must be agile and responsive to physical interaction with humans and their surroundings.
In my experience, it is often very effective to synthesize novel robotic technology directly from physical principles, applying good engineering judgment rather than trying to build systems around collections of existing components. For this approach, a broad background including knowledge of physics, electrical and mechanical engineering, computer programming, and design is helpful. It is also extremely valuable to apply newly developed robot technology to real-world problems. Only in this way can one gain insight into the requirements for the technology.
Labs: Microdynamic Systems Laboratory
Projects: Visual-Haptic Interface to Virtual Environment, Vision-Guided Precision Assembly, Teleoperation with a 12-DOF Coarse-Fine Manipulator, Magnetic Levitation Haptic Interfaces, Architecture for Agile Assembly, Robots in Scansorial Environments, Free-Roaming Planar Motors, Psychophysics of Haptic Interaction, Dynamically-Stable Mobile Robots in Human Environments, and Magnetic Levitation Haptic Consortium
Research Interests: acoustics, actuators, architectures, assembly, control, design, factory and warehouse automation, haptics, human-computer interaction, legged locomotion, manipulation, manufacturing, mechanisms, mechatronics, microrobotics, multi-agent systems, sensors, and teleoperation
My area of expertise is three dimensional (3D) computer vision, specifically using high accuracy range sensors such as laser scanners for problems in the areas of modeling, recognition, and visualization. My goal is to make progress toward solving the problem of scene understanding using 3D computer vision. I am particularly interested in methods that combine image-based approaches from traditional computer vision with 3D computer vision algorithms. I am interested in methods to extract high-level semantics from 3D models, such as models of buildings. The ability to reverse engineer buildings has enormous potential benefit in a variety of fields, ranging from robotics to civil engineering to homeland security. I am also interested in methods for processing and visualizing 3D data in real time for high-speed teleoperation telepresence applications. Finally, I am studying the effects of sensor noise and data artifacts on the accuracy of 3D models for precision measurement applications.
Labs: Vision and Mobile Robotics Lab and 3D Computer Vision Group
Projects: Unmanned Ground Vehicles, Terrain Classification, Exploitation of 3-D Data, CTA Robotics, Automatic 3D Modeling from Range Images, Advanced Sensor Based Defect Management at Construction Sites, and 3D Terrain Mapping
Research Interests: range data, mobile robots, computer vision, 3-D perception, teleoperation, and range finders
My primary interests are in computational biomechanics and computational geometry and applications in computer assisted surgery, medical planning, simulation and analysis. Most of my work is focused on applications in orthopaedic surgery, primarily on the adult reconstruction surgery. Our computer assisted surgical system for total hip replacement - HipNav, is currently used in clinical pre-trials in UPMC Shadyside. My current work includes development of new image guided surgical systems and applications, and development of robust methods for patient-specific computational biomechanics models.
Labs: Medical Robotics and Computer Assisted Surgery, Biomedical Robotics Lab, and Medical Instrumentation Lab
Projects: XAlign, Soft Tissue Simulation for Plastic Surgery, Mini Bone-Attached Robotic System, Knee Navigation Systems (KneeNav TKR/ACL), Joint Replacement Biomechanics, Hip Range of Motion, Hip Navigation System, High Performance Computing for Robot-Assisted Surgery, and 3D Image Overlay
Research Interests: medical robotics, medical imaging, medical applications, and bioengineering
My research interests are in the areas of computer vision, visual and multi-media technology, and robotics. Common themes that my students and I emphasize in performing research are the formulation of sound theories which use the physical, geometrical, and semantic properties involved in perceptual and control processes in order to create intelligent machines, and the demonstration of the working systems based on these theories.
My current projects include basic research and system development in computer vision (motion, stereo and object recognition), recognition of facial expressions, virtual(ized) reality, content-based video and image retrieval, VLSI-based computational sensors, medical robotics, and an autonomous helicopter.
Computer vision
Within the Image Understanding (IU) project, my students and I are conducting basic research in interpretation and sensing for computer vision. My major thrust is the "science of computer vision." Traditionally, many computer vision algorithms were derived heuristically either by introspection or biological analogy. In contrast, my approach to vision is to transform the physical, geometrical, optical and statistical processes, which underlie vision, into mathematical and computational models. This approach results in algorithms that are far more powerful and revealing than traditional ad hoc methods based solely on heuristic knowledge. With this approach we have developed a new class of algorithms for color, stereo, motion, and texture.
The two most successful examples of this approach are the factorization method and the multi-baseline stereo method. The factorization method is for the robust recovering of shape and motion from an image sequence. Based on this theory we have been developing a system for "modeling by video taping"; a user takes a video tape of a scene or an object by either moving a camera or moving the object, and then from the video a three-dimensional model of the scene or the object is created. The multi-baseline stereo method, the second example, is a new stereo theory that uses multi-image fusion for creating a dense depth map of a natural scene. Based on this theory, a video-rate stereo machine has been developed, which can produce a 200x200 depth image at 30 frames/sec, aligned with an intensity image; in other words, a real 3D camera!!
Currently, we are working on a rapidly trainable object recognition method, a system for modeling-by-video-taping, and a multi-camera 3D object copying/reconstruction method.
Visual media technology for human-computer interaction
A combination of computer vision and computer graphics technology presents an opportunity for a new exciting visual media. We have been developing a new visual medium, named "virtualized reality." In the existing visual medium, the view of the scene is determined at the transcription time, independent of the viewer. In contrast, the virtualized reality delays the selection of the viewing angle till view time, using techniques from computer vision and computer graphics. The visual event is captured using many cameras that cover the action from all sides. The 3D structure of the event, aligned with the pixels of the image, is computed for a few selected directions using the multi-baseline stereo technique. Triangulation and texture mapping enable the placement of a soft-camera to reconstruct the event from any new viewpoint. The viewer, wearing a stereo-viewing system, can freely move about in the world and observe it from a viewpoint chosen dynamically at view time. We have built a 3D Virtualized Studio using a hemispherical dome, 5 meters in diameter, currently with 51 cameras attached at its nodes.
There are many applications of virtualized reality. Virtualized reality starts with a real world, rather than creating an artificial model of it. So, training can become safer, more real and more effective. A surgery, recorded in a virtualized reality studio, could be revisited by medical students repeatedly, viewing it from positions of their choice. Or, an entirely new generation of entertainment media can be developed - "Let's watch NBA in the court": basketball enthusiasts could watch a game from inside the court, from a referee's point of view, or even from the "ball's eye" point of view.
A Virtualized Reality application, CBS's Eye Vision, was demonstrated during SuperBowl XXXV.
Also, I am interested in and currently working on vision techniques for recognizing facial expression, gaze, and hand-finger gestures. Such techniques will provide natural non-intrusive means for human-computer interface by replacing current clumsy mechanical devices, such as datagloves.
Informedia Project
With the growth and popularity of multimedia computing technologies, video is gaining importance and broadening its uses in libraries. Digital video libraries open up great potentials for education, training and entertainment; but to achieve this potential, the information embedded within the digital video library must be easy to locate, manage and use. Searches within a large data set or lengthy video would take a user through vast amounts of material irrelevant to the search topic. The typical database, which searches by keywords (e.g. title) - where images are only referenced and not directly searched for - is not appropriate or useful for the digital video library, since it does not provide the user a way to know the contents of the image, short of viewing it. New techniques are needed to organize these vast video collections so that users can effectively retrieve and browse their holdings based on their content. The Informedia Digital Video Library, funded by NSF, ARPA, and NASA, is developing intelligent, automatic mechanisms to populate the video library and allow for a full-content knowledge-based search, retrieval and presentation of video. The distinguishing feature of Informedia's approach is the integrated application of speech, language and image understanding technologies.
Computational Sensor
While significant advancements have been made over the last 30 years of computer vision research, the consistent paradigm has been that a "camera" sees the world and a computer "algorithm" recognizes the object. I have been undertaking a project with Dr. Vladimir Brajovic that breaks away from this traditional paradigm by integrating sensing and processing into a single VLSI chip a computational sensor. The first successful example was an ultra fast range sensor which can produce approximately 1000 frames of range images per second an improvement of two orders of magnitude over the state of the art. A few new sensors are being developed including a sorting sensor chip, a 2D salient feature detector (2D winner-take-all circuits), and others.
Medical Robotics and Computer Assisted Surgery
The emerging field of Medical Robotics and Computer Assisted Surgery strives to develop smart tools to perform medical procedures better than either a physician or machine could alone. Robotic and computer-based systems are now being applied in specialties that range from neurosurgery and laparoscopy to opthalmology and family practice. Robots are able to perform precise and repeatable tasks that would be impossible for any human. The physician provides these systems with the decision making skills and adaptable dexterity that are well beyond current technology. The potential combination of robots and physicians has created a new worldwide interest in the area of medical robotics.
We have developed a new computer assisted surgical systems for total hip replacement. The work is based on biomechanics-based surgical simulations and less invasive and more accurate vision-based techniques for determining the position of the patient anatomy during a robot surgery. The developed system, HipNav, has been already test -used in clinical setting.
Vision-based Autonomous Helicopter
An unmanned helicopter can take maximum advantage of the high maneuverability of helicopters in dangerous support tasks, such as search and rescue, and fire fighting, since it does not place a human pilot in danger. The CMU Vision-Guided Helicopter Project (with Dr. Omead Amidi) has been developing the basic technologies for an unmanned autonomous helicopter including robust control methods, vision algorithms for real-time object detection and tracking, integration of GPS, motion sensors, vision output for robust positioning, and high-speed real-time hardware. After having tested various control algorithms and real-time vision algorithms using an electric helicopter on an indoor teststand, we have developed a computer controlled helicopter (4 m long), which carries two CCD cameras, GPS, gyros and accelerometers together with a multiprocessor computing system. Autonomous outdoor free flight has been demonstrated with such capabilities as following prescribed trajectory, detecting an object, and tracking or picking it from the air.
Labs: Vision for Virtual Environments, Vision for Safe Driving, Virtualized RealityTM, Video Surveillance and Monitoring, People Image Analysis Consortium, Medical Robotics and Computer Assisted Surgery, Medical Instrumentation Lab, Human Identification at a Distance, Helicopter Lab, Face Group, Computational Sensor Laboratory, and Biomedical Image Analysis
Projects: Z-Keying, Visual-Haptic Interface to Virtual Environment, Virtualized RealityTM, Video-rate Stereo Machine, Video Surveillance and Monitoring, Ultrasonic Bone Imaging, Textureless Layers, Temporal Shape-From-Silhouette, Super-Resolution Optical Flow, Spatio-Temporal View Interpolation, Spatio-Temporal Facial Expression Segmentation, Soft Tissue Simulation for Plastic Surgery, Setting Low-Level Vision Parameters, Self-Mobile Space Manipulator, Scene Flow, Reconfigurable Vision Machine, Real-time Face Detection, Quality of Life Technology Center, Prediction & Planning, Precision Freehand Sculpting, Photometric Limits on Computer Vision, Perception for Humanoid Robots, Object Recognition Using Statistical Modeling, Non-Invasive Optical Imaging in vivo for Early Detection and Advanced Diagnosis of Cancer, Neural Network-Based Face Detection, Multi-view Car Detection and Registration, Multi-People Tracking, Modeling by Videotape, Metaphor, Medical Image Registration, Magic Eye, Light-fields, Knowledge-Guided Deformable Registration, Knee Surgery Simulation, Integrated Vision and Sensing for Human Sensory Augmentation, Informedia Digital Video Library, Image Enhancement for Faces, Humanoid Vision, Human Motion Transfer, Human Kinematic Modeling and Motion Capture, High Performance Computing for Robot-Assisted Surgery, Hand Tracking and 3-D Pose Estimation, Hallucinating Faces, GPU-accelerated Computer Vision, Gaze Estimation, Frontal Face Alignment, Feature-based 3D Head Tracking, Fast VLSI Range-Image Sensor, Factorization Method, Facial Expression Analysis, Face Video Hallucination, Face Recognition Across Illumination, Face Recognition, Face Model Building and Fitting, Face Detection Databases, Face Detection, Face Databases, Face and Facial Feature Tracking, EyeVision, Dynamic Conformal Radiotherapy, DigitEyes, Deception Detection, Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects, Cooperative Stereo Vision, Computer Assisted Medical Instrument Navigation, Component Analysis for Data Analysis, Cohn-Kanade AU-Coded Facial Expression Database, Cell Tracking, Car Tracking, Camera Assisted Meeting Event Observer, Autonomous Land Vehicle In a Neural Network, Autonomous Helicopter, Ambler, ALVINN-On-A-Chip, Accurate Camera Calibration from Planar Patterns, A Statistical Quantification of Human Brain Asymmetry, 3D Video Reconstruction of Skeletal Anatomy, 3D Optical Reconstruction of Cell Shape, 3D Image Overlay, 3D Head Motion Recovery in Real Time, and 2D->3D Face Model Construction
Research Interests: stereo vision, quality-of-life technology, mobile robots, medical applications, human-computer interaction, computer vision, and computational sensors
The control of dynamical systems becomes increasingly important as the era of robotics research dominated by quasi-static machines rapidly comes to a close. Similarly, the importance of state estimation grows as robotic applications require robots to function in larger, more complex environments. My research addresses both of these issues by focusing on the dual problems of controlling robotic mechanisms with non-trivial dynamics and perceiving the state of world through indirect measurements. My approach is both analytical and experimental: I use mathematics to understand the physical behavior of a given system and then use that understanding to create algorithms for control or estimation. I strive to develop new theoretical concepts and translate them into real-world implementations that solve problems such as balancing an unstable robot or estimating the location of an autonomous vehicle.
Projects: EnviroBlimp, Dynamically-Stable Mobile Robots in Human Environments, and DEPTHX: Deep Phreatic Thermal Explorer
Research Interests: sensor fusion, range data, field robotics, and control
I am interested in building robots and related systems that are cost-effective in today's marketplace. It is clear that sensing and cognition have a long way to go before an autonomous system can match the ability of even a small child. Yet, it is also clear that autonomous systems have a place in our world now if they can compete with humans because they are better, faster, cheaper, safer or even more entertaining. Some recent areas of interest include:
Off-Road Mobility
The goal of this work is to improve the performance and reliability of vehicles that drive themselves "in the rough" - outdoors, off the road. I am interested in the systems aspects of constructing a high-performance autonomous vehicle and particularly in the perception, planning, and control software. Ideally, I'd like to build a software system that runs on a single processor that automates any vehicle when appropriate control systems are added. In particular, I am now working on fast stereo algorithms that exploit the constraints of most off-road environments in order to generate near frame-rate stereo vision on a standard personal computer platform.
Position Estimation for Structured Environments
It has long been the practice in industrial automation to make up for lack of intelligence with repeatable positioning and/or knowledge of environmental structure. While such teach-playback techniques have become standard for manipulators, the lack of repeatability of mobile robot positioning systems has made it difficult to simply teach a vehicle a trajectory and have it follow that trajectory repeatably enough to function effectively.
I am working on a system to compute repeatable vehicle positions from image mosaics constructed from factory floors. The basic idea is that the markings on the floor over a sufficiently large area are or can be made unique enough to unambiguously locate the vehicle. This approach is superior to competitive landmark based systems and all dead reckoning systems because the repeatability is bounded by the footprint area of a single image pixel - which can be made arbitrarily small. Coincidentally, the image processing involved is a 2D analog of the Global Positioning Satellite navigation system (GPS).
Visual Servoing of Implements on Moving Vehicles
A baseball player who catches a ball while running solves a complex problem of coordinated perception and control. Based on a 2D image and only rough position feedback, the player simultaneously tracks the 3D position of a moving object relative to a moving observer and coordinates many degrees of freedom of manipulation to execute a capture trajectory.
I am working on a vision and control system to enable a fork truck to autonomously unload trailers and rail cars in the auto industry. The challenge is to robustly identify the containers to be loaded, compute their 3D positions, and servo both the vehicle and forks to acquire one load at a time - all while moving at high speed.
Projects: Vehicle Stability Prediction, Unmanned Ground Vehicles, Transitional Unmanned Ground Vehicle, Ranger, PerceptOR (NREC), Lunar Rover Initiative, Very Rough Terrain Nonholonomic Trajectory Generation and Motion Planning for Rovers, Autonomous Navigation System, Autonomous Loading System, Tartan Racing, Automated Material Transport System, and Urban Challenge
Research Interests: motion planning, mobile robots, field robotics, factory and warehouse automation, control, and computer vision
My research interests are in the areas of sensor-based manipulator control, real-time architectures for control, design for assembly, methodologies for manipulator design, and applications of robotics in assembly and manufacturing. The goal of my research is twofold: to develop the basis for incorporating multiple sensing modalities in the dynamic loop of a manipulator, and to apply such a system in automatic assembly and manufacturing applications. My research thus involves both theory and experimental implementation in a laboratory.
CMU Direct-Drive Arm II Testbed: My research on sensor-based control revolves around this project. Strategies for using position, velocity, force/torque, vision, proximity, and tactile sensors for both controlling a manipulator and for interacting intelligently with the environment are being addressed in this work. One project addresses the use of joint position, velocity and end-effector force/torque sensing for obstacle avoidance and force control. In another project techniques are being developed for using tactile data for dynamic object exploration. The use of a camera as a sensor in the dynamic feedback loop is being studied in a project on dynamic visual servoing. The goal here is to bridge the gap between traditional vision research and control theory. The CMU Direct-Drive Arm II testbed is equipped with the above mentioned sensors and one of the projects aims at developing a real-time kernel, called CHIMERA, and a hierarchical controller structure for incorporating these multiple sensors, in the control loop, in an unified manner.
CMU Reconfigurable Modular Manipulator System (RMMS): My research on methodologies for manipulator design revolve around the RMMS. In this project, the goal is to address theoretical issues in mapping kinematic and dynamic task requirements into kinematic and dynamic configurations of a manipulator that is configured from a set of modular joints and links. Projects that address the automatic generation of kinematic and dynamic equations, reconfigurable controllers, dynamic control of redundant manipulators are being pursued in this context.
Reconfigurable Control Software and Programming Interface: In this project, we are interested in developing techniques for reconfiguring control software in real-time. We are also developing an icon based programming interface for rapid development of applications.
Rapid Assembly System: In this project our goal is to develop an integrated design-manufacturing environment. We are developing reasoning, planning, and representational methodologies for the assembly and the facility. Our initial work has shown that it is possible to accept a 3-D solid modeler description of an assembly as input, and automatically generate and execute real-time code to create the physical assembly.
Combined Mobility and Manipulation: In this research, we are interested in developing algorithms to utilize the redundancy provided by combining mobile platforms with manipulators. We are also developing techniques for multiple task execution.
Labs: Advanced Mechatronics Lab and Medical Instrumentation Lab
Projects: Transformer Winding Automation, Robotic Neurosurgery Probe Guide, Reconfigurable Software Design for Robotic and Automation Applications, Reconfigurable Modular Manipulator System, Real and Virtual Environment for Multiple Robots, Onika, Millibots, Metaphor, Interactive Robot Programming, Intelligent Assembly Modelling and Simulation, I-Cubes, High Bandwidth Visual Feedback for Robust Manipulation, Gesture Based Programming, Distributed Surveillance & Sensing, Distributed Robotics Systems - CyberScout, Distributed Design System - CODES, Dexterous Haptic Interface for Interaction with Remote/Virtual Environments, CyberRAVE, CyberATV, Composable Simulation, Collaborative Agents, Chimera, BORG, Biologically Inspired Micro Robotics, Amaranth, Micron: Intelligent Microsurgical Instruments, and Agent Based Design
I am interested in developing algorithms and software for simulating and synthesizing motion for complex kinematic and dynamic systems. This research involves interdisciplinary work in robotics, computer graphics, and computational geometry.
Motion Planning:
Fundamental to motion synthesis research is the development of efficient search techniques. Solving a motion synthesis problem involves constructing a suitable model and searching an appropriate space of possibilities. I am interested in developing efficient motion planning algorithms for searching high-dimensional configuration spaces, with applications ranging from path planning for autonomous robots, humanoids and animated characters, CAD assembly analysis (part removability and maintenance), and computer-aided drug design.
Humanoid Robotics:
For the past several years, I have been building general software components for autonomous humanoids based on planning, sensing, and control. This has concurrently involved researching techniques for automatically generating gross body motions for complex simulated models of human figures given high-level navigation or manipulation task commands, as well as generating motion trajectories for real humanoid robot hardware. Specifically, I have focused on path planning for obstacle avoidance, balance control, self-collision detection, footstep placement, and integrated sensor feedback systems.
Computer Animation:
I am researching methods to automatically generate motion for animated characters. The goal is to create software that will enable an autonomous virtual human to move naturally in response to task-level commands such as "walk over to the table and pick up the book''. The underlying software automatically generates the motion necessary to perform the given task. I am also interested in developing general techniques for efficient modeling, rendering, and animation of complex geometric models such as articulated characters.
Labs: Vision for Safe Driving, Computer Graphics Lab, and Planning and Autonomy Lab
Projects: Precomputed Search Trees: Planning for Interactive Goal-Driven Animation, Perception for Humanoid Robots, Navigation Among Movable Obstacles, GPU-accelerated Computer Vision, Footstep Planning for Biped Robots, Behavior Planning for Character Animation, Quality of Life Technology Center, and Learning Locomotion
Research Interests: planning, obstacle avoidance, motion planning, legged locomotion, humanoid robotics, human motion simulation, graphics, artificial intelligence, and animation
My research interests span a range of applications in computer vision and robotics, with a central research theme: computational symmetry. Computational symmetry addresses issues on robust representation, detection, and reasoning about symmetry, as well as diverse applications of applying (a)symmetry analysis on computers (see projects).
Symmetry is a pervasive phenomena in both natural and man-made environments. Humans have an innate ability to perceive and take advantage of symmetry in everyday life, but it is not obvious how to automate this powerful insight on man-made intelligent beings, such as robots. On the surface, symmetry is simple and basic. In essence, the concept of symmetry is much more than a mirror reflection with binary choices, rather, it can span a continuous spectrum of multi-dimensional spaces.
In basic sciences, the understanding of symmetry played a profound role in several important discoveries including: relativity theory (the symmetry of time and space); human DNA structure (double helix); the quasicrystals and their mathematical counterpart penrose tiles. We argue that reasoning about symmetry can likewise play a crucial part in the advance of artificial/machine intelligence.
A computational model for symmetry is especially pertinent to robotics, computer vision and machine intelligence, because in these fields we are studying how a man-made intelligent being can perceive and interact with the chaotic real world in the most effective way. Recognition of symmetries is the first step towards capturing the essential structure of a real world problem, and minimizing redundancy which can often lead to drastic reductions in computation. One fundamental limitation of computers is their finite representational power. One simple floating point error can destroy any perfect symmetry. One's ability to tolerate departure from perfect symmetry reflects one's level of sophistication in perception, which need to be built into the development of machine/artificial intelligence. Besides our poor understanding of human?s natural capability of symmetry perception, the mathematical theory for symmetry, group theory, has not been utilized effectively in practice. Group theory is usually introduced in classrooms in an abstract way (if it is introduced to computer science majors at all in the United States) that is hard to relate to everyday life. More importantly, the non-coherent topological nature of symmetry groups poses challenging computation problems on computers. I am finishing up a textbook for engineering students to learn group theory through concrete examples from applications in robotics (assembly planning) and computer vision (repeated pattern perception).
My current projects related to computational symmetry include:
1) Brain Asymmetry
how symmetrical are the normal (human, mice, . . . ) brains?
2) Facial Asymmetry
Using facial asymmetry as a biometric to identify faces under expression, post and lighting variations. The questions we are seeking the answers for are:
3) Repeated Pattern Perception using Crystallographic Groups
What do you see when you look at a regularly textured surface? do you see tiles? or do you see structures? We are developing a computational model for repeated pattern perception that is able to automatically classify a given pattern into one of the 7 frieze groups (patterns repeating along one direction), or one of the 17 wallpaper groups (patterns repeating along two linearly independent directions), or one of the 230 space groups (patterns repeating in 3D Euclidean space). It can also automatically generate a finite set of possible tiles (based on our theoretical proofs). Furthermore, we study repeated patterns under different viewing directions to find out what happens to a periodic pattern when it is deformed by Affine or perspective transformations?
4) Gait analysis using wallpaper groups
Spatiotemporal representation of human or animal gaits form a naturally appreciable periodic pattern. Different gaits are reflected by different symmetries and symmetry groups of such patterns. We study the possibility of using cues extracted from such patterns for identity and activity classification.
In addition to computational symmetry, I am interested in discovering hidden patterns from large image sets, in particular, large biomedical image databases. These images are especially attractive for studying image meanings since they are normally associated with unambiguous, objective underlying semantics from physical causes (versus images that can be interpreted one way or the other subjectively by the viewer). With the worldwide trend towards ?paperless? hospitals, the commercially available Picture Archiving and Communication System (PACS) installed in many hospitals collects a large amount of digital biomedical image data monthly, weekly even daily. However, the utilization of such data for research and education is hampered by the lack of intelligent, effective image analysis, comparison and retrieval tools.
My research focus is to learn semantically discriminative image features using statistical learning theory, information theory, and pattern recognition, image processing and computer vision tools. The goal is to seek the true fundamental dimensionality and separability of a given image set and image features. The philosophy of our approach is "un-biased least commitment", and it is executed as follows:
We close the learning loop from imaging process --> image feature extraction --> image feature screening --> image feature grouping --> image feature subset selection --> image classification and image retrieval. We have applied these ideas in multiple application domains (pathological neuroimages, facial expression videos, multispectral microscopic images) with very promising results (see our publications). We have several on-going projects exploring along these research directions intensively (see our projects). The results from this research are directly applicable to the fast growing biomedical informatics industry and hospitals, with which we have and we continue establishing tight collaborations.
Elected Robotics Institute Faculty Senator, Carnegie Mellon University (2000-2004)
Elected member of the executive committee of CMU Faculty Senator, Carnegie Mellon University (2003-2004)
Labs: Medical Robotics and Computer Assisted Surgery, Human Identification at a Distance, Face Group, Computer Graphics Lab, Computational Symmetry, and Biomedical