Visual Tracking of a Moving Target by a Camera Mounted on a Robot: A Combination of Control and Vision - Robotics Institute Carnegie Mellon University

Visual Tracking of a Moving Target by a Camera Mounted on a Robot: A Combination of Control and Vision

N. Papanikolopoulos, Pradeep Khosla, and Takeo Kanade
Journal Article, IEEE Transactions on Robotics and Automation, Vol. 9, No. 1, pp. 14 - 35, February, 1993

Abstract

The authors present algorithms for robotic (eye-in-hand configuration) real-time visual tracking of arbitrary 3D objects traveling at unknown velocities in a 2D space (depth is given as known). Visual tracking is formulated as a problem of combining control with computer vision. A mathematical formulation of the control problem that includes information from a novel feedback vision sensor and represents everything with respect to the camera frame is presented. The sum-of-squared differences (SSD) optical flow is used to compute the vector of discrete displacements each instant of time. These displacements can be fed either directly to a PI (proportional-integral) controller or to a pole assignment controller or discrete steady-state Kalman filter. In the latter case, the Kalman filter calculates the estimated values of the system's states and the exogenous disturbances, and a discrete LQG (linear-quadratic Gaussian) controller computes the desired motion of the robotic system. The outputs of the controllers are sent to the Cartesian robotic controller. Performance results are presented.

BibTeX

@article{Papanikolopoulos-1993-13456,
author = {N. Papanikolopoulos and Pradeep Khosla and Takeo Kanade},
title = {Visual Tracking of a Moving Target by a Camera Mounted on a Robot: A Combination of Control and Vision},
journal = {IEEE Transactions on Robotics and Automation},
year = {1993},
month = {February},
volume = {9},
number = {1},
pages = {14 - 35},
}