SVM Decision Boundary Based Discriminative Subspace Induction

Jiayong Zhang and Yanxi Liu
tech. report CMU-RI-TR-02-15, Robotics Institute, Carnegie Mellon University, June, 2002


Download
  • Adobe portable document format (pdf) (548KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Dimensionality reduction is widely accepted as an analysis and modeling tool to deal with high-dimensional spaces, although researches from different disciplines have different interpretations of what properties should be preserved in the reduction process. We study the problem of linear dimension reduction for classification, with a focus on sufficient dimension reduction, i.e., inducing subspaces without loss of discriminative information. Decision boundary analysis (DBA), originally proposed by Lee & Landgrebe (1993), can directly find the smallest subspace with such property. However, existing DBA implementations are computationally expensive and sensitive to sample size. In this paper, we first formulate the problem of sufficient dimension reduction for classification in parallel terms as for regression. Disclosures of these connections lead to several meaningful observations. Then we present a novel space reduction algorithm that combines SVM and DBA, thus inheriting several appealing properties from kernel machines such as good generalization, weak assumption, and efficient computation. In addition, the proposed method provides a natural way to reduce the complexity, and even improve the accuracy, of SVM itself. We demonstrate its superiority by comparative experiments on one simulated and four real-world benchmark datasets.

Keywords
classification, linear dimension reduction, sufficient dimension reduction, decision boundary analysis, support vector machine, regression

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center

Text Reference
Jiayong Zhang and Yanxi Liu, "SVM Decision Boundary Based Discriminative Subspace Induction," tech. report CMU-RI-TR-02-15, Robotics Institute, Carnegie Mellon University, June, 2002

BibTeX Reference
@techreport{Zhang_2002_4048,
   author = "Jiayong Zhang and Yanxi Liu",
   title = "SVM Decision Boundary Based Discriminative Subspace Induction",
   booktitle = "",
   institution = "Robotics Institute",
   month = "June",
   year = "2002",
   number= "CMU-RI-TR-02-15",
   address= "Pittsburgh, PA",
}