The Robotics Institute

Carnegie Mellon Robotics Institute

Over the last century Component Analysis (CA) methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Canonical Correlation Analysis (CCA) and Spectral Clustering(SC) have been extensively used as a feature extraction step for modeling, classification, visualization and clustering. This technical report proposes a unified framework to formulate many CA methods as a least-squares estimation problem. We show how many CA methods correspond to a particular instance of a weighted kernel reduced rank regression (KRRR). The least-squares formulation allows better understanding of normalization factors, and provides an easier generalization of CA techniques. In particular, we derive the matrix expressions for weighted generalizations of PCA, LDA, SC and CCA (including kernel extensions), and show its effectiveness on synthetic and real problems. Finally, we suggest an efficient numerical method to solve WKRRR.

Keywords

Component Analysis, PCA,, LDA, CCA, Normalized cuts, Clasification, Clustering

Notes

Note: projects associated component analysis for data processing.

Text Reference

Fernando De la Torre Frade, "A Least-Squares Unified View of PCA, LDA, CCA and Spectral Graph Methods," tech. report CMU-RI-TR-08-29, Robotics Institute, Carnegie Mellon University, May, 2008

BibTeX Reference

@techreport{De_la_Torre_Frade_2008_6086,
   author = "Fernando {De la Torre Frade}",
   title = "A Least-Squares Unified View of PCA, LDA, CCA and Spectral Graph Methods",
   booktitle = "",
   institution = "Robotics Institute",
   month = "May",
   year = "2008",
   number= "CMU-RI-TR-08-29",
   address= "Pittsburgh, PA",
   Notes = "projects associated component analysis for data processing."
}

The Robotics Institute

Menu