Logistic Regression for Data Mining and High-Dimensional Classification

Paul Komarek
tech. report CMU-RI-TR-04-34, Robotics Institute, Carnegie Mellon University, May, 2004


Download
  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
The focus of this thesis is fast and robust adaptations of logistic regression (LR) for data mining and high-dimensional classification problems. LR is well-understood and widely used in the statistics, machine learning, and data analysis communities. Its benefits include a firm statistical foundation and a probabilistic model useful for "explaining" the data. There is a perception that LR is slow, unstable, and unsuitable for large learning or classification tasks. Through fast approximate numerical methods, regularization to avoid numerical instability, and an efficient implementation we will show that LR can outperform modern algorithms like Support Vector Machines (SVM) on a variety of learning tasks. Our novel implementation, which uses a modified iteratively re-weighted least squares estimation procedure, can compute model parameters for sparse binary datasets with hundreds of thousands of rows and attributes, and millions or tens of millions of nonzero elements in just a few seconds. Our implementation also handles real-valued dense datasets of similar size.

Keywords
logistic regression, data mining, classification, artificial intelligence, machine learning, statistical learning, irls, iteratively re-weighted least squares

Notes
Associated Lab(s) / Group(s): Auton Lab
Associated Project(s): Auton Project

Text Reference
Paul Komarek, "Logistic Regression for Data Mining and High-Dimensional Classification," tech. report CMU-RI-TR-04-34, Robotics Institute, Carnegie Mellon University, May, 2004

BibTeX Reference
@techreport{Komarek_2004_4693,
   author = "Paul Komarek",
   title = "Logistic Regression for Data Mining and High-Dimensional Classification",
   booktitle = "",
   institution = "Robotics Institute",
   month = "May",
   year = "2004",
   number= "CMU-RI-TR-04-34",
   address= "Pittsburgh, PA",
}