Generalized Boosting Algorithms for Convex Optimization

Alexander Grubb and J. Andrew (Drew) Bagnell
Proceedings of the 28th International Conference on Machine Learning, May, 2011.

  • Adobe portable document format (pdf) (836KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Boosting is a popular way to derive powerful learners from simpler hypothesis classes. Following previous work (Mason et al., 1999; Friedman, 2000) on general boosting frameworks, we analyze gradient-based descent algorithms for boosting with respect to any convex objective and introduce a new measure of weak learner performance into this setting which generalizes existing work. We present the first weak to strong learning guarantees for the existing gradient boosting work for smooth convex objectives, and also demonstrate that this work fails for non-smooth objectives. To address this issue, we present new algorithms which extend this boosting approach to arbitrary convex loss functions and give corresponding weak to strong convergence results. In addition, we demonstrate experimental results that support our analysis and demonstrate the need for the new algorithms we present.

machine learning, boosting, convex optimization, functional gradient descent

Sponsor: U.S Army Research Laboratory under the Collaborative Technology Alliance Program
Associated Center(s) / Consortia: Vision and Autonomous Systems Center and Field Robotics Center
Associated Project(s): CTA Robotics

Text Reference
Alexander Grubb and J. Andrew (Drew) Bagnell, "Generalized Boosting Algorithms for Convex Optimization," Proceedings of the 28th International Conference on Machine Learning, May, 2011.

BibTeX Reference
   author = "Alexander Grubb and J. Andrew (Drew) Bagnell",
   title = "Generalized Boosting Algorithms for Convex Optimization",
   booktitle = "Proceedings of the 28th International Conference on Machine Learning",
   month = "May",
   year = "2011",