Robotics Institute: Blockwise Coordinate Descent Procedures for the Multi-task Lasso, with Applications to Neural Semantic Basis Discovery

Blockwise Coordinate Descent Procedures for the Multi-task Lasso, with Applications to Neural Semantic Basis Discovery

Han Liu, Mark Palatucci, and Jian Zhang
International Conference on Machine Learning, June, 2009.

Download

Adobe portable document format (pdf) (257KB)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

We develop a cyclical blockwise coordinate descent algorithm for the multi-task Lasso that efficiently solves problems with thousands of features and tasks. The main result shows that a closed-form Winsorization operator can be obtained for the sup-norm penalized least squares regression. This allows the algorithm to find solutions to very large-scale problems far more efficiently than existing methods. This result complements the pioneering work of Friedman, et al. (2007) for the single-task Lasso. As a case study, we use the multi-task Lasso as a variable selector to discover a semantic basis for predicting human neural activation. The learned solution outperforms the standard basis for this task on the majority of test participants, while requiring far fewer assumptions about cognitive neuroscience. We demonstrate how this learned basis can yield insights into how the brain represents the meanings of words.

Notes

Note: Best Student Paper Award

Text Reference

Han Liu, Mark Palatucci, and Jian Zhang, "Blockwise Coordinate Descent Procedures for the Multi-task Lasso, with Applications to Neural Semantic Basis Discovery," International Conference on Machine Learning, June, 2009.

BibTeX Reference

@inproceedings{Palatucci_2009_6390,
   author = "Han Liu and Mark Palatucci and Jian Zhang",
   title = "Blockwise Coordinate Descent Procedures for the Multi-task Lasso, with Applications to Neural Semantic Basis Discovery",
   booktitle = "International Conference on Machine Learning",
   month = "June",
   year = "2009",
   Notes = "Best Student Paper Award"
}

The Robotics Institute

Menu