The Hessian Screening Rule

Research output: Chapter in Book/Report/Conference proceedingPaper in conference proceedingpeer-review

Abstract

Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `1-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems
EditorsS. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
PublisherCurran Associates, Inc
Pages25404-25421
Volume35
ISBN (Electronic)9781713871088
Publication statusPublished - 2022 Dec 6
Event36th Conference on Neural Information Processing Systems, NeurIPS 2022 - New Orleans, United States
Duration: 2022 Nov 282022 Dec 9

Publication series

NameAdvances in Neural Information Processing Systems
Volume35
ISSN (Print)1049-5258

Conference

Conference36th Conference on Neural Information Processing Systems, NeurIPS 2022
Country/TerritoryUnited States
CityNew Orleans
Period2022/11/282022/12/09

Bibliographical note

Funding Information:
We would like to thank Małgorzata Bogdan for valuable comments. This work was funded by the Swedish Research Council through grant agreement no. 2020-05081 and no. 2018-01726. The computations were enabled by resources provided by LUNARC. The results shown here are in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.

Publisher Copyright:
© 2022 Neural information processing systems foundation. All rights reserved.

Subject classification (UKÄ)

  • Probability Theory and Statistics

Fingerprint

Dive into the research topics of 'The Hessian Screening Rule'. Together they form a unique fingerprint.

Cite this