The Hessian Screening Rule

Johan Larsson; Jonas Wallin

The Hessian Screening Rule

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

Abstract

Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `₁-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems
Editors	S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
Publisher	Curran Associates, Inc
Pages	25404-25421
Volume	35
ISBN (Electronic)	9781713871088
Publication status	Published - 2022 Dec 6
Event	36th Conference on Neural Information Processing Systems, NeurIPS 2022 - New Orleans, United States Duration: 2022 Nov 28 → 2022 Dec 9

Publication series

Name	Advances in Neural Information Processing Systems
Volume	35
ISSN (Print)	1049-5258

Conference

Conference	36th Conference on Neural Information Processing Systems, NeurIPS 2022
Country/Territory	United States
City	New Orleans
Period	2022/11/28 → 2022/12/09

Bibliographical note

Funding Information:
We would like to thank Małgorzata Bogdan for valuable comments. This work was funded by the Swedish Research Council through grant agreement no. 2020-05081 and no. 2018-01726. The computations were enabled by resources provided by LUNARC. The results shown here are in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.

Publisher Copyright:
© 2022 Neural information processing systems foundation. All rights reserved.

Subject classification (UKÄ)

Probability Theory and Statistics

Cite this

@inproceedings{8303a35ca8304ed59ea41f33d6c0fb66,

title = "The Hessian Screening Rule",

abstract = "Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `1-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.",

author = "Johan Larsson and Jonas Wallin",

note = "Funding Information: We would like to thank Ma{\l}gorzata Bogdan for valuable comments. This work was funded by the Swedish Research Council through grant agreement no. 2020-05081 and no. 2018-01726. The computations were enabled by resources provided by LUNARC. The results shown here are in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga. Publisher Copyright: {\textcopyright} 2022 Neural information processing systems foundation. All rights reserved.; 36th Conference on Neural Information Processing Systems, NeurIPS 2022 ; Conference date: 28-11-2022 Through 09-12-2022",

year = "2022",

month = dec,

day = "6",

language = "English",

volume = "35",

series = "Advances in Neural Information Processing Systems",

publisher = "Curran Associates, Inc",

pages = "25404--25421",

editor = "S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh",

booktitle = "Advances in Neural Information Processing Systems",

}

Larsson, J & Wallin, J 2022, The Hessian Screening Rule. in S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho & A Oh (eds), Advances in Neural Information Processing Systems. vol. 35, Advances in Neural Information Processing Systems, vol. 35, Curran Associates, Inc, pp. 25404-25421, 36th Conference on Neural Information Processing Systems, NeurIPS 2022, New Orleans, United States, 2022/11/28.

The Hessian Screening Rule. / Larsson, Johan; Wallin, Jonas.
Advances in Neural Information Processing Systems. ed. / S. Koyejo; S. Mohamed; A. Agarwal; D. Belgrave; K. Cho; A. Oh. Vol. 35 Curran Associates, Inc, 2022. p. 25404-25421 (Advances in Neural Information Processing Systems; Vol. 35).

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

TY - GEN

T1 - The Hessian Screening Rule

AU - Larsson, Johan

AU - Wallin, Jonas

N1 - Funding Information: We would like to thank Małgorzata Bogdan for valuable comments. This work was funded by the Swedish Research Council through grant agreement no. 2020-05081 and no. 2018-01726. The computations were enabled by resources provided by LUNARC. The results shown here are in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga. Publisher Copyright: © 2022 Neural information processing systems foundation. All rights reserved.

PY - 2022/12/6

Y1 - 2022/12/6

N2 - Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `1-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.

AB - Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `1-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.

UR - http://www.scopus.com/inward/record.url?scp=85163193107&partnerID=8YFLogxK

M3 - Paper in conference proceeding

AN - SCOPUS:85163193107

VL - 35

T3 - Advances in Neural Information Processing Systems

SP - 25404

EP - 25421

BT - Advances in Neural Information Processing Systems

A2 - Koyejo, S.

A2 - Mohamed, S.

A2 - Agarwal, A.

A2 - Belgrave, D.

A2 - Cho, K.

A2 - Oh, A.

PB - Curran Associates, Inc

T2 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022

Y2 - 28 November 2022 through 9 December 2022

ER -

The Hessian Screening Rule

Abstract

Publication series

Conference

Bibliographical note

Subject classification (UKÄ)

Other files and links

Fingerprint

Optimization and Algorithms in Sparse Regression: Screening Rules, Coordinate Descent, and Normalization

Optimization and Algorithms in Sparse Regression: Screening Rules, Coordinate Descent, and Normalization

Cite this

The Hessian Screening Rule

Abstract

Publication series

Conference

Bibliographical note

Subject classification (UKÄ)

Other files and links

Fingerprint

Research output

Optimization and Algorithms in Sparse Regression: Screening Rules, Coordinate Descent, and Normalization

Projects

Optimization and Algorithms in Sparse Regression: Screening Rules, Coordinate Descent, and Normalization

Cite this