Hyperparameter-selection for sparse regression: A probablistic approach

Research output: Chapter in Book/Report/Conference proceedingPaper in conference proceedingpeer-review


The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter becomes almost independent of the noise variance. Simulation examples illustrate how the proposed method outperforms both cross-validation and the Bayesian Information Criterion in terms of computational complexity and support recovery.

Original languageEnglish
Title of host publicationConference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
PublisherIEEE - Institute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Electronic)9781538618233
Publication statusPublished - 2018 Apr 10
Event51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 - Pacific Grove, United States
Duration: 2017 Oct 292017 Nov 1


Conference51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
Country/TerritoryUnited States
CityPacific Grove

Subject classification (UKÄ)

  • Probability Theory and Statistics

Free keywords

  • extreme value distribution
  • model order estimation
  • regularization
  • sparse estimation


Dive into the research topics of 'Hyperparameter-selection for sparse regression: A probablistic approach'. Together they form a unique fingerprint.

Cite this