Hyperparameter-selection for sparse regression: A probablistic approach

Research output: Chapter in Book/Report/Conference proceedingPaper in conference proceeding

Abstract

The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter becomes almost independent of the noise variance. Simulation examples illustrate how the proposed method outperforms both cross-validation and the Bayesian Information Criterion in terms of computational complexity and support recovery.

Details

Authors
Organisations
Research areas and keywords

Subject classification (UKÄ) – MANDATORY

  • Probability Theory and Statistics

Keywords

  • extreme value distribution, LASSO, model order estimation, regularization, sparse estimation
Original languageEnglish
Title of host publicationConference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
PublisherIEEE - Institute of Electrical and Electronics Engineers Inc.
Pages853-857
Number of pages5
Volume2017-October
ISBN (Electronic)9781538618233
Publication statusPublished - 2018 Apr 10
Publication categoryResearch
Peer-reviewedYes
Event51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 - Pacific Grove, United States
Duration: 2017 Oct 292017 Nov 1

Conference

Conference51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
CountryUnited States
CityPacific Grove
Period2017/10/292017/11/01

Related projects

Tönu Pullerits, Andreas Jakobsson, Khadga Jung Karki & Zhengjun Wang

2017/07/012020/06/30

Project: ResearchInterdisciplinary research, Internal collaboration (LU)

View all (1)