The Lasso and Ridge Regression Yield Biased Estimates of Imbalanced Binary Features

Johan Larsson, Jonas Wallin

Research output: Other contributionMiscellaneousResearch

35 Downloads (Pure)

Abstract

Many regularized methods, such as the lasso and ridge regression, are sensitive to the scales of the features in the data. As a consequence, it has become standard practice to normalize
(center and scale) features such that they share the same scale. For continuous data, the most common strategy is standardization: centering and scaling each feature by its mean and
and standard deviation, respectively. For binary data, especially when it is high-dimensional and sparse, the most common strategy, however, is to not scale at all. In this paper, we show
that this choice has dramatic effects for the estimated model in the case when the binary features are imbalanced and that these effects, moreover, depend on the type regularization
(lasso or ridge) used. In particular, we demonstrate the size of a feature’s corresponding coefficient in the lasso is directly related to its class imbalance and that this effect depends
on the normalization used. We suggest possible remedies for this problem and also discuss the case when data is mixed, that is, contains both continuous and binary features.
Original languageEnglish
Number of pages27
Publication statusUnpublished - 2024

Subject classification (UKÄ)

  • Probability Theory and Statistics

Fingerprint

Dive into the research topics of 'The Lasso and Ridge Regression Yield Biased Estimates of Imbalanced Binary Features'. Together they form a unique fingerprint.

Cite this