Statistics and Computing

, Volume 22, Issue 1, pp 153–168

The structured elastic net for quantile regression and support vector classification

Authors

    • Machine Learning Group, Department of Computer ScienceSaarland University
Article

DOI: 10.1007/s11222-010-9214-z

Cite this article as:
Slawski, M. Stat Comput (2012) 22: 153. doi:10.1007/s11222-010-9214-z

Abstract

In view of its ongoing importance for a variety of practical applications, feature selection via 1-regularization methods like the lasso has been subject to extensive theoretical as well empirical investigations. Despite its popularity, mere 1-regularization has been criticized for being inadequate or ineffective, notably in situations in which additional structural knowledge about the predictors should be taken into account. This has stimulated the development of either systematically different regularization methods or double regularization approaches which combine 1-regularization with a second kind of regularization designed to capture additional problem-specific structure. One instance thereof is the ‘structured elastic net’, a generalization of the proposal in Zou and Hastie (J. R. Stat. Soc. Ser. B 67:301–320, 2005), studied in Slawski et al. (Ann. Appl. Stat. 4(2):1056–1080, 2010) for the class of generalized linear models.

In this paper, we elaborate on the structured elastic net regularizer in conjunction with two important loss functions, the check loss of quantile regression and the hinge loss of support vector classification. Solution paths algorithms are developed which compute the whole range of solutions as one regularization parameter varies and the second one is kept fixed.

The methodology and practical performance of our approach is illustrated by means of case studies from image classification and climate science.

Keywords

Double regularizationElastic netQuantile regressionSolution pathSupport vector classification
Download to read the full article text

Copyright information

© Springer Science+Business Media, LLC 2010