Skip to main content

Quantile regression with group lasso for classification

Abstract

Applications of regression models for binary response are very common and models specific to these problems are widely used. Quantile regression for binary response data has recently attracted attention and regularized quantile regression methods have been proposed for high dimensional problems. When the predictors have a natural group structure, such as in the case of categorical predictors converted into dummy variables, then a group lasso penalty is used in regularized methods. In this paper, we present a Bayesian Gibbs sampling procedure to estimate the parameters of a quantile regression model under a group lasso penalty for classification problems with a binary response. Simulated and real data show a good performance of the proposed method in comparison to mean-based approaches and to quantile-based approaches which do not exploit the group structure of the predictors.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  • Alhamzawi R, Yu K (2013) Conjugate priors and variable selection for Bayesian quantile regression. Comput Stat Data Anal 64:209–219

    MathSciNet  Article  Google Scholar 

  • Alhamzawi R, Yu K, Benoit D (2012) Bayesian adaptive lasso quantile regression. Stat Model 12(3):279–297

    MathSciNet  Article  MATH  Google Scholar 

  • Andrews DF, Mallows CL (1974) Scale mixtures of normal distributions. J R Stat Soc Ser B 36:99–102

    MathSciNet  MATH  Google Scholar 

  • Bach F (2008) Consistency of the group lasso and multiple kernel learning. J Mach Learn Res 9:1179–1225

    MathSciNet  MATH  Google Scholar 

  • Bae K, Mallick B (2004) Gene selection using a two-level hierarchical Bayesian model. Bioinformatics 20(18):3423–3430

    Article  Google Scholar 

  • Belloni A, Chernozhukov V (2011) Post l\(_1\)-penalized quantile regression in high-dimensional sparse models. Ann Stat 39:82–130

    MathSciNet  Article  MATH  Google Scholar 

  • Benoit D, Poel D (2012) Binary quantile regression: a Bayesian approach based on the asymmetric laplace density. J Appl Econ 27(7):1174–1188

    Article  Google Scholar 

  • Breheny P, Huang J (2015) Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors. Stat Comput 25:173–187

    MathSciNet  Article  MATH  Google Scholar 

  • Davino C, Furno M, Vistocco D (2013) Quantile regression: theory and applications. Wiley, Chichester

  • Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 31(1):1–22

    Google Scholar 

  • Genkin A, Lewis DD, Madigan D (2007) Large-scale Bayesian logistic regression for text categorization. Technometrics 49(14):291–304

    MathSciNet  Article  Google Scholar 

  • Geraci M, Bottai M (2007) Quantile regression for longitudinal data using the asymmetric Laplace distribution. Biostatistics 8(1):140–154

    Article  MATH  Google Scholar 

  • Gramacy R, Polson N (2012) Simulation-based regularized logistic regression. Bayesian Anal 7(3):503–770

    MathSciNet  Article  MATH  Google Scholar 

  • Hand D, Vinciotti V (2003) Local versus global models for classification problems: fitting models where it matters. Am Stat 57(2):124–131

    MathSciNet  Article  Google Scholar 

  • Huang J, Zhang T (2010) The benefit of group sparsity. Ann Stat 38(4):1978–2004

    MathSciNet  Article  MATH  Google Scholar 

  • Ji Y, Lin N, Zhang B (2012) Model selection in binary and tobit quantile regression using the Gibbs sampler. Comput Stat Data Anal 56(4):827–839

    MathSciNet  Article  MATH  Google Scholar 

  • Koenker R (2005) Quantile regression. CRC Press, Boca Raton

  • Koenker R, Bassett GW (1978) Regression quantiles. Econometrica 46:33–50

    MathSciNet  Article  MATH  Google Scholar 

  • Kordas G (2002) Credit scoring using binary quantile regression. In: Statistical data analysis based on the L1-norm and related methods. Statistics for industry and technology. Birkhäuser, Basel, pp 125–137

  • Kordas G (2006) Smoothed binary regression quantiles. J Appl Econ 21(3):387–407

    MathSciNet  Article  Google Scholar 

  • Kozumi H, Kobayashi G (2011) Gibbs sampling methods for Bayesian quantile regression. J Stat Comput Simul 81:1565–1578

    MathSciNet  Article  MATH  Google Scholar 

  • Krishnapuram B, Carin L, Figueiredo MA, Hartemink AJ (2005) Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Trans Pattern Anal Mach Intell 27:957–968

    Article  Google Scholar 

  • Li Y, Zhu J (2008) L1-norm quantile regressions. J Comput Graph Stat 17:163–185

    Article  Google Scholar 

  • Li Q, Xi R, Lin N (2010) Bayesian regularized quantile regression. Bayesian Anal 5:1–24

    MathSciNet  Article  MATH  Google Scholar 

  • Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml

  • Liu X, Wang Z, Wu Y (2013) Group variable selection and estimation in the tobit censored response model. Comput Stat Data Anal 60:80–89

    MathSciNet  Article  Google Scholar 

  • Lounici K, Pontil M, Tsybakov A, van de Geer S (2011) Oracle inequalities and optimal inference under group sparsity. Ann Stat 39:2164–2204

    MathSciNet  Article  MATH  Google Scholar 

  • Lum K, Gelfand A (2012) Spatial quantile multiple regression using the asymmetric laplace process. Bayesian Anal 7(2):235–258

    MathSciNet  Article  MATH  Google Scholar 

  • Manski C (1975) Maximum score estimation of the stochastic utility model of choice. J Econ 3(3):205–228

    MathSciNet  Article  MATH  Google Scholar 

  • Manski C (1985) Semiparametric analysis of discrete response: asymptotic properties of the maximum score estimator. J Econ 27(3):313–333

    MathSciNet  Article  MATH  Google Scholar 

  • Meier L, van de Geer S, Bühlmann P (2008) The group lasso for logistic regression. J R Stat Soc Ser B 70(1):53–71

    MathSciNet  Article  MATH  Google Scholar 

  • Miguéis LV, Benoit DF, Van den Poel D (2013) Enhanced decision support in credit scoring using Bayesian binary quantile regression. J Oper Res Soc 64(9):1374–1383

    Article  Google Scholar 

  • Powell J (1984) Least absolute deviations estimation for the censored regression model. J Econ 25:303–325

    MathSciNet  Article  MATH  Google Scholar 

  • Sharma D, Bondell H, Zhang H (2013) Consistent group identification and variable selection in regression with correlated predictors. J Comput Graph Stat 22(2):319–340

    MathSciNet  Article  Google Scholar 

  • Simon N, Friedman J, Hastie T, Tibshirani R (2013) A sparse-group lasso. J Comput Graph Stat 22(2):231–245

    MathSciNet  Article  Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58:267–288

    MathSciNet  MATH  Google Scholar 

  • Tibshirani R (1997) The lasso method for variable selection in the cox model. Stat Med 16:385–395

    Article  Google Scholar 

  • Wei F, Huang J (2010) Consistent group selection in high-dimensional linear regression. Stat Med 16:1369–1384

    MathSciNet  MATH  Google Scholar 

  • Yang Y, Zou H (2015) A fast unified algorithm for solving group-lasso penalized learning problems. Stat Comput (to appear)

  • Yu K, Moyeed R (2001) Bayesian quantile regression. Stat Probab Lett 54:437–447

    MathSciNet  Article  MATH  Google Scholar 

  • Yu K, Cathy C, Reed C, Dunson D (2013) Bayesian variable selection in quantile regression. Stat Interface 6:261–274

    MathSciNet  Article  MATH  Google Scholar 

  • Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B 68:49–67

    MathSciNet  Article  MATH  Google Scholar 

  • Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101:1418–1429

    MathSciNet  Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Veronica Vinciotti.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hashem, H., Vinciotti, V., Alhamzawi, R. et al. Quantile regression with group lasso for classification. Adv Data Anal Classif 10, 375–390 (2016). https://doi.org/10.1007/s11634-015-0206-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11634-015-0206-x

Keywords

  • Quantile regression
  • Binary regression
  • Regularized regression
  • Gibbs sampling

Mathematics Subject Classification

  • 62H12
  • 62F15