Advertisement

Advances in Data Analysis and Classification

, Volume 10, Issue 3, pp 375–390 | Cite as

Quantile regression with group lasso for classification

  • Hussein Hashem
  • Veronica VinciottiEmail author
  • Rahim Alhamzawi
  • Keming Yu
Regular Article

Abstract

Applications of regression models for binary response are very common and models specific to these problems are widely used. Quantile regression for binary response data has recently attracted attention and regularized quantile regression methods have been proposed for high dimensional problems. When the predictors have a natural group structure, such as in the case of categorical predictors converted into dummy variables, then a group lasso penalty is used in regularized methods. In this paper, we present a Bayesian Gibbs sampling procedure to estimate the parameters of a quantile regression model under a group lasso penalty for classification problems with a binary response. Simulated and real data show a good performance of the proposed method in comparison to mean-based approaches and to quantile-based approaches which do not exploit the group structure of the predictors.

Keywords

Quantile regression Binary regression Regularized regression  Gibbs sampling 

Mathematics Subject Classification

62H12 62F15 

References

  1. Alhamzawi R, Yu K (2013) Conjugate priors and variable selection for Bayesian quantile regression. Comput Stat Data Anal 64:209–219MathSciNetCrossRefGoogle Scholar
  2. Alhamzawi R, Yu K, Benoit D (2012) Bayesian adaptive lasso quantile regression. Stat Model 12(3):279–297MathSciNetCrossRefzbMATHGoogle Scholar
  3. Andrews DF, Mallows CL (1974) Scale mixtures of normal distributions. J R Stat Soc Ser B 36:99–102MathSciNetzbMATHGoogle Scholar
  4. Bach F (2008) Consistency of the group lasso and multiple kernel learning. J Mach Learn Res 9:1179–1225MathSciNetzbMATHGoogle Scholar
  5. Bae K, Mallick B (2004) Gene selection using a two-level hierarchical Bayesian model. Bioinformatics 20(18):3423–3430CrossRefGoogle Scholar
  6. Belloni A, Chernozhukov V (2011) Post l\(_1\)-penalized quantile regression in high-dimensional sparse models. Ann Stat 39:82–130MathSciNetCrossRefzbMATHGoogle Scholar
  7. Benoit D, Poel D (2012) Binary quantile regression: a Bayesian approach based on the asymmetric laplace density. J Appl Econ 27(7):1174–1188CrossRefGoogle Scholar
  8. Breheny P, Huang J (2015) Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors. Stat Comput 25:173–187MathSciNetCrossRefzbMATHGoogle Scholar
  9. Davino C, Furno M, Vistocco D (2013) Quantile regression: theory and applications. Wiley, ChichesterGoogle Scholar
  10. Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 31(1):1–22Google Scholar
  11. Genkin A, Lewis DD, Madigan D (2007) Large-scale Bayesian logistic regression for text categorization. Technometrics 49(14):291–304MathSciNetCrossRefGoogle Scholar
  12. Geraci M, Bottai M (2007) Quantile regression for longitudinal data using the asymmetric Laplace distribution. Biostatistics 8(1):140–154CrossRefzbMATHGoogle Scholar
  13. Gramacy R, Polson N (2012) Simulation-based regularized logistic regression. Bayesian Anal 7(3):503–770MathSciNetCrossRefzbMATHGoogle Scholar
  14. Hand D, Vinciotti V (2003) Local versus global models for classification problems: fitting models where it matters. Am Stat 57(2):124–131MathSciNetCrossRefGoogle Scholar
  15. Huang J, Zhang T (2010) The benefit of group sparsity. Ann Stat 38(4):1978–2004MathSciNetCrossRefzbMATHGoogle Scholar
  16. Ji Y, Lin N, Zhang B (2012) Model selection in binary and tobit quantile regression using the Gibbs sampler. Comput Stat Data Anal 56(4):827–839MathSciNetCrossRefzbMATHGoogle Scholar
  17. Koenker R (2005) Quantile regression. CRC Press, Boca RatonGoogle Scholar
  18. Koenker R, Bassett GW (1978) Regression quantiles. Econometrica 46:33–50MathSciNetCrossRefzbMATHGoogle Scholar
  19. Kordas G (2002) Credit scoring using binary quantile regression. In: Statistical data analysis based on the L1-norm and related methods. Statistics for industry and technology. Birkhäuser, Basel, pp 125–137Google Scholar
  20. Kordas G (2006) Smoothed binary regression quantiles. J Appl Econ 21(3):387–407MathSciNetCrossRefGoogle Scholar
  21. Kozumi H, Kobayashi G (2011) Gibbs sampling methods for Bayesian quantile regression. J Stat Comput Simul 81:1565–1578MathSciNetCrossRefzbMATHGoogle Scholar
  22. Krishnapuram B, Carin L, Figueiredo MA, Hartemink AJ (2005) Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Trans Pattern Anal Mach Intell 27:957–968CrossRefGoogle Scholar
  23. Li Y, Zhu J (2008) L1-norm quantile regressions. J Comput Graph Stat 17:163–185CrossRefGoogle Scholar
  24. Li Q, Xi R, Lin N (2010) Bayesian regularized quantile regression. Bayesian Anal 5:1–24MathSciNetCrossRefzbMATHGoogle Scholar
  25. Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
  26. Liu X, Wang Z, Wu Y (2013) Group variable selection and estimation in the tobit censored response model. Comput Stat Data Anal 60:80–89MathSciNetCrossRefGoogle Scholar
  27. Lounici K, Pontil M, Tsybakov A, van de Geer S (2011) Oracle inequalities and optimal inference under group sparsity. Ann Stat 39:2164–2204MathSciNetCrossRefzbMATHGoogle Scholar
  28. Lum K, Gelfand A (2012) Spatial quantile multiple regression using the asymmetric laplace process. Bayesian Anal 7(2):235–258MathSciNetCrossRefzbMATHGoogle Scholar
  29. Manski C (1975) Maximum score estimation of the stochastic utility model of choice. J Econ 3(3):205–228MathSciNetCrossRefzbMATHGoogle Scholar
  30. Manski C (1985) Semiparametric analysis of discrete response: asymptotic properties of the maximum score estimator. J Econ 27(3):313–333MathSciNetCrossRefzbMATHGoogle Scholar
  31. Meier L, van de Geer S, Bühlmann P (2008) The group lasso for logistic regression. J R Stat Soc Ser B 70(1):53–71MathSciNetCrossRefzbMATHGoogle Scholar
  32. Miguéis LV, Benoit DF, Van den Poel D (2013) Enhanced decision support in credit scoring using Bayesian binary quantile regression. J Oper Res Soc 64(9):1374–1383CrossRefGoogle Scholar
  33. Powell J (1984) Least absolute deviations estimation for the censored regression model. J Econ 25:303–325MathSciNetCrossRefzbMATHGoogle Scholar
  34. Sharma D, Bondell H, Zhang H (2013) Consistent group identification and variable selection in regression with correlated predictors. J Comput Graph Stat 22(2):319–340MathSciNetCrossRefGoogle Scholar
  35. Simon N, Friedman J, Hastie T, Tibshirani R (2013) A sparse-group lasso. J Comput Graph Stat 22(2):231–245MathSciNetCrossRefGoogle Scholar
  36. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58:267–288MathSciNetzbMATHGoogle Scholar
  37. Tibshirani R (1997) The lasso method for variable selection in the cox model. Stat Med 16:385–395CrossRefGoogle Scholar
  38. Wei F, Huang J (2010) Consistent group selection in high-dimensional linear regression. Stat Med 16:1369–1384MathSciNetzbMATHGoogle Scholar
  39. Yang Y, Zou H (2015) A fast unified algorithm for solving group-lasso penalized learning problems. Stat Comput (to appear)Google Scholar
  40. Yu K, Moyeed R (2001) Bayesian quantile regression. Stat Probab Lett 54:437–447MathSciNetCrossRefzbMATHGoogle Scholar
  41. Yu K, Cathy C, Reed C, Dunson D (2013) Bayesian variable selection in quantile regression. Stat Interface 6:261–274MathSciNetCrossRefzbMATHGoogle Scholar
  42. Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B 68:49–67MathSciNetCrossRefzbMATHGoogle Scholar
  43. Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101:1418–1429MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Hussein Hashem
    • 1
  • Veronica Vinciotti
    • 1
    Email author
  • Rahim Alhamzawi
    • 2
  • Keming Yu
    • 1
  1. 1.Department of MathematicsBrunel University LondonLondonUK
  2. 2.College of ArtsUniversity of Al-QadisiyahAl DiwaniyahIraq

Personalised recommendations