Skip to main content
SpringerLink
Log in

Ridge estimation for multinomial logit models with symmetric side constraints

  • Original Paper
  • Published:
Computational Statistics Aims and scope Submit manuscript

Abstract

In multinomial logit models, the identifiability of parameter estimates is typically obtained by side constraints that specify one of the response categories as reference category. When parameters are penalized, shrinkage of estimates should not depend on the reference category. In this paper we investigate ridge regression for the multinomial logit model with symmetric side constraints, which yields parameter estimates that are independent of the reference category. In simulation studies the results are compared with the usual maximum likelihood estimates and an application to real data is given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Anderson JA (1984) Regression and ordered categorical variables (with discussion). J R Statis Soc Series B 46:1–30

    MATH  Google Scholar 

  • Bühlmann P, Hothorn T (2007) Boosting algorithms: regularization, prediction and model fitting (with discussion). Stat Sci 22:477–505

    Article  MATH  Google Scholar 

  • Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 33(1):1–22

    Google Scholar 

  • Hoerl E, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12:55–67

    Article  MATH  Google Scholar 

  • James G, Radchenko P (2009) A generalized dantzig selector with shrinkage tuning. Biometrika 96:323–337

    Article  MathSciNet  MATH  Google Scholar 

  • Krishnapuram B, Carin L, Figueiredo MA, Hartemink AJ (2005) Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Trans Pattern Anal Mach Intell 27:957–968

    Article  Google Scholar 

  • LeCessie S, Houwelingen V (1992) Ridge estimators in logistic regression. Appl Stat 41:191–201

    Article  Google Scholar 

  • Nyquist H (1991) Restricted estimation of generalized linear models. Appl Stat 40:133–141

    Article  MATH  Google Scholar 

  • Park MY, Hastie T (2007) L1-regularization path algorithm for generalized linear models. J R Stat Soc B 69:659–677

    Article  MathSciNet  Google Scholar 

  • Raza MA (2008) Risk factors associated with injection initiation among drug users: comparing hierarchical modeling with traditional logistic regression. Master’s thesis, College of Statistical and Actuarial Sciences, University of the Punjab, Lahore

  • Schaefer R (1986) Alternative estimators in logistic regression when the data are collinear. J Stat Comput Simul 25:75–91

    Article  MATH  Google Scholar 

  • Schaefer R, Roi L, Wolfe R (1984) A ridge logistic estimator. Commun Stat Theory Methods 13:99–113

    Article  Google Scholar 

  • Segerstedt B (1992) On ordinary ridge regression in generalized linear models. Commun Stat Theory Methods 21:2227–2246

    Article  MathSciNet  MATH  Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via lasso. J R Stat Soc B 58:267–288

    MathSciNet  MATH  Google Scholar 

  • Tutz G, Binder H (2006) Generalized additive modelling with implicit variable selection by likelihood based boosting. Biometrics 62:961–971

    Article  MathSciNet  MATH  Google Scholar 

  • Zhu J, Hastie T (2004) Classification of gene microarrays by penalized logistic regression. Biostatistics 5:427–443

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Government College University Faisalabad, Faisalabad, Pakistan

    Faisal Maqbool Zahid

  2. Ludwig–Maximilians-Universität Munich, Akademiestraße 1, 80799 , Munich, Germany

    Gerhard Tutz

Authors
  1. Faisal Maqbool Zahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerhard Tutz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Faisal Maqbool Zahid.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zahid, F.M., Tutz, G. Ridge estimation for multinomial logit models with symmetric side constraints. Comput Stat 28, 1017–1034 (2013). https://doi.org/10.1007/s00180-012-0341-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-012-0341-1

Keywords

Search

Navigation