Skip to main content
SpringerLink
Log in

A first-order approximated jackknifed ridge estimator in binary logistic regression

  • Original Paper
  • Published:
Computational Statistics Aims and scope Submit manuscript

Abstract

The purpose of this paper is to solve the problem of multicollinearity that affects the estimation of logistic regression model by introducing first-order approximated jackknifed ridge logistic estimator which is more efficient than the first-order approximated maximum likelihood estimator and has smaller variance than the first-order approximated jackknife ridge logistic estimator. Comparisons of the first-order approximated jackknifed ridge logistic estimator to the first-order approximated maximum likelihood, first-order approximated ridge, first-order approximated r-k class and principal components logistic regression estimators according to the bias, covariance and mean square error criteria are done. Three different estimators for the ridge parameter are also proposed. A real data set is used to see the performance of the first-order approximated jackknifed ridge logistic estimator over the first-order approximated maximum likelihood, first-order approximated ridge logistic, first-order approximated r-k class and first-order approximated principal components logistic regression estimators. Finally, two simulation studies are conducted in order to show the performance of the first-order approximated jackknife ridge logistic estimator.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. \(V(\beta _{0})\) will be denoted as \( V^{(0)}=diag({\hat{\pi }}_{i}^{(0)}(1-{\hat{\pi }}_{i}^{(0)}))\) in the following sections where \({\hat{\pi }}_{i}^{(0)}\) is evaluated at \(\beta _{0}\).

  2. There is no global mechanism for good starting values. Therefore, starting value \({\widehat{\beta }}^{(0)}\) should be known which can equivalently be taken as the real parameter value \(\beta _{0}\). \({\widehat{\beta }} ^{(0)}=\beta _{0}\) is usually the ordinary least squares estimate in practice (see for example Schaefer 1986).

  3. LeCessie and VanHouwelingen (1992) defined a one-step approximation for \( {\hat{\beta }}_{-i}(k)\) as \({\widehat{\beta }}_{-i}^{(1)}(k)={\widehat{\beta }} ^{(1)}(k)-\frac{(X^{\prime }{\widehat{V}}(k)X+kI)^{-1}X_{i}^{\prime }v_{ii}(y_{i}-{\widehat{\pi }}_{i})}{1-h_{ii}(k)}\) where \({\widehat{V}}(k)\), \( {\widehat{\pi }}_{i}\) and \(h_{ii}(k)\) are evaluated at \({\widehat{\beta }}^{(1)}(k)\). However, their approximation does not yield the jackknife estimator.

  4. From Eq. (1), \({\widehat{\alpha }}^{(1)}\) equals \({\widehat{\alpha }}^{(1)}= {\widehat{\alpha }}^{(0)}+(Z^{\prime }V^{(0)}Z)^{-1}Z^{\prime }(y_{i}-{\hat{\pi }}_{i}^{(0)})\) where \({\widehat{\alpha }}^{(0)}\) is the starting value of \( \alpha \). To start the iteration \({\widehat{\alpha }}^{(0)}\) should be known. Therefore, a real parameter, say \(\alpha ^{0}=T^{\prime }\beta _{0}\), can be used.

  5. All comments, results and views in this study belong to the author(s) and these do not bind TURKSTAT.

  6. OECD equivalence scale is used to reflect differences in a household size and composition. This scale gives a weight to all members of the household (and then adds these up to arrive at the equivalised household size): 1 to the first adult; 0.5 to the second and each subsequent person aged 14 and over; 0.3 to each child aged under 14 (see EUROSTAT 2014).

  7. The sub-indices e and j respectively denote the exact and jackknife CIs.

References

  • Aguilera AM, Escabias M, Valderrama MJ (2006) Using principal components for estimating logistic regression with high-dimensional multicollinear data. Comput Stat Data Anal 50:1905–1924

    Article  MathSciNet  MATH  Google Scholar 

  • Bull SB, Hauck WW, Greenwood CMT (1994) Two-step jackknife bias reduction for logistic regression mles. Commun Stat Simul Comput 23(1):59–88

    Article  MATH  Google Scholar 

  • Bull SB, Shelley B, Greenwood CMT, Hauck WW (1997) Jackknife bias reduction for polychotomous logistic regression. Stat Med 16(5):545–560

    Article  Google Scholar 

  • EUROSTAT (2014) Living conditions in Europe. Publication Office of the European Union, Belgium

  • Farebrother RW (1976) Further results on the mean square error of ridge regression. J R Stat Soc B 38:248–250

    MathSciNet  MATH  Google Scholar 

  • Gruber MHJ (1991) The efficiency of jack-knifed and usual ridge type estimators: a comparison. Stat Probab Lett 11(1):49–51

    Article  Google Scholar 

  • Hinkley DV (1977) Jackknifing in unbalanced situations. Technometrics 19(285):292

    MATH  Google Scholar 

  • Kibria BMG (2003) Performance of some new ridge regression estimators. Commun Stat Simul Comput 32(2):419–435

    Article  MathSciNet  MATH  Google Scholar 

  • LeCessie S, VanHouwelingen JC (1992) Ridge estimators in logistic regression. Appl Stat 41(1):191–201

    Article  Google Scholar 

  • McDonald GC, Galarneau DI (1975) A monte carlo evaluation of ridge-type estimators. J Am Stat Assoc 70:407–416

    Article  MATH  Google Scholar 

  • Nelder JA, Wedderburn WM (1972) Generalized linear models. J R Stat Soc A 135(3):370–384

    Article  Google Scholar 

  • Nomura M (1988) On the almost unbiased ridge regression estimator. Commun Stat Simul Comput 17:729–743

    Article  MathSciNet  MATH  Google Scholar 

  • Nyquist H (1988) Applications of the jackknife procedure in ridge regression. Comput Stat Data Anal 6(2):177–183

    Article  MathSciNet  MATH  Google Scholar 

  • Ogoke UP, Nduka EC, Nja ME (2013) The logistic regression model with a modified weight function in survival analysis. Math Theory Model 3(8):12–17

    Google Scholar 

  • Ohtani K (1986) On small sample properties of the almost unbiased generalized ridge estimator. Commun Stat Theory Methods 15:1571–1578

    Article  MathSciNet  MATH  Google Scholar 

  • Özkale MR (2008) A jackknifed ridge estimator in the linear regression model with heteroscedastic or correlated error. Stat Probab Lett 78:3159–3169

    Article  MathSciNet  MATH  Google Scholar 

  • Özkale MR, Arıcan E (2015) First-order r-d class estimator in fbinary logistic regression model. Stat Probab 106:19–29

    Article  MATH  Google Scholar 

  • Özkale MR, Arıcan E (2016) A new biased estimator in logistic regression model. Statistics 50(2):233–253

    MathSciNet  MATH  Google Scholar 

  • Özkale MR, Lemeshow S, Sturdivant R (2018) Logistic regression diagnostics in ridge regression. Comput Stat 33:563–593

    Article  MathSciNet  MATH  Google Scholar 

  • Pregibon D (1981) Logistic regression diagnostics. Ann Stat 9(4):705–724

    Article  MathSciNet  MATH  Google Scholar 

  • Quenouille MH (1956) Notes on bias in estimation. Biometrika 43:353–360

    Article  MathSciNet  MATH  Google Scholar 

  • Rao CR (1973) Linear statistical inference and its applications, 2nd edn. Wiley, New York

    Book  MATH  Google Scholar 

  • Schaefer RL (1983) Bias correction in maximum likelihood logistic regression. Stat Med 2:71–78

    Article  Google Scholar 

  • Schaefer RL (1986) Alternative estimators in logistic regression when the data are collinear. J Stat Comput Simul 25:75–91

    Article  MATH  Google Scholar 

  • Schaefer RL, Roi LD, Wolfe RA (1984) A ridge logistic estimator. Commun Stat Theory Methods 13:99–113

    Article  Google Scholar 

  • Singh B, Chaubey YP (1987) On some improved ridge estimators. Stat Pap 28:53–67

    MathSciNet  MATH  Google Scholar 

  • Singh B, Chaubey YP, Dwivedi TD (1986) An almost unbiased ridge estimator. Sankhya 48:342–346

    MathSciNet  MATH  Google Scholar 

  • Tukey JW (1958) Bias and confidence in not quite large samples (abstract). Ann Math Stat 29:614

    Article  Google Scholar 

  • TurkStat (2011) Income and living conditions survey 2011 micro. Data set (cross-sectional). Ankara

  • Wu J, Asar Y (2016) On almost unbiased ridge logistic estimator for the logistic regression model. Hacet J Math Stat 45(3):989–998

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Faculty of Science and Letters, Cukurova University, Adana, Turkey

    M. Revan Özkale & Engin Arıcan

Authors
  1. M. Revan Özkale
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Engin Arıcan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Revan Özkale.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Özkale, M.R., Arıcan, E. A first-order approximated jackknifed ridge estimator in binary logistic regression. Comput Stat 34, 683–712 (2019). https://doi.org/10.1007/s00180-018-0851-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-018-0851-6

Keywords

Search

Navigation