Skip to main content
SpringerLink
Log in

Preliminary test and Stein-type shrinkage ridge estimators in robust regression

  • Regular Article
  • Published:
Statistical Papers Aims and scope Submit manuscript

Abstract

A statistician may face with a dataset that suffers from multicollinearity and outliers, simultaneously. The Huberized ridge (HR) estimator is a technique that can be used here. On the other hand, an expert may claim that some/all the variables should be removed from the analysis, due to inappropriateness, that imposes a prior information that all coefficients equal to zero (in the form of a restriction) to the analysis. In such situations, one may consider the HR estimation under the subspace restriction. In this paper, we introduce some improved estimators for verifying this claim. They are employed to improve the performance of the HR estimator in the multiple regression model. Advantages of the proposed estimators over the usual HR estimator are demonstrated through a Monte Carlo simulation as well as two real data examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The data are available on the website (http://people.umass.edu/be640/yr2004/resources/data2002/VO2.txt).

  2. This data set is available at http://www.stat.wisc.edu/~gvludwig/fall_2012/bodyfat.csv.

References

  • Akdeniz F (2002) More on the pre-test estimator in ridge regression. Commun Stat Theory Methods 31:987–994

    MathSciNet  MATH  Google Scholar 

  • Akdeniz F, Kaciranlar S (2001) More on the new biased estimator in linear regression. Sankhy B 63:321–325

    MathSciNet  MATH  Google Scholar 

  • Alheety MI, Kibria BMG (2014) A generalized stochastic restricted ridge regression estimator. Commun Stat Theory Methods 43:4415–4427

    MathSciNet  MATH  Google Scholar 

  • Alpu O, Samkar H (2010) Liu estimator based on an m estimator. Turk Klin J Biostat 2:49–53

    Google Scholar 

  • Arslan O, Billor N (2000) Robust liu estimator for regression based on an M-estimator. J Appl Stat 27:39–47

    MATH  Google Scholar 

  • Askin RG, Montgomery DC (1980) Augmented robust estimation. Technometrics 22:333–341

    MATH  Google Scholar 

  • Askin RG, Montgomery DC (1984) An analysis of condtrained robust regression estimators. Navoal Logist Q 32:283–296

    MATH  Google Scholar 

  • El-Salam M (2013) The efficiency of some robust ridge regression for handling multicollinearity and non-normals errors problems. Appl Math Sci 7:3831–3846

    MathSciNet  Google Scholar 

  • Gibbons DGA (1981) Simulation study of some ridge estimators. J Am Stat Assoc 76:131–139

    MATH  Google Scholar 

  • Gruber MHJ (1986) Improving efficiency by shrinkage the James-Stein and ridge regression estimators. Springer, New York

    MATH  Google Scholar 

  • Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12:55–67

    MATH  Google Scholar 

  • Huber PJ (1981) Robust statistics. Wiley, Hoboken

    MATH  Google Scholar 

  • Jadhav NH, Kashid DN (2011) A jackknifed ridge M-estimator for regression model with multicollinearity and outliers. J Stat Theory Pract 5:659–673

    MathSciNet  MATH  Google Scholar 

  • Jadhav NH, Kashid DN (2014) Robust winsorized shrinkage estimators for linear regression models. J Modern Appl Stat Methods 13:131–150

    Google Scholar 

  • James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, pp. 361–379, University of California Press, Berkeley. http://projecteuclid.org/euclid.bsmsp/1200512173

  • Kan B, Alpu O, Yazici B (2013) Robust ridge and robust liu estimator for regression based on the lts estimator. J Appl Stat 40:644–655

    MathSciNet  Google Scholar 

  • Kibria BMG (1996) On preliminary test ridge regression estimators for linear restrictions in a regression model with non-normal disturbances. Commun Stat Theory Methods 25:2349–2369

    MathSciNet  MATH  Google Scholar 

  • Kibria BMG (2003) Performance of some new ridge regression estimators. Commun Stat Simul Comput 32:419–435

    MathSciNet  MATH  Google Scholar 

  • Kibria BMG (2004a) One some ridge regression estimators under possible stochastic constraints. Pak J Stat All Ser 20:1–24

    MATH  Google Scholar 

  • Kibria BMG (2004b) Performance of the shrinkage preliminary test ridge regression estimators based on the conflicting of w, lr and lm tests. J Stat Comput Simul 74:793–810

    MathSciNet  MATH  Google Scholar 

  • Kibria BMG (2012) Some liu and ridge-type estimators and their properties under the ill-conditioned gaussian linear regression model. J Stat Comput Simul 82:1–17

    MathSciNet  MATH  Google Scholar 

  • Kibria BMG, Saleh AKME (2003) Effect of w, lr, and lm tests on the performance of preliminary test ridge regression estimators. J Jpn Stat Soc 33:119–136

    MathSciNet  MATH  Google Scholar 

  • Kibria BMG, Saleh AKME (2004a) Performance of positive rule estimator in the ill-conditioned gaussian regression model. Calcutta Stat Assoc Bull 55:209–239

    MathSciNet  MATH  Google Scholar 

  • Kibria BMG, Saleh AKME (2004b) Preliminary test ridge regression estimators with students t errors and conflicting test-statistics. Metrika 59:105–124

    MathSciNet  MATH  Google Scholar 

  • Lawrence KD, Marsh LC (1984) Robust ridge estimation methods for predicting US coal mining fatalities. Commun Stat Theory Methods 13:139–149

    Google Scholar 

  • McDonald GC, Galarneau DI (1975) A monte carlo evaluation of some ridge-type estimators. J Am Stat Assoc 70:407–416

    MATH  Google Scholar 

  • Montgomery DC, Askin RG (1981) Problems of nonnormality and multicollinearity for forecasting methods based on least squares. AIIE Trans 13:102–115

    MathSciNet  Google Scholar 

  • Montgomery DC, Peck EA (1992) Introduction to linear regression analysis, 2nd edn. Wiley, Hoboken

    MATH  Google Scholar 

  • Muniz G, Kibria BMG (2010) On some ridge regression estimators: an empirical comparisons. Commun Stat Simul Comput 38:621–630

    MathSciNet  MATH  Google Scholar 

  • Muniz G, Kibria BMG, Mansson KM, Shukur G (2012) On developing ridge regression parameters: a graphical investigation. SORT 36:115–138

    MathSciNet  MATH  Google Scholar 

  • Norouzirad M, Arashi M (2017) Supplemental materials: The necessity of using shrinkage ridge M-estimator when multicollinearity and outliers are present in a dataset

  • Ozturk F, Akdeniz F (2000) Ill-conditioning and multicollinearity. Linear Algebra Appl 321:295–305

    MathSciNet  MATH  Google Scholar 

  • Pati KD, Adnan R, Rasheed BA (2014) Ridge least trimmed squares estimators in presence of multicollinearity and outliers. Nat Sci 12:1–8

    Google Scholar 

  • Penrose K, Nelson A, Fisher A (1985) Generalized body composition prediction equation for men using simple measurement techniques. Med Sci Sports Exerc 17:189

    Google Scholar 

  • Pfaffenberger RC, Dielman TE (1984) A modified ridge regression estimator using the least absolute value criterion in the multiple linear regression model. In: Proceedings of the American Institute for Decision Sciences, pp. 791–793

  • Pfaffenberger RC, Dielman TE (1985) A comparison of robust ridge estimators. In: Proceedings of the American Statistical Association Business and Economic Statistics Section, pp. 631–635

  • Saleh AKME (2006) Theory of preliminary test and Stein-type estimation with applications. Wiley, Hoboken

    MATH  Google Scholar 

  • Saleh AKME, Kibria BMG (1993) Performance of some new preliminary test ridge regression estimators and their properties. Commun Stat Theory Methods 22:2747–2764

    MathSciNet  MATH  Google Scholar 

  • Saleh AKME, Shiraishi T (1989) On some r and m estimators of regression parameters under uncertain restriction. J Jpn Stat Soc 19:129–137

    MathSciNet  MATH  Google Scholar 

  • Samkar H, Alpu O (2010) Ridge regression based on some robust estimators. J Modern Appl Stat Methods 9:495–501

    Google Scholar 

  • Sarker N (1992) A new estimator combining the ridge regression and the restricted least squares method of estimation. Commun Stat Theory Methods 21:1987–2000

    MathSciNet  MATH  Google Scholar 

  • Sengupta D, Jammalamadaka SR (2003) Linear models: an integrated approach. World Scientific Publishing Company, Singapore

    MATH  Google Scholar 

  • Silvapull MJ (1991) Robust ridge regression based on an M estimator. Aust N Z J Stat 33:319–333

    MathSciNet  Google Scholar 

  • Susanti Y, Pratiwi H, Sulistijowati S, Liana T (2014) M estimation, S estimation, and MM estimation in robust regression. Int J Pure Appl Math 91:349–360

    MATH  Google Scholar 

  • Tabakan G, Akdeniz F (2010) Difference-based ridge estimator of parameters in partial linear model. Stat Papers 51:357–368

    MathSciNet  MATH  Google Scholar 

  • Tabatabaey SMM, Kibria BGM, Saleh AKMdE (2004a) Estimation strategies for parameters of the linear regression models with spherically symmetric distributions. J Stat Res 38:13–31

    MathSciNet  Google Scholar 

  • Tabatabaey SMM, Saleh AKME, Kibria BGM (2004b) Simultaneous estimation of regression parameters with spherically symmetric errors under possible stochastic constraints. Int J Stat Sci 3:1–20

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, School of Mathematical Sciences, Shahrood University of Technology, Shahrood, Iran

    M. Norouzirad & M. Arashi

Authors
  1. M. Norouzirad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Arashi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Arashi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1182 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Norouzirad, M., Arashi, M. Preliminary test and Stein-type shrinkage ridge estimators in robust regression. Stat Papers 60, 1849–1882 (2019). https://doi.org/10.1007/s00362-017-0899-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00362-017-0899-3

Keywords

Search

Navigation