Skip to main content
SpringerLink
Log in

Robust ridge M-estimators with pretest and Stein-rule shrinkage for an intercept term

  • Original Paper
  • Published:
Japanese Journal of Statistics and Data Science Aims and scope Submit manuscript

Abstract

If the data contain both multicollinearity and outliers, the ridge M-estimator is the preferred estimator to the usual least square estimator (Silvapulle, Aust J Stat 33:319–333, 1991). Many other estimators, such as the pretest ridge M-estimator and Stein-rule shrinkage ridge M-estimator, have been developed on the basis of the ridge M-estimator. However, all these existing estimators do not consider shrinkage estimation for the intercept term. Hence, there are some rooms for improving the existing estimators by improving the estimator for the intercept term. In this paper, we propose several new ridge M-estimators for regression coefficients and an intercept term by introducing pretest and Stein-rule shrinkage schemes. Our estimators are obtained by using the Jimichi-type ridge matrix that allows shrinkage operations to be applicable to both the intercept term and regression coefficients. We conduct Monte Carlo simulation studies to examine the performance of the proposed estimators. For demonstration, we analyze the corporate finance data from the Nikkei Economic Electronic Databank System in Japan, and the gene expression data from Japanese ovarian cancer patients.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Ahmed, S. E. (2014). Penalty, shrinkage and pretest strategies: Variable selection and estimation. New York: Springer.

    Book  Google Scholar 

  • Ahmed, S. E., & Krzanowski, W. J. (2004). Biased estimation in a simple multivariate regression model. Computational Statistics and Data Analysis, 45, 689–696.

    Article  MathSciNet  Google Scholar 

  • Allen, D. M. (1974). The relationship between variable selection and data augmentation and a method for prediction. Technometrics, 16, 125–127.

    Article  MathSciNet  Google Scholar 

  • Bachmaier, M. (2000). Efficiency comparison of M-estimates for scale at t-distributions. Statistical Papers, 41, 53–64.

    Article  Google Scholar 

  • Beaton, A. E., & Tukey, J. W. (1974). The fitting of power series, meaning polynomials, illustrated on band-spectroscopic data. Technometrics, 16, 147–185.

    Article  Google Scholar 

  • Brown, P. J. (1977). Centering and scaling in ridge regression. Technometrics, 19, 35–36.

    MathSciNet  MATH  Google Scholar 

  • Chen, A. C., & Emura, T. (2017). A modified Liu-type estimator with an intercept term under mixture experiments. Communications in Statistics - Theory and Methods, 46, 6645–6667.

    Article  MathSciNet  Google Scholar 

  • Emura, T. (2020). joint.Cox: Penalized likelihood estimation and dynamic prediction under the joint frailty-copula models between tumour progression and death for meta-analysis. CRAN, version: 3.8.

  • Emura, T., Nakatochi, M., Matsui, S., Michimae, H., & Rondeau, V. (2018). Personalized dynamic prediction of death according to tumour progression and high-dimensional genetic factors: meta-analysis with a joint model. Statistical Methods in Medical Research, 27, 2842–2858.

    Article  MathSciNet  Google Scholar 

  • Emura, T., Matsui, S., & Chen, H. Y. (2019). compound.Cox: Univariate feature selection and compound covariate for predicting survival. Computer Methods and Program in Biomedicine, 168, 21–37.

    Article  Google Scholar 

  • Farcomeni, A., & Ventura, L. (2012). An overview of robust methods in medical research. Statistical Methods in Medical Research, 21, 111–133.

    Article  MathSciNet  Google Scholar 

  • Ganzfried, B. F., Riester, M., Haibe-Kains, B., et al. (2013). curatedOvarianData: clinically annotated data for the ovarian cancer transcriptome. Database (Oxford). https://doi.org/10.1093/database/bat013.

    Article  Google Scholar 

  • Golub, G. H., Heath, M., & Wahba, G. (1979). Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21, 215–223.

    Article  MathSciNet  Google Scholar 

  • Gruber, M. H. J. (1998). Improving efficiency by shrinkage: The James-Stein and ridge regression estimators. New York: Routledge.

    MATH  Google Scholar 

  • Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.

    Article  Google Scholar 

  • Hoerl, A. E., Kennard, R. W., & Baldwin, K. F. (1975). Ridge regression: some simulations. Communications in Statistics, 4, 105–123.

    Article  Google Scholar 

  • Huang, Y. F., & Hwang, C. H. (2013). Compare of the influence measures in linear regression models. Journal of the Chinese Statistical Association, 51, 225–244.

    Google Scholar 

  • Huber, P. J. (1981). Robust statistic. New York: Wiley.

    Book  Google Scholar 

  • James. W., Stein, C. (1961). Estimation with quadratic loss, in Proceedings of the Fourth Berkeley Symposium. Berkeley: University of California Press, 1: 361–379.

  • Jang, D. H., & Anderson-Cook, C. M. (2015). Visualization approaches for evaluating ridge regression estimators in mixture and mixture-process experiments. Quality and Reliability Engineering International, 31, 1483–1494.

    Article  Google Scholar 

  • Jimichi, M. (2010). Building of financial database servers. Technical report, ISBN: 9784990553005, https://kwansei.repo.nii.ac.jp/?action=repository_uri&item_id=20624.

  • Jimichi, M. (2016). Shrinkage regression estimators and their feasibilities. Kwansei Gakuin University Press.

  • Jimichi, M., & Inagaki, N. (1993). Centering and scaling in ridge regression. Statistical Science and Data Analysis, 3, 77–86.

    MathSciNet  MATH  Google Scholar 

  • Loesgen, K. H. (1990). A generalization and Bayesian interpretation of ridge-type estimators with good prior means. Statistical Papers, 31, 147–154.

    Article  MathSciNet  Google Scholar 

  • Maronna, R. A. (2011). Robust ridge regression for high-dimensional data. Technometrics, 53, 44–53.

    Article  MathSciNet  Google Scholar 

  • McDonald, G. C., & Galarneau, D. I. (1975). A Monte Carlo evaluation of some ridge-type estimators. Journal of the American Statistical Association, 70, 407–416.

    Article  Google Scholar 

  • Michimae, H., Yoshida, A., Emura, T., Matsunami, M., & Nishimura, K. (2018). Reconsidering the estimation of costs of phenotypic plasticity using the robust ridge estimator. Ecological Informatics, 44, 7–20.

    Article  Google Scholar 

  • Michimae, H., Matsunami, M., & Emura, T. (2020). Robust ridge regression for estimating the effects of correlated gene expressions on phenotypic traits. Environmental and Ecological Statistics, 27, 41–72.

    Article  Google Scholar 

  • Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis (5th ed.). Hoboken: Wiley.

    MATH  Google Scholar 

  • Norouzirad, M., & Arashi, M. (2019). Preliminary test and Stein-type shrinkage ridge estimators in robust regression. Statistical Papers, 60, 1849–1882.

    Article  MathSciNet  Google Scholar 

  • Norouzirad, M., Arashi, M., & Ahmed, S. E. (2017). Improved robust ridge M-estimation. Journal of Statistical Computation and Simulation, 87, 3469–3490.

    Article  MathSciNet  Google Scholar 

  • Saleh, A. K. M. E. (2006). Theory of preliminary test and stein-type estimation with applications. Hoboken: Wiley.

    Book  Google Scholar 

  • Saleh, A. K. M. E., & Shiraishi, T. (1989). On some R- and M-estimators of regression parameters under uncertain restriction. Journal of the Japan Statistical Society, 19, 129–137.

    MathSciNet  MATH  Google Scholar 

  • She, Y., & Owen, A. B. (2011). Outlier detection using nonconvex penalized regression. Journal of the American Statistical Association, 106, 626–639.

    Article  MathSciNet  Google Scholar 

  • Silvapulle, M. J. (1991). Robust ridge regression based on an M-estimator. Australian Journal of Statistics, 33, 319–333.

    Article  MathSciNet  Google Scholar 

  • Tabatabaey, S. M. M., Saleh, A. K. M. E., & Kibria, B. M. G. (2004). Estimation strategies for parameters of the linear regression models with spherically symmetric distributions. Journal of Statistical Research, 38, 13–31.

    MathSciNet  Google Scholar 

  • Tukey, J. M. (1977). Exploratory data analysis. Reading: Addison-Wesley.

    MATH  Google Scholar 

  • Wang, Y. G., Lin, X., Zhu, M., & Bai, Z. (2007). Robust estimation using the Huber function with a data-dependent tuning constant. Journal of Computational and Graphical Statistics, 16, 468–481.

    Article  MathSciNet  Google Scholar 

  • Wisnowski, J. W., Montgomery, D. C., & Simpson, J. R. (2001). A comparative analysis of multiple outlier detection procedures in the linear regression model. Computational Statistics and Data Analysis, 36, 351–382.

    Article  MathSciNet  Google Scholar 

  • Wong, K. Y., & Chiu, S. N. (2015). An iterative approach to minimize the mean-squared error in ridge regression. Computational Statistics, 30, 625–639.

    Article  MathSciNet  Google Scholar 

  • Yang, S. P., & Emura, T. (2017). A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing. Communications in Statistics - Simulation and Computation, 46, 6083–6105.

    Article  MathSciNet  Google Scholar 

  • Yoshihara, K., Tajima, A., Yahata, T., et al. (2010). Gene expression profile for predicting survival in advanced-stage serous ovarian cancer across two independent datasets. PLoS ONE, 5, e9615.

    Article  Google Scholar 

Download references

Acknowledgements

The authors kindly thank associate editor and one referee for their helpful comments and corrections that greatly improved the paper. Emura T is funded by the grant from the Ministry of Science and Technology, Taiwan (MOST, 107-2118-M-008-003-MY3).

Author information

Authors and Affiliations

  1. Institute of Statistical Science, Academia Sinica, Taipei, Taiwan

    Jia-Han Shih

  2. Graduate Institute of Statistics, National Central University, Taoyuan, Taiwan

    Ting-Yu Lin

  3. School of Business Administration, Kwansei Gakuin University, Nishinomiya, Japan

    Masayuki Jimichi

  4. Department of Information Management, Chang Gung University, Taoyuan, Taiwan

    Takeshi Emura

  5. Division of Hematology-Oncology, Chang Gung Memorial Hospital at Linkou, No.259, Wenhua 1st Rd., Guishan Dist., Taoyuan, 33302, Taiwan

    Takeshi Emura

Authors
  1. Jia-Han Shih
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ting-Yu Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masayuki Jimichi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takeshi Emura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Takeshi Emura.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shih, JH., Lin, TY., Jimichi, M. et al. Robust ridge M-estimators with pretest and Stein-rule shrinkage for an intercept term. Jpn J Stat Data Sci 4, 107–150 (2021). https://doi.org/10.1007/s42081-020-00089-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42081-020-00089-6

Keywords

Search

Navigation