Skip to main content
SpringerLink
Log in

Likelihood analysis and stochastic EM algorithm for left truncated right censored data and associated model selection from the Lehmann family of life distributions

  • Original Paper
  • Recent Statistical Methods for Survival Analysis
  • Published:
Japanese Journal of Statistics and Data Science Aims and scope Submit manuscript

Abstract

The Lehmann family of distributions includes Weibull, Gompertz, and Lomax models as special cases, all of which are quite useful for modeling lifetime data. Analyses of left truncated right censored data from the Lehmann family of distributions are carried out in this article. In particular, the special cases of Weibull, Gompertz, and Lomax distributions are considered. Maximum likelihood estimates of the model parameters are obtained. The steps of the stochastic expectation maximization algorithm are developed in this context. Asymptotic confidence intervals for the model parameters are constructed using the missing information principle and two parametric bootstrap approaches. Through extensive Monte Carlo simulations, performances of the inferential methods are examined. The effect of model misspecification is also assessed within this family of distributions through simulations. As it is quite important to select a suitable model for a given data, a study of model selection based on left truncated right censored data is carried out through extensive Monte Carlo simulations. Two examples based on real datasets are provided for illustrating the models and methods of inference discussed here.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Balakrishnan, N., & Liu, K. (2018). Semi-parametric likelihood inference for Birnbaum–Saunders frailty model. REVSTAT - Statistical Journal, 16, 231–255.

    MathSciNet  MATH  Google Scholar 

  • Balakrishnan, N., & Mitra, D. (2011). Likelihood inference for log-normal data with left truncation and right censoring with an illustration. Journal of Statistical Planning and Inference, 141, 3536–3553.

    MathSciNet  MATH  Google Scholar 

  • Balakrishnan, N., & Mitra, D. (2012). Left truncated and right censored Weibull data and likelihood inference with an illustration. Computational Statistics and Data Analysis, 56, 4011–4025.

    MathSciNet  MATH  Google Scholar 

  • Balakrishnan, N., & Mitra, D. (2013). Likelihood inference based on left truncated and right censored data from a gamma distribution. IEEE Transactions on Reliability, 62, 679–688.

    Google Scholar 

  • Balakrishnan, N., & Mitra, D. (2014a). Some further issues concerning likelihood inference for left truncated and right censored lognormal data. Communications in Statistics - Simulation and Computation, 43, 400–416.

    MathSciNet  MATH  Google Scholar 

  • Balakrishnan, N., & Mitra, D. (2014b). EM-based likelihood inference for some lifetime distributions based on left truncated and right censored data and associated model discrimination (with discussions). South African Statistical Journal, 48, 125–204.

    MathSciNet  MATH  Google Scholar 

  • Bordes, L., & Chauveau, D. (2014). Comments: EM-based likelihood inference for some lifetime distributions based on left truncated and right censored data and associated model discrimination. South African Statistical Journal, 48, 197–200.

    MATH  Google Scholar 

  • Celeux, G., Chauveau, D., & Diebolt, J. (1996). Stochastic versions of the EM algorithm: an experimental study in the mixture case. Journal of Statistical Computation and Simulation, 55, 287–314.

    MATH  Google Scholar 

  • Celeux, G., & Diebolt, J. (1985). The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Computational Statistics Quarterly, 2, 73–82.

    Google Scholar 

  • Chauveau, D. (1995). A stochastic EM algorithm for mixtures with censored data. Journal of Statistical Planning and Inference, 46, 1–25.

    MathSciNet  MATH  Google Scholar 

  • Chernobay, A., Rachev, S., & Fabozzi, F. (2005). Composite goodness-of-fit tests for left-truncated loss samples, Technical Report, University of California Santa Barbara.

  • Cox, D. R. (1961). Tests of separate families of hypotheses. Proceedings of the 4th Berkeley Symposium in Mathematical Statistics and Probability, Vol. 1, University of California Press.

  • Cox, D. R. (1962). Further results on test of separate families of hypotheses. Journal of the Royal Statistical Society, Series B, 24, 406–424.

    MathSciNet  MATH  Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  • Dey, A. K., & Kundu, D. (2009). Discriminating among the log-normal, Weibull and generalized exponential distributions. IEEE Transactions on Reliability, 58, 416–424.

    Google Scholar 

  • Dey, A. K., & Kundu, D. (2010). Discriminating between the log-normal and log-logistic distributions. Communications in Statistics - Theory and Methods, 39, 280–292.

    MathSciNet  MATH  Google Scholar 

  • Dey, A. K., & Kundu, D. (2012). Discriminating between the Weibull and log-normal distribution for Type-II censored data. Statistics, 46, 197–214.

    MathSciNet  MATH  Google Scholar 

  • Diebolt, J., & Celeux, G. (1993). Asymptotic properties of a stochastic EM algorithm for estimating mixing proportions. Communications in Statistics: Stochastic Models, 9, 599–613.

    MathSciNet  MATH  Google Scholar 

  • Dörre, A. (2020). Bayesian estimation of a lifetime distribution under doublt truncation caused by time-restricted data collection. Statistical Papers, 61, 945–965.

    MathSciNet  MATH  Google Scholar 

  • Dörre, A., Huang, C. Y., Tseng, Y. K., & Emura, T. (2021). Likelihood-based analysis of doubly-truncated data under the location-scale and AFT model. Computational Statistics, 36, 375–408.

    MathSciNet  MATH  Google Scholar 

  • Dufour, R., & Maag, U. R. (1978). Distribution results for modified Kolmogorov-Smirnov statistics for truncated and censored samples. Technometrics, 20, 29–32.

    MATH  Google Scholar 

  • Emura, T., Matsui, S., & Rondeau, V. (2019). Personalized Dynamic Prediction of Survival in Survival Analysis with Correlated Endpoints (pp. 77–93). Singapore: Springer.

    MATH  Google Scholar 

  • Emura, T., & Shiu, S. (2016). Estimation and model selection for left-truncated and right-censored lifetime data with application to electric power transformers analysis. Communications in Statistics - Simulation and Computation, 45, 3171–3189.

    MathSciNet  MATH  Google Scholar 

  • Gijbels, I., & Wang, J. L. (1993). Strong representations of the survival function estimator for truncated and censored data with applications. Journal of Multivariate Analysis, 47, 210–229.

    MathSciNet  MATH  Google Scholar 

  • Gompertz, B. (1825). On the nature of the function expressive of the law of human mortality, and on a new mode of determining the value of life contingencies. Philosophical Transactions of the Royal Society of London, 115, 513–583.

    Google Scholar 

  • Gotwalt, C., Xu, L., Hong, Y., & Meeker, W. Q. (2018). Applications of the Fractional-Random-Weight Bootstrap. arXiv preprint arXiv:1808.08199.

  • Hong, Y., Meeker, W. Q., & McCalley, J. D. (2009). Prediction of remaining life of power transformers based on left truncated and right censored lifetime data. The Annals of Applied Statistics, 3, 857–879.

    MathSciNet  MATH  Google Scholar 

  • Kaplan, E. L., & Meier, P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53, 457–481.

    MathSciNet  MATH  Google Scholar 

  • Klein, J. P., & Moeschberger, M. L. (2003). Survival Analysis: Techniques for Censored and Truncated Data (2nd ed.). New York: Springer-Verlag.

    MATH  Google Scholar 

  • Kundu, D. (2005). Discriminating between the normal and Laplace distributions. In N. Balakrishnan, H. N. Nagaraja, & N. Kannan (Eds.), Advances in Ranking and Selection, Multiple Comparisons, and Reliability (pp. 65–79). Boston: Birkhauser.

    Google Scholar 

  • Kundu, D., & Manglick, A. (2004). Discriminating between the Weibull and log-normal distributions. Naval Research Logistics, 51, 893–905.

    MathSciNet  MATH  Google Scholar 

  • Kundu, D., Mitra, D., & Ganguly, A. (2017). Analysis of left truncated right censored competing risks data. Computational Statistics and Data Analysis, 108, 12–26.

    MathSciNet  MATH  Google Scholar 

  • Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 226–233.

    MathSciNet  MATH  Google Scholar 

  • Marshall, A. W., Meza, J. C., & Olkin, I. (2001). Can data recognize its parent distribution? Journal of Computational and Graphical Statistics, 10, 555–580.

    MathSciNet  Google Scholar 

  • Marshall, A. W., & Olkin, I. (2007). Life Distributions; Structure of Nonparametric, Semiparametric, and Parametric Families. New York: Springer.

    MATH  Google Scholar 

  • McLachlan, G., & Krishnan, T. (2008). The EM Algorithm and Extensions (2nd ed.). New Jersey: Wiley.

    MATH  Google Scholar 

  • Ng, H. K. T., & Ye, Z. (2014). Comments: EM-based likelihood inference for some lifetime distributions based on left truncated and right censored data and associated model discrimination. South African Statistical Journal, 48, 177–180.

    MATH  Google Scholar 

  • Nielsen, S. F. (2000). The stochastic EM algorithm: Estimation and asymptotic results. Bernoulli, 6, 457–489.

    MathSciNet  MATH  Google Scholar 

  • Pereira, B. B., & Pereira, C. A. B. (2016). Model Choice in Nonnested Families. Berlin: Springer-Verlag.

    MATH  Google Scholar 

  • Schey, H. M. (1977). The asymptotic distribution of the one-sided Kolmogorov-Smirnov statistic for truncated data. Communications in Statistics - Theory and Methods, 6, 1361–1366.

    MathSciNet  MATH  Google Scholar 

  • Shinohara, S., Lin, Y. H., Michimae, H., & Emura, T. (2020). Dynamic lifetime prediction using a Weibull-based bivariate failure time model. Communications in Statistics - Simulation and Computation. https://doi.org/10.1080/03610918.2020.1855449.

    Article  Google Scholar 

  • Taylor, J. A., & Jakeman, A. J. (1985). Identification of a distributional model. Communications in Statistics - Simulation and Computation, 14, 497–508.

    Google Scholar 

  • Tsai, W. Y., Jewel, N. P., & Wang, M. C. (1987). A note on the product-limit estimator under right censoring and left truncation. Biometrika, 74, 883–886.

    MATH  Google Scholar 

  • Tse, S. (2003). Strong Gaussian approximations in the left truncated and right censored model. Statistica Sinica, 13, 275–282.

    MathSciNet  MATH  Google Scholar 

  • Wang, B. X., Yu, K., & Jones, M. C. (2010). Inference under progressively type-II right-censored sampling for certain lifetime distributions. Technometrics, 52, 453–460.

    MathSciNet  Google Scholar 

  • Ye, Z., & Ng, H. K. T. (2014). On analysis of incomplete field failure data. The Annals of Applied Statistics, 8, 1713–1727.

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are thankful to the anonymous reviewers for their many helpful comments and suggestions. Thanks are also due to Professor Emura for extending an invitation to contribute this article to the Special Issue.

Author information

Authors and Affiliations

  1. Indian Institute of Management Udaipur, Udaipur, Rajasthan, 313001, India

    Debanjan Mitra

  2. Indian Institute of Technology Kanpur, Kanpur, Uttar Pradesh, 208016, India

    Debasis Kundu

  3. McMaster University, Hamilton, Ontario, L8S 4K1, Canada

    Narayanaswamy Balakrishnan

Authors
  1. Debanjan Mitra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debasis Kundu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Narayanaswamy Balakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debanjan Mitra.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mitra, D., Kundu, D. & Balakrishnan, N. Likelihood analysis and stochastic EM algorithm for left truncated right censored data and associated model selection from the Lehmann family of life distributions. Jpn J Stat Data Sci 4, 1019–1048 (2021). https://doi.org/10.1007/s42081-021-00115-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42081-021-00115-1

Keywords

Search

Navigation