Abstract
The Expectation Maximization (EM) algorithm, a popular method for maximum likelihood estimation of parameters, requires a complete data space and construction of a conditional expectation. For many statistical models, these may not be straightforward. This paper proposes a simpler Alternating Minimization (AM) algorithm using a probability generating function (pgf)-based divergence measure for estimation in univariate and bivariate distributions. The performance of the estimation method is studied for the negative binomial and Neyman Type-A distributions in the univariate setting, while for bivariate cases, the bivariate Poisson and the bivariate negative binomial distributions are considered. Comparison is made with direct optimization of pgf-based divergence measure and maximum likelihood (ML) estimates. Results produced via AM in both simulated and real-life datasets show an improvement in comparison to direct pgf optimization, especially in the bivariate setting, with the execution time showing an improvement for large sample sizes when compared to ML. Goodness-of-fit tests show that the pgf divergence measure with AM estimates mostly perform similarly to the ML estimates in terms of power of the test.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Simpson DG (1987) Minimum Hellinger distance estimation for the analysis of count data. J Am Stat Assoc 82:802–807
Kemp CD, Kemp AW (1988) Rapid estimation for discrete distributions. J R Stat Soc D 37:243–255
Sim SZ, Ong SH (2010) Parameter estimation for discrete distributions by generalized Hellinger type divergence based on probability generating function. Commun Stat - Simul Comput 39:305–314
Ng CM, Ong SH, Srivastava HM (2013) Parameter estimation by Hellinger type distance for multivariate distributions based upon probability generating functions. Appl Math Model 37:7374–7385
Sharifdoust M, Ng CM, Ong SH (2014) Probability generating function based Jeffrey’s divergence for statistical inference. Commun Stat - Simul Comput 45:2445–2458
Brusa L, Bartolucci F, Pennoni F (2023) Tempered expectation-maximization algorithm for the estimation of discrete latent variable models. Comput Stat 38:1391–1424
Tseng P (1991) Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J Control Optim 29:119–138
Padcharoen A, Kitkuan D, Kumam P, Rilwan J, Kumam W (2020) Accelerated alternating minimization algorithm for Poisson noisy image recovery. Inverse Probl Sci Eng 28:1031–1056
Zhang T, Liu G, Wang L, Lu Z (2023) Adaptive integral alternating minimization method for robust learning of nonlinear dynamical systems from highly corrupted data. Chaos 33:123112
Csiszar I, Tusnady G (1984) Information geometry and alternating minimization procedures. Stat Decis Suppl Issue 1:205–237
Kocherlakota S, Kocherlakota K (1986) Goodness of fit test for discrete distributions. Commun Stat - Theory Methods 15(3):815–829
Gürtler N, Henze N (2000) Recent and classical goodness-of-fit test for the Poisson distribution. J Stat Plan Inference 90:207–225
Jiménez-Gamero MD, Batsidis A (2017) Minimum distance estimators for count data based on the probability generating function with applications. Metrika 80:503–545
Baringhaus L, Gürtler N, Henze N (2000) Theory & methods: weighted integral test statistics and components of smooth tests of fit. Aust N Z J Stat 42:179–192
Ong SH (1992) The computer generation of bivariate binomial variables with given marginals and correlation. Commun Stat - Simul Comput 21:285–299
Kocherlakota S, Kocherlakota K (1992) Bivariate discrete distributions. Monograph, 1st edn. CRC, New York
Edwards CB, Gurland J (1961) A class of distributions applicable to accidents. J Am Stat Assoc 56:503–517
Zangwill WI (1969) Nonlinear programming: a unified approach. Prentice-Hall, Englewood Cliffs
Gunawardana A, Byrne W (2005) Convergence theorems for generalized alternating minimization procedures. J Mach Learn Res 6:2049–2073
Al-Saleh MF, Al-Batainah FK (2003) Estimation of the shape parameter k of the negative binomial distribution. Appl Math Comput 143:431–441
Bliss CI, Fisher RA (1953) Fitting the negative binomial distribution to biological data. Biometrics 9:176–200
Farmer E, Chambers EG (1940) A study of accident proneness among motor drivers. J R Stat Soc 103:254–256
Acknowledgements
The authors wish to thank the reviewers for their helpful comments which have greatly improved the article.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception, design, simulation study, and data analysis. The first draft of the manuscript was written by R.R. and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Conflict of interest
No funds, grants, or other support was received for the submitted work. The authors have no competing interests to declare that are relevant to the content of the submitted work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rajakrishnan, R., Ong, S.H. & Ng, C.M. Alternating minimization algorithm with a probability generating function-based distance measure. J Eng Math 146, 15 (2024). https://doi.org/10.1007/s10665-024-10349-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10665-024-10349-z
Keywords
- Alternating minimization
- Goodness-of-fit
- Hellinger-type distance measure
- Probability generating function