Skip to main content
SpringerLink
Log in

Acceleration of the EM algorithm using the vector epsilon algorithm

  • Original Paper
  • Published:
Computational Statistics Aims and scope Submit manuscript

Abstract

The expectation–maximization (EM) algorithm is a very general and popular iterative computational algorithm to find maximum likelihood estimates from incomplete data and broadly used to statistical analysis with missing data, because of its stability, flexibility and simplicity. However, it is often criticized that the convergence of the EM algorithm is slow. The various algorithms to accelerate the convergence of the EM algorithm have been proposed. The vector ε algorithm of Wynn (Math Comp 16:301–322, 1962) is used to accelerate the convergence of the EM algorithm in Kuroda and Sakakihara (Comput Stat Data Anal 51:1549–1561, 2006). In this paper, we provide the theoretical evaluation of the convergence of the ε-accelerated EM algorithm. The ε-accelerated EM algorithm does not use the information matrix but only uses the sequence of estimates obtained from iterations of the EM algorithm, and thus it keeps the flexibility and simplicity of the EM algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Avram S (2003). Practical extrapolation methods, theory and applications. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  2. Brezinski C and Zaglia MR (1991). Extrapolation methods: theory and practice. Elsevier, Amsterdam

    MATH  Google Scholar 

  3. Dempster AP, Laird NM and Rubin DB (1977). Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39: 1–22

    MATH  MathSciNet  Google Scholar 

  4. Jamshidian M and Jennrich RI (1993). Conjugate gradient acceleration of the EM algorithm. J Am Stat Assoc 88: 221–228

    Article  MATH  MathSciNet  Google Scholar 

  5. Jamshidian M and Jennrich RI (1997). Acceleration of the EM algorithm by using quasi-Newton methods. J R Stat Soc Ser B 59: 569–587

    Article  MATH  MathSciNet  Google Scholar 

  6. Kuroda M and Sakakihara M (2006). Accelerating the convergence of the EM algorithm using the vector ε algorithm. Comput Stat Data Anal 51: 1549–1561

    Article  MathSciNet  Google Scholar 

  7. Laird NM, Lange K and Stram DO (1987). Maximum likelihood computations with repeated measures: application of the EM algorithm. J Am Stat Ass 82: 97–105

    Article  MATH  MathSciNet  Google Scholar 

  8. Lange K (1995). A quasi Newton acceleration of the EM algorithm. Stat Sin 5: 1–18

    MATH  Google Scholar 

  9. Louis TA (1982). Finding the observed information matrix when using the EM algorithm. J R Stat Soc Ser B 44: 226–233

    MATH  MathSciNet  Google Scholar 

  10. Meilijson I (1989). A fast improvement to the EM algorithm on its own terms. J Roy Stat Soc Ser B 51: 127–138

    MATH  MathSciNet  Google Scholar 

  11. Meng XL and Rubin DB (1994). On the global and componentwise rates of convergence of the EM algorithm. Linear Algebra Appli 199: 413–425

    Article  MATH  MathSciNet  Google Scholar 

  12. Schafer JL (1997). Analysis of incomplete multivariate data. Chapman and Hall, London

    MATH  Google Scholar 

  13. Wynn P (1961). The epsilon algorithm and operational formulas of numerical analysis. Math Comp 15: 151–158

    Article  MATH  MathSciNet  Google Scholar 

  14. Wynn P (1962). Acceleration techniques for iterated vector and matrix problems. Math Comp 16: 301–322

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Mathematical Sciences, Peking University, Beijing, 100871, China

    Mingfeng Wang & Zhi Geng

  2. Department of Socio-Information, Okayama University of Science, 1-1 Ridaicho, Okayama, 700-0005, Japan

    Masahiro Kuroda

  3. Department of Information Science, Okayama University of Science, 1-1 Ridaicho, Okayama, 700-0005, Japan

    Michio Sakakihara

Authors
  1. Mingfeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masahiro Kuroda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michio Sakakihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masahiro Kuroda.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, M., Kuroda, M., Sakakihara, M. et al. Acceleration of the EM algorithm using the vector epsilon algorithm. Comput Stat 23, 469–486 (2008). https://doi.org/10.1007/s00180-007-0089-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-007-0089-1

Keywords

Search

Navigation