Skip to main content
SpringerLink
Log in

Asymptotically efficient recursive estimation for incomplete data models using the observed information

  • Published:
Metrika Aims and scope Submit manuscript

Abstract

For a recursive maximum-likelihood estimator with step lengths decaying as 1/n, an adaptive matrix needs to be incorporated to obtain asymptotic efficiency. Ideally, this matrix should be chosen as the inverse Fisher information matrix, which is usually very difficult to compute for incomplete data models. In this paper we give conditions under which the observed information can be incorporated into the recursive procedure to yield an efficient estimator, and we also investigate the finite sample properties of these estimators by simulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Behboodian J (1972) Information matrix for a mixture of two normal distributions. J. Statist. Comput. Simul. 1:295–314

    Article  MATH  Google Scholar 

  • Csörgö M, Fischler R (1973) Some examples and results in the theory of mixing and random-sum central limit theorems. Period. Math. Hungar. 3:41–57

    Article  MATH  MathSciNet  Google Scholar 

  • Dattatreya GR, Kanal LN (1990) Estimation of mixing probabilities in multiclass finite mixtures. IEEE Trans. Syst. Man Cybern. 20:149–158

    Article  MATH  MathSciNet  Google Scholar 

  • Dattatreya GR, Kanal LN (1991) Asymptotically efficient estimation of prior probabilities in multiclass finite mixtures. IEEE Trans. Inform. Theory 37:482–489

    Article  Google Scholar 

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statist. Soc. 39:1–22

    MATH  MathSciNet  Google Scholar 

  • Dubins LE, Freedman DA (1965) A sharper form of the Borel-Cantelli lemma and the strong law. Ann. Math. Statist. 36:800–807

    MathSciNet  Google Scholar 

  • Fabian V (1978) On asymptotically efficient recursive estimation. Ann. Statist. 6:854–866

    MATH  MathSciNet  Google Scholar 

  • Fabian V (1988) The local asymptotic minimax adaptive property of a recursive estimate. Stat. Prob. Letters 6:383–388

    Article  MATH  MathSciNet  Google Scholar 

  • Hall P, Heyde CC (1980) Martingale limit theory and its applications. Academic Press, New York

    Google Scholar 

  • Hall P, Patil P (1994) On the efficiency of on-line density estimators. IEEE Trans. Inform. Theory 40:1504–1512

    Article  MATH  Google Scholar 

  • Holst U (1987) Recursive estimation of quantiles using recursive kernel density estimators. Seq. Anal. 6:219–237

    Article  MATH  MathSciNet  Google Scholar 

  • Holst U, Lindgren G (1991) Recursive estimation in mixture models with Markov regime. IEEE Trans. Inform. Theory 37:1683–1690

    Article  MATH  MathSciNet  Google Scholar 

  • Kazakos D (1977) Recursive estimation of prior probabilities using a mixture. IEEE Trans. Inform. Theory 23:203–211

    Article  MATH  MathSciNet  Google Scholar 

  • Kushner HJ, Clark DS (1978) Stochastic approximation methods for constrained and unconstrained systems. Springer-Verlag, New York

    Google Scholar 

  • Ljung L (1978) Strong convergence of a stochastic approximation algorithm. Ann. Statist. 6:680–696

    MATH  MathSciNet  Google Scholar 

  • Ljung L, Söderström T (1983) Theory and practice of recursive identification. MIT Press, Cambridge

    MATH  Google Scholar 

  • Major P, Révész P (1973) A limit theorem for the Robbins-Monro approximation. Z. Wahrsch. verw. Geb. 27:79–87

    Article  MATH  Google Scholar 

  • Polyak BT, Juditsky AB (1992) Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30:838–855

    Article  MATH  MathSciNet  Google Scholar 

  • Ruppert D (1982) Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise. Ann. Prob. 10:178–187

    MATH  MathSciNet  Google Scholar 

  • Ruppert D (1991) Stochastic approximation. In: Ghosh BK, Sen PK (ed) Handbook of sequential analysis, Marcel Dekker, New York.

    Google Scholar 

  • Schwabe R (1986) Strong representation of an adaptive stochastic approximation procedure. Stoch. Proc. Appl. 23:115–130

    Article  MATH  MathSciNet  Google Scholar 

  • Stout WF (1974) Almost sure convergence. Academic Press, New York

    MATH  Google Scholar 

  • Titterington DM (1984) Recursive parameter estimation using incomplete data. J. Royal Statist. Soc. B 46:257–267

    MATH  MathSciNet  Google Scholar 

  • Weinstein E, Feder M, Oppenheim AV (1990) Sequential algorithms for parameter estimation based on the Kullback-Leibler information measure. IEEE Trans. Acoust. Speech Signal Process. 38:1652–1654

    Article  MATH  Google Scholar 

Download references

Author information

Author notes

  1. Tobias Rydén

    Present address: Department of Mathematical Statistics, Lund University, Box 118, S-221 00, Lund, Sweden

Authors and Affiliations

  1. Department of Statistics, University of California, Evans Hall, 94720, Berkeley, CA, USA

    Tobias Rydén

Authors
  1. Tobias Rydén
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Supported by the Swedish Natural Science Research Council (contract no. F-PD 10538-301), the Fulbright Commission, and Kungliga Fysiografiska Sällskapet i Lund.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rydén, T. Asymptotically efficient recursive estimation for incomplete data models using the observed information. Metrika 47, 119–145 (1998). https://doi.org/10.1007/BF02742868

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02742868

Key Words

Search

Navigation