Skip to main content
SpringerLink
Log in

Indirect Maximum Likelihood Estimation

  • Chapter
  • First Online:
Optimization and Its Applications in Control and Data Sciences

Part of the book series: Springer Optimization and Its Applications ((SOIA,volume 115))

  • 1300 Accesses

Abstract

We study maximum likelihood estimators (henceforth MLE) in experiments consisting of two stages, where the first-stage sample is unknown to us, but the second-stage samples are known and depend on the first-stage sample. The setup is similar to that in parametric empirical Bayes models, and arises naturally in numerous applications. However, problems arise when the number of second-level observations is not the same for all first-stage observations. As far as we know, this situation has been discussed in very few cases (see Brandel, Empirical Bayes methods for missing data analysis. Technical Report 2004:11, Department of Mathematics, Uppsala University, Sweden, 2004 and Carlin and Louis, Bayes and Empirical Bayes Methods for Data Analysis, 2nd edn. Chapman & Hall, Boca Raton, 2000) and no analytic expression for the indirect maximum likelihood estimator was derived there. The novelty of our paper is that it details and exemplifies this point. Specifically, we study in detail two situations:

  1. 1.

    Both levels correspond to normal distributions; here we are able to find an explicit formula for the MLE and show that it forms uniformly minimum-variance unbiased estimator (henceforth UMVUE).

  2. 2.

    Exponential first-level and Poissonian second-level; here the MLE can usually be expressed only implicitly as a solution of a certain polynomial equation. It seems that the MLE is usually not a UMVUE.

In both cases we discuss the intuitive meaning of our estimator, its properties, and show its advantages vis-\(\grave{\mathrm{a}}\)-vis other natural estimators.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aït-Sahalia, Y., Kimmela, R.: Maximum likelihood estimation of stochastic volatility models. J. Financ. Econ. 83 (2), 413–452 (2007)

    Article  Google Scholar 

  2. Anderson, M.P., Woessner, W.W.: Applied Groundwater Modeling: Simulation of Flow and Advective Transport, 2nd edn. Academc, New York (1992)

    Google Scholar 

  3. Assefi, T.: Stochastic Processes and Estimation Theory with Applications. Wiley, New York (1979)

    MATH  Google Scholar 

  4. Ayebo, A., Kozubowski, T.J.: An asymmetric generalization of Gaussian and Laplace laws. J. Probab. Stat. Sci. 1 (2), 187–210 (2003)

    Google Scholar 

  5. Berouti, M., Schwartz, R., Makhoul, J.: Enhancement of speech corrupted by acoustic noise. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 4, pp.208–211 (1979)

    Google Scholar 

  6. Bishop, C.M.: Latent variable models. In: Jordan, M. (ed.) Learning in Graphical Models, pp. 371–403. MIT, London (1999)

    Google Scholar 

  7. Brenner, J.L.: A unified treatment and extension of some means of classical analysis–I: comparison theorems. J. Combin. Inform. Syst. Sci. 3, 175–199 (1978)

    MathSciNet  MATH  Google Scholar 

  8. Brandel, J.: Empirical Bayes methods for missing data analysis. Technical Report 2004:11, Department of Mathematics, Uppsala University, Sweden (2004)

    Google Scholar 

  9. Carlin, B.P., Louis, T.A.: Bayes and Empirical Bayes Methods for Data Analysis, 2nd edn. Chapman & Hall, Boca Raton (2000)

    Book  MATH  Google Scholar 

  10. Chen, X., Hong, H., Tamer, E.: Measurement error models with auxiliary data. Rev. Econ. Stud. 72 (2), 343–366 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  11. DeGroot, M.: Probability and Statistics, 2nd edn. Addison-Wesley, Boston (1986)

    MATH  Google Scholar 

  12. Douarche, F., Buisson, L., Ciliberto, S., Petrosyan, A.: A simple noise subtraction technique. Rev. Sci. Instrum. 75 (12), 5084–5089 (2004)

    Article  Google Scholar 

  13. Gertsbakh, I.: Reliability Theory: With Applications to Preventive Maintenance. Springer, New York (2000)

    MATH  Google Scholar 

  14. Gourieroux, C., Monfort, A., Renault, E.: Indirect inference. J. Appl. Econ. 8, S85–S118 (1993)

    Article  MATH  Google Scholar 

  15. Hardy, G.H., Littlewood, J.E., Pólya, G.: Inequalities, 2nd edn. Cambridge University Press, Cambridge (1952)

    MATH  Google Scholar 

  16. Ideker, T., Thorsson, V., Siegel, A.F., Hood, L.E.: Testing for differentially-expressed genes by maximum-likelihood analysis of microarray data. J. Comput. Biol. 7 (6), 805–817 (2000)

    Article  Google Scholar 

  17. Jøsang, A., Ismail, R.: The beta reputation system. In: Proceedings of the 15-th Bled Conference on Electronic Commerce, Bled, Slovenia, pp.17–19 (2002)

    Google Scholar 

  18. Jøsang, A., Haller, J.: Dirichlet reputation systems. In: Proceedings of the Second International Conference on Availability, Reliability and Security (ARES 2007), Vienna, pp.112–119 (2007)

    Google Scholar 

  19. Kagan, A., Li, B.: An identity for the Fisher information and Mahalanobis distance. J. Stat. Plann. Inference 138 (12), 3950–3959 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  20. Lehmann, E.L., Casella, G.: Theory of Point Estimation, 2nd edn. Springer, New York (1998)

    MATH  Google Scholar 

  21. Linhard, K., Haulick, T.: Spectral noise subtraction with recursive gain curves. In: Fifth International Conference on Spoken Language Processing, (ICSLP-1998), paper 0109, Sydney (1998)

    Google Scholar 

  22. Maritz, J. S., Lwin, T.: Assessing the performance of empirical Bayes estimators. Ann. Inst. Stat. Math. 44 (4), 641–657 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  23. Mood, A.M., Graybill, F., Boes, D.: Introduction to the Theory of Statistics, 3rd edn. McGraw Hill, Singapore (1974)

    MATH  Google Scholar 

  24. Shi, G., Nehorai, A.: Maximum likelihood estimation of point scatterers for computational time-reversal imaging. Commun. Inform. Syst. 5 (2), 227–256 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  25. Smith, Jr. A.A.: Indirect inference. In: Steven, N.D., Lawrence, E.B. (eds.) The New Palgrave Dictionary of Economics. Palgrave Macmillan (2008). DOI: 10.1057/9780230226203.0778

  26. Tuchler, M., Singer, A.C., Koetter, R.: Minimum mean squared error equalization using a priori information. IEEE Trans. Signal Process. 50 (3), 673–683 (2002)

    Article  Google Scholar 

  27. Walter, G.G., Hamedani, G.G.: Bayes empirical Bayes estimation for discrete exponential families. Ann. Inst. Stat. Math. 41 (1), 101–119 (1989)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors express their gratitude to E. Gudes and N. Gal-Oz for encouraging them to look into various questions regarding reputation systems, which eventually led to this research, and for their comments on the first draft of the paper. The authors also thank I. Gertsbakh for many discussions related to this topic, and A. Kagan and L. Stefanski for their comments on the first draft of the paper.

The authors acknowledge Deutsche Telekom Laboratories at Ben-Gurion University for support of this research.

Author information

Authors and Affiliations

  1. Departments of Mathematics and Computer Science, Ben-Gurion University, Beer Sheva, Israel

    Daniel Berend

  2. Department of Mathematics, Ben-Gurion University and Deutsche Telekom Laboratories at Ben-Gurion University, Beer Sheva, Israel

    Luba Sapir

Authors
  1. Daniel Berend
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luba Sapir
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Berend .

Editor information

Editors and Affiliations

  1. Department of Industrial and Systems Engineering, Ohio University, Athens, Ohio, USA

    Boris Goldengorin

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Berend, D., Sapir, L. (2016). Indirect Maximum Likelihood Estimation. In: Goldengorin, B. (eds) Optimization and Its Applications in Control and Data Sciences. Springer Optimization and Its Applications, vol 115. Springer, Cham. https://doi.org/10.1007/978-3-319-42056-1_4

Download citation

Publish with us

Policies and ethics

Search

Navigation