Skip to main content
SpringerLink
Log in

Likelihood Construction and Estimation

  • Chapter
  • First Online:
Essential Statistical Inference

Part of the book series: Springer Texts in Statistics ((STS,volume 120))

Abstract

After a statistical model for the observed data has been formulated, the likelihood of the data is the natural starting point for inference in many statistical problems. This function typically leads to essentially automatic methods of inference, including point and interval estimation, and hypothesis testing. In fact, likelihood methods are generally asymptotically optimal provided the assumed statistical model is correct. Thus, the likelihood is the foundation of classical model-based statistical inference. In this chapter we describe, and often derive, basic likelihood methods and results. We start with construction of the likelihood in numerous examples and then move to various inferential techniques. The development in this chapter follows the classical frequentist approach to inference. However, the Bayesian approach to inference is equally dependent on the likelihood, and the material in this chapter is essential to understanding the Bayesian methods in Chapter 4.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Agresti, A. (2002).Categorical Data Analysis. New Jersey: Wiley.

    Book  MATH  Google Scholar 

  • Barndorff-Nielsen, O. (1978).Information and Exponential Families in Statistical Theory. John Wiley & Sons.

    Google Scholar 

  • Barndorff-Nielsen, O. (1982). Exponential families. In Banks, D. L., Read, C. B., and Kotz, S., editors,Encyclopedia of Statistical Sciences (9 vols. plus Supplement), Volume 2, pages 587–596. John Wiley & Sons.

    Google Scholar 

  • Bartholomew, D. J. (1957). A problem in life testing.Journal of the American Statistical Association, 52:350–355.

    Article  Google Scholar 

  • Berger, J. O. and Wolpert, R. L. (1984).The Likelihood Principle. Institute of Mathematical Statistics.

    Google Scholar 

  • Bickel, P. J. and Doksum, K. A. (1981). An analysis of transformations revisited.Journal of the American Statistical Association, 76:296–311.

    Article  MATH  MathSciNet  Google Scholar 

  • Box, G. E. P. and Cox, D. R. (1964). An analysis of transformations.Journal of the Royal Statistical Society Series B-Statistical Methodology, 26:211–252.

    MATH  MathSciNet  Google Scholar 

  • Breslow, N. E. and Clayton, D. G. (1993). Approximate inference in generalized linear mixed models.Journal of the American Statistical Association, 88:9–25.

    MATH  Google Scholar 

  • Brown, L. D. (1986).Fundamentals of Statistical Exponential Families: with Applications in Statistical Decision Theory. Institute of Mathematical Statistics.

    Google Scholar 

  • Brownie, C., Anderson, D. R., Burnham, K. P., and Robson, D. S. (1985).Statistical Inference from Band Recovery Data: A Handbook (Second Edition). U.S. Fish and Wildlife Service [U.S. Department of Interior].

    Google Scholar 

  • Carroll, R. J., Ruppert, D., Stefanski, L. A., and Crainiceanu, C. M. (2006).Measurement error in nonlinear models: a modern perspective. Chapman & Hall.

    Google Scholar 

  • Casella, G. and Berger, R. L. (2002).Statistical Inference. Duxbury Press.

    Google Scholar 

  • Chernoff, H. and Lehmann, E. L. (1954). The use of maximum likelihood estimates in χ2 tests for goodness of fit.Annals of Mathematical Statistics, 25:579–586.

    Article  MATH  MathSciNet  Google Scholar 

  • Cox, D. R. (1970).Analysis of binary Data. Chapman & Hall.

    Google Scholar 

  • Cox, D. R. and Snell, E. J. (1989).Analysis of Binary Data. Chapman & Hall Ltd.

    Google Scholar 

  • Cressie, N. and Read, T. R. C. (1984). Multinomial goodness-of-fit tests.Journal of the Royal Statistical Society, Series B: Methodological, 46:440–464.

    MATH  MathSciNet  Google Scholar 

  • Darmois, G. (1935). The laws of probability to exhaustive estimation.Comptes rendus hebdomadaires des seances de l academie des sciences, 200:1265–1266.

    Google Scholar 

  • Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via em algorithm.Journal of the Royal Statistical Society Series B-Methodological, 39:1–38.

    MATH  MathSciNet  Google Scholar 

  • Efron, B. and Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information (C/R: P482-487).Biometrika, 65:457–481.

    Article  MATH  MathSciNet  Google Scholar 

  • Embrechts, P. A. L., Pugh, D., and Smith, R. L. (1985).Statistical Extremes and Risks. Course Notes. Imperial College, London, Dept. of Mathematics.

    Google Scholar 

  • Fisher, R. A. (1912). On an absolute criterion for fitting frequency curves.Messenger of Mathematics, 41:155–160.

    Google Scholar 

  • Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics.Philos. Trans. Roy. Soc. London Ser. A, 222:309–368.

    Article  MATH  Google Scholar 

  • Fouskakis, D. and Draper, D. (2002). Stochastic optimization: A review.International Statistical Review, 70(3):315–349.

    Article  MATH  Google Scholar 

  • Glasser, M. (1965). Regression analysis with dependent variable censored.Biometrics, 21:300–306.

    Article  MathSciNet  Google Scholar 

  • Graybill, F. A. (1988).Matrices with Applications in Statistics. Wadsworth.

    Google Scholar 

  • Hald, A. (1998).A History of Mathematical Statistics from 1750 to 1930. Wiley, New York.

    MATH  Google Scholar 

  • Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems (C/R: P338-340).Journal of the American Statistical Association, 72:320–338.

    Article  MATH  MathSciNet  Google Scholar 

  • Hernandez, F. and Johnson, R. A. (1980). The large-sample behavior of transformations to normality.Journal of the American Statistical Association, 75:855–861.

    Article  MATH  MathSciNet  Google Scholar 

  • Hinkley, D. V. and Runger, G. (1984). The analysis of transformed data (C/R: P309-320).Journal of the American Statistical Association, 79:302–309.

    Article  MATH  MathSciNet  Google Scholar 

  • Hirji, K. F., Mehta, C. R., and Patel, N. R. (1987). Computing distributions for exact logistic regression.Journal of the American Statistical Association, 82:1110–1117.

    Article  MATH  MathSciNet  Google Scholar 

  • Johansen, S. (1979).Introduction to the Theory of Regular Exponential Families. University of Copenhagen.

    Google Scholar 

  • Koopman, B. O. (1936). On distributions admitting a sufficient statistic.Transactions of the American Mathematical Society, 39(3):399–409.

    Article  MathSciNet  Google Scholar 

  • Lambert, D. (1992). Zero-inflated Poisson regression, with an application to defects in manufacturing.Technometrics, 34:1–14.

    Article  MATH  Google Scholar 

  • Larsen, R. J. and Marx, M. L. (2001).An Introduction to Mathematical Statistics and Its Applications. Prentice-Hall Inc.

    Google Scholar 

  • Lawless, J. F. (1982).Statistical Models and Methods for Lifetime Data. John Wiley & Sons.

    Google Scholar 

  • Lehmann, E. L. (1983).Theory of Point Estimation. John Wiley & Sons.

    Google Scholar 

  • Lehmann, E. L. (1986).Testing Statistical Hypotheses. John Wiley & Sons.

    Google Scholar 

  • Lehmann, E. L. and Casella, G. (1998).Theory of Point Estimation. Springer-Verlag Inc.

    Google Scholar 

  • Leroux, B. G. and Puterman, M. L. (1992). Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models.Biometrics, 48:545–558.

    Article  Google Scholar 

  • Lindley, D. V. and Phillips, L. D. (1976). Inference for a Bernoulli process (A Bayesian view).The American Statistician, 30:112–119.

    MATH  MathSciNet  Google Scholar 

  • Lindsay, B. G. (1994). Efficiency versus robustness: The case for minimum Hellinger distance and related methods.The Annals of Statistics, 22:1081–1114.

    Article  MATH  MathSciNet  Google Scholar 

  • Little, R. J. A. and Rubin, D. B. (1987).Statistical Analysis with Missing Data. J. Wiley & Sons.

    Google Scholar 

  • Makelainen, T., Schmidt, K., and Styan, G. P. H. (1981). On the existence and uniqueness of the maximum-likelihood estimate of a vector-valued parameter in fixed-size samples.Annals of Statistics, 9:758–767.

    Article  MathSciNet  Google Scholar 

  • McCullagh, P. (1997). Linear models, vector spaces, and residual likelihood. In Gregoire, T. G., Brillinger, D. R., Diggle, P. J., Russek-Cohen, E., Warren, W. G., and Wolfinger, R. D., editors,Modelling Longitudinal and Spatially Correlated Data: Methods, Applications, and Future Directions. Lecture Notes in Statistics, Vol. 122, pages 1–10. Springer-Verlag Inc.

    Google Scholar 

  • McLachlan, G.J. & Krishnan, T. (1997).The EM Algorithm and Extensions. J. Wiley & Sons.

    Google Scholar 

  • Messig, M. A. and Strawderman, W. E. (1993). Minimal sufficiency and completeness for dichotomous quantal response models.The Annals of Statistics, 21:2149–2157.

    Article  MATH  MathSciNet  Google Scholar 

  • Monahan, J. F. (2001).Numerical Methods of Statistics. Cambridge University Press.

    Google Scholar 

  • Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models.Journal of the Royal Statistical Society, Series A: General, 135:370–384.

    Article  Google Scholar 

  • Neyman, J. and Scott, E. L. (1948). Consistent estimates based on partially consistent observations.Econometrica, 16:1–32.

    Article  MathSciNet  Google Scholar 

  • Pace, L. and Salvan, A. (1997).Principles of Statistical Inference: from a Neo-Fisherian Perspective. World Scientific Publishing Company.

    Google Scholar 

  • Pirazzoli, P. A. (1982). Maree estreme a venezia (periodo 1872-1981).Acqua Aria, 10:1023–1029.

    Google Scholar 

  • Pitman, E. J. G. (1936). Sufficient statistics and intrinsic accuracy.Proc. Camb. Phil. Soc., 32:567–579.

    Article  Google Scholar 

  • Quesenberry, C. P. (1975). Transforming samples from truncation parameter distributions to uniformity.Communications in Statistics, 4:1149–1156.

    Article  MathSciNet  Google Scholar 

  • Read, T. R. C. and Cressie, N. A. C. (1988).Goodness-of-fit Statistics for Discrete Multivariate Data. Springer-Verlag Inc.

    Google Scholar 

  • Sampson, A. R. (1974). A tale of two regressions.Journal of the American Statistical Association, 69:682–689.

    Article  MATH  MathSciNet  Google Scholar 

  • Siegel, A. F. (1985). Modelling data containing exact zeroes using zero degrees of freedom.Journal of the Royal Statistical Society, Series B: Methodological, 47:267–271.

    MATH  MathSciNet  Google Scholar 

  • Simpson, D. G. (1987). Minimum Hellinger distance estimation for the analysis of count data.Journal of the American Statistical Association, 82:802–807.

    Article  MATH  MathSciNet  Google Scholar 

  • Stacy, E. W. (1962). A generalization of the gamma distribution.The Annals of Mathematical Statistics, 33:1187–1191.

    Article  MATH  MathSciNet  Google Scholar 

  • Stefanski, L. A. and Carroll, R. J. (1987). Conditional scores and optimal scores for generalized linear measurement-error models.Biometrika, 74:703–716.

    MATH  MathSciNet  Google Scholar 

  • Tobin, J. (1958). Estimation of relationships for limited dependent-variables.Econometrica, 26:24–36.

    Article  MATH  MathSciNet  Google Scholar 

  • Tsiatis, A. A. and Davidian, M. (2001). A semiparametric estimator for the proportional hazards model with longitudinal covariates measured with error.Biometrika, 88:447–458.

    Article  MATH  MathSciNet  Google Scholar 

  • Weir, B. S. (1996).Genetic Data Analysis 2: Methods for Discrete Population Genetic Data. Sunderland: Sinauer Associates.

    Google Scholar 

  • Wu, C. F. J. (1983). On the convergence properties of the EM algorithm.The Annals of Statistics, 11:95–103.

    Article  MATH  MathSciNet  Google Scholar 

  • Zehna, P. W. (1966). Invariance of maximum likelihood estimations.The Annals of Mathematical Statistics, 37:744–744.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA

    Denni D Boos & L A Stefanski

Authors
  1. Denni D Boos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L A Stefanski
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Boos, D.D., Stefanski, L.A. (2013). Likelihood Construction and Estimation. In: Essential Statistical Inference. Springer Texts in Statistics, vol 120. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4818-1_2

Download citation

Publish with us

Policies and ethics

Search

Navigation