Skip to main content
SpringerLink
Log in
Book cover

International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems

IPMU 2016: Information Processing and Management of Uncertainty in Knowledge-Based Systems pp 279–290Cite as

Belief Revision and the EM Algorithm

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 611))

Abstract

This paper provides a natural interpretation of the EM algorithm as a succession of revision steps that try to find a probability distribution in a parametric family of models in agreement with frequentist observations over a partition of a domain. Each step of the algorithm corresponds to a revision operation that respects a form of minimal change. In particular, the so-called expectation step actually applies Jeffrey’s revision rule to the current best parametric model so as to respect the frequencies in the available data. We also indicate that in the presence of incomplete data, one must be careful in the definition of the likelihood function in the maximization step, which may differ according to whether one is interested by the precise modeling of the underlying random phenomenon together with the imperfect observation process, or by the modeling of the underlying random phenomenon alone, despite imprecision.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In some cases, they are not artificial, and are naturally present in the problem.

  2. 2.

    In the expression in line 2 of the above derivation, \( F(\mathbf P,\theta )\) could be, with some abuse of notation, written \( -D(\mathbf P,\mathbf P(\cdot ,\mathbf{y};\theta ))\) as it is a kind of divergence from \(\mathbf P(\cdot ,\mathbf{y};\theta ))\). However the sum on \(\mathcal {X}^N\) of the latter quantities is not 1 (it is \(\mathbf{p}(\mathbf{y};\theta )\)) and this pseudo-divergence can be negative.

References

  1. Domotor, Z.: Probability kinematics - conditional and entropy principles. Synthese 63, 74–115 (1985)

    Article  MathSciNet  Google Scholar 

  2. Couso, I., Dubois, D.: Statistical reasoning with set-valued information: ontic vs. epistemic views. Int. J. Approximate Reasoning 55(7), 1502–1518 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Dempster, A.P.: Upper and lower probabilities induced by a multivalued mapping. Ann. Math. Stat. 38, 325–339 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  4. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Stat. Soc. B 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  5. Do, C.B., Batzoglou, S.: What is the expectation maximization algorithm? Nat. Biotechnol. 26(8), 897–899 (2009)

    Article  MathSciNet  Google Scholar 

  6. Edwards, A.W.F.: Likelihood. Cambridge University Press, Cambridge (1972)

    MATH  Google Scholar 

  7. Guillaume, R., Dubois, D.: Robust parameter estimation of density functions under fuzzy interval observations. In: 9th ISIPTA Symposium, Pescara, Italy, pp. 147–156 (2015)

    Google Scholar 

  8. Haas, S.E.: The Expectation-Maximization and Alternating Minimization Algorithms. MIT, Cambridge (2002)

    Google Scholar 

  9. Hüllermeier, E.: Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization. Int. J. Approximate Reasoning 55(7), 1519–1534 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hüllermeier, E., Cheng, W.: Superset learning based on generalized loss minimization. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds.) ECML PKDD 2015. LNCS, vol. 9285, pp. 260–275. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  11. Jeffrey, R.C.: The Logic of Decision. McGraw-Hill, New York (1965), 2nd edn.: University of Chicago Press, Chicago, (1983). Paperback Edition (1990)

    Google Scholar 

  12. McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (2007)

    MATH  Google Scholar 

  13. Russell, S.: The EM algorithm, Lecture notes CS 281, Computer Science Department University of California, Berkeley (1998)

    Google Scholar 

  14. Schafer, J.L.: Multiple imputation: a primer. Stat. Methods Med. Res. 8, 3–15 (1999)

    Article  Google Scholar 

  15. Ting, A., Lin, H.: Comparison of multiple imputation with EM algorithm and MCMC method for quality of life missing data. Qual. Quant. 44, 277–287 (2010)

    Article  Google Scholar 

  16. Williams, P.: Bayesian conditionalization and the principle of minimum information. Brit. J. Philos. Sci. 31, 131–144 (1980)

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by ANR-11-LABX-0040-CIMI (Centre International de Mathématiques et d’Informatique) within the program ANR-11-IDEX-0002-02, while Inés Couso was a visiting expert scientist at CIMI, Toulouse, and by TIN2014-56967-R (Spanish Ministry of Science and Innovation) and FC-15-GRUPIN14-073 (Regional Ministry of the Principality of Asturias).

Author information

Authors and Affiliations

  1. Universidad Oviedo, Gijon, Spain

    Inés Couso

  2. IRIT, CNRS and University of Toulouse, Toulouse, France

    Didier Dubois

Authors
  1. Inés Couso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Didier Dubois
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Inés Couso .

Editor information

Editors and Affiliations

  1. INESC-ID,Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal

    Joao Paulo Carvalho

  2. LIP 6, Université Pierre et Marie Curie, Paris, France

    Marie-Jeanne Lesot

  3. School of Industrial Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

    Uzay Kaymak

  4. IDMEC,Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal

    Susana Vieira

  5. LIP6, Université Pierre et Marie Curie, CNRS, Paris, France

    Bernadette Bouchon-Meunier

  6. Iona College, Machine Intelligence Institute, New Rochelle, New York, USA

    Ronald R. Yager

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Couso, I., Dubois, D. (2016). Belief Revision and the EM Algorithm. In: Carvalho, J., Lesot, MJ., Kaymak, U., Vieira, S., Bouchon-Meunier, B., Yager, R. (eds) Information Processing and Management of Uncertainty in Knowledge-Based Systems. IPMU 2016. Communications in Computer and Information Science, vol 611. Springer, Cham. https://doi.org/10.1007/978-3-319-40581-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-40581-0_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-40580-3

  • Online ISBN: 978-3-319-40581-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation