Statistics and Computing

, Volume 10, Issue 1, pp 25–37

Bayesian parameter estimation via variational methods

Authors

  • Tommi S. Jaakkola
    • Dept. of Elec. Eng. & Computer ScienceMassachusetts Institute of Technology
  • Michael I. Jordan
    • Computer Science Division and Department of StatisticsUniversity of California
Article

DOI: 10.1023/A:1008932416310

Cite this article as:
Jaakkola, T.S. & Jordan, M.I. Statistics and Computing (2000) 10: 25. doi:10.1023/A:1008932416310

Abstract

We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that an accurate variational transformation can be used to obtain a closed form approximation to the posterior distribution of the parameters thereby yielding an approximate posterior predictive model. This approach is readily extended to binary graphical model with complete observations. For graphical models with incomplete observations we utilize an additional variational transformation and again obtain a closed form approximation to the posterior. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates.

logistic regressiongraphical modelsbelief networksvariational methodsBayesian estimationincomplete data

Copyright information

© Kluwer Academic Publishers 2000