Making stochastic networks deterministic

  • Stefan M. Rüger
Part III: Learning: Theory and Algorithms
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1327)


Graphical models are considered more and more as a key technique for describing the dependency relations of random variables. Various learning and inference algorithms have been described and analysed. This article demonstrates how an important subclass of graphical models can be treated by transforming the underlying model into a regular feedforward network with special, yet deterministic, activation functions. Inference and the relevant quantities for learning can be calculated exactly in these networks. Moreover, all the known techniques for feedforward networks can be exploited and applied here.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. F. Cooper. The computational complexity of probabilistic inference using Bayesian belief networks. Artificial Intelligence, 42:393–405, 1990.Google Scholar
  2. 2.
    D. J. C. MacKay. Equivalence of Boltzmann chains and hidden Markov models. Neural Computation, 8(1):178–181, 1996.Google Scholar
  3. 3.
    M. J. Nijman and H. J. Kappen. Efficient learning in sparsely connected Boltzmann machines. In C. von der Malsburg, W. von Seelen, J. C. Vorbriiggen and B. Sendhoff, editors, Artificial Neural Networks — ICANN 96, pages 41–46. Springer-Verlag, 1996.Google Scholar
  4. 4.
    W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery. Numerical Recipes in C. Cambridge University Press, 1988.Google Scholar
  5. 5.
    S. M. Rüger. Stable dynamic parameter adaptation. In D. Touretzky, M. Mozer and M. Hasselmo, editors, Advances in Neural Information Processing Systems 8, pages 225–231. MIT Press, 1996.Google Scholar
  6. 6.
    S. M. Rüger. Decimatable boltzmann machines for diagnosis: Efficient learning and inference. In Proceedings of IMACS'97, Berlin (accepted), 1997.Google Scholar
  7. 7.
    S. M. Rüger. Zur Theorie künstlicher neuronaler Netze. Verlag Harri Deutsch, Thun, Frankfurt/Main, 1997.Google Scholar
  8. 8.
    L. K. Saul and M. I. Jordan. Boltzmann chains and hidden Markov models. In G. Tesauro, D. S. Touretzky and T. K. Leen, editors, Advances in Neural Information Processing Systems 7, pages 435–442. MIT Press, 1995.Google Scholar
  9. 9.
    P. Smyth, D. Heckerman and M. I. Jordan. Probabilistic independence networks for hidden Markov probability models. Neural Computation, 9(2), 20–27. 1997.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Stefan M. Rüger
    • 1
  1. 1.Department of ComputingImperial College of Science, Technology and MedicineLondonEngland

Personalised recommendations