High Order Boltzmann Machines with continuous units: Some experimental results

  • M. Graña
  • A. D'Anjou
  • F. X. Albizuri
  • A. de la Hera
  • I. Garci
Computational Models of Neurons and Neural Nets
Part of the Lecture Notes in Computer Science book series (LNCS, volume 930)


This work reports the results obtained with the application of High Order Boltzmann Machines without hidden units to classification problems with continuous features. The Boltzmann Machine weight updating algorithm is easily generalised when some of the units can take values inside a continuous interval. The absence of hidden units and the restriction to classification problems allows for the estimation of the connection statistics, without the computational cost involved in the application of simulated annealing. In this setting, the learning process can be speed up several orders of magnitude with no appreciable loss of quality of the results obtained.


Simulated Annealing Classification Problem Input Feature Hide Unit Output Unit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Aarts E.H.L., J.H.M. Korst “Simulated Annealing and Boltzmann Machines: a stochastic approach to combinatorial optimization and neural computing” John Wiley & Sons (1989)Google Scholar
  2. [2]
    Ackley D.H., G.E. Hinton, T.J. Sejnowski “A learning algorithm for Boltzmann Machines” Cogn. Sci. 9 (1985) pp. 147–169Google Scholar
  3. [3]
    Albizuri F.X., A. D'Anjou, M. Graña, F.J. Torrealdea, M.C. Hernandez “The High Order Boltzmann Machine: learned distribution and topology” IEEE Trans. Neural Networks en prensaGoogle Scholar
  4. [8]
    Deterding D. H. (1989) “Speaker Normalisation for Automatic Speech Recognition”, PhD Thesis University of Cambridge.Google Scholar
  5. [10]
    Gorman, R. P., and Sejnowski, T. J. (1988). “Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets” in Neural Networks, Vol. 1, pp. 75–89.Google Scholar
  6. [11]
    Graña M., V. Lavin, A. D'Anjou, F.X. Albizuri, J.A. Lozano “High-order Boltzmann Machines applied to the Monk's problems” ESSAN'94, DFacto press, Brussels, Belgium, pp117–122Google Scholar
  7. [16]
    Perantonis S.J., P.J.G. Lisboa “Translation, rotation and scale invariant pattern recognition by high-order neural networks and moment classifiers” IEEE Trans. Neural Net. 3(2) pp.241–251Google Scholar
  8. [17]
    Pinkas G. “Energy Minimization and Satisfiability of Propositional Logic” en Touretzky, Elman, Sejnowski, Hinton (eds) 1990. Connectionist Models Summer SchoolGoogle Scholar
  9. [18]
    Robinson A. J. (1989) “Dynamic Error Propagation Networks”.PhD Thesis Cambridge University Engineering Department.Google Scholar
  10. [19]
    Robinson A. J., F. Fallside (1988), “A Dynamic Connectionist Model for Phoneme Recognition” Proceedings of nEuro'88, Paris, June.Google Scholar
  11. [20]
    Sejnowski TJ. “Higher order Boltzmann Machines” in Denker (ed) Neural Networks for computing AIP conf. Proc. 151, Snowbird UT (1986) pp.398–403Google Scholar
  12. [24]
    R. Durbin, D.E. Rumelhart “Product Units: A computationally powerful and biologically plausible extension to backpropagation networks” Neural Computation 1pp133–142Google Scholar
  13. [25]
    M. Graña, A. D'Anjou, F.X. Albizuri, J.A. Lozano, P. Larrañaga, Y. Yurramendi, M. Hernandez, J.L. Jimenez, F.J. Torrealdea, M. Poza “Experimentos de aprendizaje con Máquinas de Boltzmann de alto orden“ submitted to Informática y AutomáticaGoogle Scholar
  14. [27]
    Lenze B. “How to make sigma-pi neural networks perform perfectly on regular training sets” Neural Networks 7(8) pp. 1285–1293Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • M. Graña
    • 1
  • A. D'Anjou
    • 1
  • F. X. Albizuri
    • 1
  • A. de la Hera
    • 1
  • I. Garci
    • 1
  1. 1.Dept. CCIAUPV/EHUSan Sebastián

Personalised recommendations