Advertisement

Adaptive cooperation between processors in a parallel Boltzmann machine implementation

  • J. Ortega
  • L. Parrilla
  • J. L. Bernier
  • C. Gil
  • B. Pino
  • M. Anguita
Bio-inspired Systems
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1607)

Abstract

The fine grain, data-driven parallelism shown by neural models as the Boltzmann machine cannot be implemented in an entirely efficient way either in general-purpose multicomputers or in networks of computers, which are nowadays the most common parallel computer architectures.

In this paper we present a parallel implementation of a modified Boltzmann machine where the processors, with disjoint subsets of neurons allocated, asynchronously compute the evolution of their neurons by using values that might not be updated for the remaining neurons, thus reducing interprocessor communication requirements. An evolutionary algorithm is used to learn the rules that allow the processors to cooperate by interchanging the local optima that they find while concurrently exploring different zones of the Boltzmann machine state space. Thus, the way the processors interact changes dynamically during execution of the algorithm, adapted to the problem at hand. Good figures for speedup with respect to the Boltzmann machine computation in a uniprocessor computer have been experimentally obtained.

Key words

Boltzmann machines combinatorial optimization evolutionary computation multicomputers and networks of computers parallel processing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Anderson, T.E.; Culler, T.E.; Patterson, D.A.; and the NOW team: “A Case for NOW (Networks of Workstations”. IEEE Micro, pp. 54–64. February, 1995.Google Scholar
  2. [2]
    Wolpert, D.H.; Macready, W.G.: “No Free Lunch Theorems for Optimization”. IEEE Trans. on Evolutionary Computation, Vol. 1, No. 1, pp. 67–82. April, 1997.CrossRefGoogle Scholar
  3. [3]
    Aarts, E.H.L.; Korst, J.H.M.: “Simulated Annealing and Boltzmann Machines”. Wiley, 1988.Google Scholar
  4. [4]
    Oh, D.H.; Nang, J.H.; Yoon, H.; Maeng, S.R.: “An efficient mapping of Boltzmann Machine computations onto distributed-memory multiprocessors”. Microprocessing and Microprogramming, Vol. 33, pp. 223–236, 1991/92.CrossRefGoogle Scholar
  5. [5]
    De Gloria, A.; Faraboschi, P.; Ridella, S.: “A dedicated Massively Parallel Architecture for the Boltzmann Machine”. Parallel Comp., Vol. 18, No. 1, pp. 57–75, 1993.Google Scholar
  6. [6]
    Ortega, J.; Rojas, I.; Diaz, A.F.; Prieto, A.: “Parallel Coarse Grain Computing of Boltzmann Machines”. Neural Processing Letters, Vol. 7, No. 3, pp. 1–16, 1998.CrossRefGoogle Scholar
  7. [7]
    Hong, C.-E.; McMillin, B.M.: “Relaxing Synchronization in Distributed Simulated Annealing”. IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 2, pp. 189–195. Febuary, 1995.CrossRefGoogle Scholar
  8. [8]
    Pramanick, I.; Kuhl, J.G.: “An inherently Parallel Method for Heuristic Problem-Solving: Part I—General Framework”. IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 10, pp. 1006–1015. October, 1995.CrossRefGoogle Scholar
  9. [9]
    Zissimopoulos, V.; Paschos, V.T.; Pekergin, F.: “On the approximation of NP-complete problems by using the Boltzmann Machine method: The cases of some covering and packing problems”. IEEE Trans. on Computers, Vol. 40, No. 12, pp. 1413–1418. December, 1991.CrossRefMathSciNetGoogle Scholar
  10. [10]
    Potter, M.A.; De Jong, K.A.: “A Cooperative Coevolutionary Aproach to Function Optimization”. In Third Conference on Parallel Problem Solving from Nature, Y. Davidor and H.P. Schwefel (Eds.), Lecture Notes in Computer Science, Vol. 866, Springer-Verlag, pp. 249–257, 1994.Google Scholar
  11. [11]
    Martin, O.; Otto, S.W.; Felten, E.W.: “Large-step Markov chains for the TSP incorporation local search heuristics”. Operations Research Letters, 11, pp. 219–224. May 1992.MATHCrossRefMathSciNetGoogle Scholar
  12. [12]
    Lourenço, H.R.: “Job-shop scheduling: Computational study of local search and large-step optimization methods”. European J. of Operation Research, 83, pp. 347–364, 1995.MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • J. Ortega
    • 1
  • L. Parrilla
    • 1
  • J. L. Bernier
    • 1
  • C. Gil
    • 1
  • B. Pino
    • 1
  • M. Anguita
    • 1
  1. 1.Departamento de Arquitectura y Tecnología de ComputadoresUniversidad de GranadaGranada

Personalised recommendations