Adaptive cooperation between processors in a parallel Boltzmann machine implementation
The fine grain, data-driven parallelism shown by neural models as the Boltzmann machine cannot be implemented in an entirely efficient way either in general-purpose multicomputers or in networks of computers, which are nowadays the most common parallel computer architectures.
In this paper we present a parallel implementation of a modified Boltzmann machine where the processors, with disjoint subsets of neurons allocated, asynchronously compute the evolution of their neurons by using values that might not be updated for the remaining neurons, thus reducing interprocessor communication requirements. An evolutionary algorithm is used to learn the rules that allow the processors to cooperate by interchanging the local optima that they find while concurrently exploring different zones of the Boltzmann machine state space. Thus, the way the processors interact changes dynamically during execution of the algorithm, adapted to the problem at hand. Good figures for speedup with respect to the Boltzmann machine computation in a uniprocessor computer have been experimentally obtained.
Key wordsBoltzmann machines combinatorial optimization evolutionary computation multicomputers and networks of computers parallel processing
Unable to display preview. Download preview PDF.
- Anderson, T.E.; Culler, T.E.; Patterson, D.A.; and the NOW team: “A Case for NOW (Networks of Workstations”. IEEE Micro, pp. 54–64. February, 1995.Google Scholar
- Aarts, E.H.L.; Korst, J.H.M.: “Simulated Annealing and Boltzmann Machines”. Wiley, 1988.Google Scholar
- De Gloria, A.; Faraboschi, P.; Ridella, S.: “A dedicated Massively Parallel Architecture for the Boltzmann Machine”. Parallel Comp., Vol. 18, No. 1, pp. 57–75, 1993.Google Scholar
- Potter, M.A.; De Jong, K.A.: “A Cooperative Coevolutionary Aproach to Function Optimization”. In Third Conference on Parallel Problem Solving from Nature, Y. Davidor and H.P. Schwefel (Eds.), Lecture Notes in Computer Science, Vol. 866, Springer-Verlag, pp. 249–257, 1994.Google Scholar