Parallel Coarse Grain Computing of Boltzmann Machines
The resolution of combinatorial optimization problems can greatly benefit from the parallel and distributed processing which is characteristic of neural network paradigms. Nevertheless, the fine grain parallelism of the usual neural models cannot be implemented in an entirely efficient way either in general-purpose multicomputers or in networks of computers, which are nowadays the most common parallel computer architectures. Therefore, we present a parallel implementation of a modified Boltzmann machine where the neurons are distributed among the processors of the multicomputer, which asynchronously compute the evolution of their subset of neurons using values for the other neurons that might not be updated, thus reducing the communication requirements. Several alternatives to allow the processors to work cooperatively are analyzed and their performance detailed. Among the proposed schemes, we have identified one that allows the corresponding Boltzmann Machine to converge to solutions with high quality and which provides a high acceleration over the execution of the Boltzmann machine in uniprocessor computers.
Unable to display preview. Download preview PDF.
- 1.E.H.L. Aarts and J.H.M. Korst, Simulated Annealing and Boltzmann Machines, New York: Wiley, 1988.Google Scholar
- 2.D.H. Oh, J.H. Nang, H. Yoon and S.R. Maeng, “An efficient mapping of Boltzmann Machine computations onto distributed-memory multiprocessors”, Microprocessing and Microprogramming, 33, pp. 223-236, 1991/92.Google Scholar
- 3.A. De Gloria, P. Faraboschi and S. Ridella, “A dedicated Massively Parallel Architecture for the Boltzmann Machine”, Parallel Comp., Vol. 18, No. 1, pp. 57-75, 1993.Google Scholar
- 4.A. De Gloria and M. Olivieri, “An asynchronous distributed architecture model for the Boltzmann machine control mechanism”, IEEE Trans. on Neural Networks, Vol. 7, No. 6., pp. 1538-1541. November, 1996.Google Scholar
- 5.O. Martin, S.W. Otto and E.W. Felten, “Large-step Markov chains for the TSP incorporation local search heuristics”, Operations Research Letters, 11, pp. 219-224. May, 1992.Google Scholar
- 6.H.R. Lourenço, “Job-shop scheduling: Computational study of local search and large-step optimization methods”, European J. of Operation Research, 83, pp. 347-364, 1995.Google Scholar
- 7.M. Livesey, “Clamping in Boltzmann machines”, IEEE Trans. on Neural Networks, Vol. 2, No. 1, pp. 143-148. January, 1991.Google Scholar
- 8.C.-E. Hong and B.M. McMillin, “Relaxing Synchronization in Distributed Simulated Annealing”, IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 2, pp. 189-195, February, 1995.Google Scholar
- 9.V. Zissimopoulos, V.T. Paschos and F. Pekergin, “On the approximation of NP-complete problems by using the Boltzmann Machine method: The cases of some covering and packing problems”, IEEE Trans. on Computers, Vol. 40, No. 12, pp. 1413-1418, December, 1991.Google Scholar
- 10.I. Pramanick and J.G. Kuhl, “An inherently Parallel Method for Heuristic Problem-Solving: Part I - General Framework”, IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 10, pp. 1006-1015, October, 1995.Google Scholar
- 11.S.-Y. Lee and K.G. Lee, “Synchronous and asynchronous parallel simulated annealing with multiple Markov chains”, IEEE Trans. on Parallel and Distributed Systems, Vol. 7, No. 10, pp. 993-1007, October, 1996.Google Scholar