Abstract
Stochastic neural networks are usually built by introducing random fluctuations into the network. A natural method is to use stochastic connections rather than stochastic activation functions. We propose a new model in which each neuron has very simple functionality but all the connections are stochastic. It is shown that the stationary distribution of the network uniquely exists and it is approximately a Boltzmann-Gibbs distribution. The relationship between the model and the Markov random field is discussed. New techniques to implement simulated annealing and Boltzmann learning are proposed. Simulation results on the graph bisection problem and image recognition show that the network is powerful enough to solve real world problems.
Similar content being viewed by others
References
Rumelhart, D. E., McClelland, J. L. and the PDP Research Group, Parallel Distributed Processing, Cambridge: The MIT Press, 1986.
Poppelbaum, W. J., Dollas, A., Glickman, J. B. et al., Unary processing, Advances in Computers, 1987, 26: 47–92.
Zhao, J., Stochastic bit stream neural networks, PhD Thesis, London: London University, 1995.
Murray, A., Tarassenko, L., Analogue Neural VLSI-A Pulse Stream Approach, London: Chapman & Hall, 1994.
Kondo, Y., Sawada, Y., Functional abilities of a stochastic logic neural network, IEEE Trans. on Neural Networks, 1992, 3: 434–443.
Maass, W., Computation with spiking neurons, The Handbook of Brain Theory and Neural Networks, 2nd ed., Cambridge: MIT Press, 2001.
Shawe-Taylor, J., Jeavons, P., van Daalen, M., Probabilistic bit stream neural chip: Theory, Connection Science, 1991, 3 (3): 317–328.
Aarts E., Korst, J., Simulated Annealing and Boltzmann Machines, New York: John Wiley & Sons, 1989.
Chuang, C., Su, S., Hsiao, C., The annealing robust backpropagation (ARBP) learning algorithm, IEEE Trans. on Neural Networks, 2000, 11(5): 1067–1077.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Zhao, J. A recurrent stochastic binary network. Sci China Ser F 44, 376–388 (2001). https://doi.org/10.1007/BF02714740
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02714740