Energy Complexity and Entropy of Threshold Circuits
Circuits composed of threshold gates (McCulloch-Pitts neurons, or perceptrons) are simplified models of neural circuits with the advantage that they are theoretically more tractable than their biological counterparts. However, when such threshold circuits are designed to perform a specific computational task they usually differ in one important respect from computations in the brain: they require very high activity. On average every second threshold gate fires (sets a “1” as output) during a computation. By contrast, the activity of neurons in the brain is much more sparse, with only about 1% of neurons firing. This mismatch between threshold and neuronal circuits is due to the particular complexity measures (circuit size and circuit depth) that have been minimized in previous threshold circuit constructions. In this article we investigate a new complexity measure for threshold circuits, energy complexity, whose minimization yields computations with sparse activity. We prove that all computations by threshold circuits of polynomial size with entropy O(logn) can be restructured so that their energy complexity is reduced to a level near the entropy of circuit states. This entropy of circuit states is a novel circuit complexity measure, which is of interest not only in the context of threshold circuits, but for circuit complexity in general. As an example of how this measure can be applied we show that any polynomial size threshold circuit with entropy O(logn) can be simulated by a polynomial size threshold circuit of depth 3.
KeywordsBoolean Function Binary Tree Internal Node Complexity Measure Binary String
Unable to display preview. Download preview PDF.
- 2.Gröger, H.D., Turán, G.: On linear decision trees computing Boolean functions. In: Leach Albert, J., Monien, B., Rodríguez-Artalejo, M. (eds.) ICALP 1991. LNCS, vol. 510, pp. 707–718. Springer, Heidelberg (1991)Google Scholar
- 4.0. M. Kasim-Zade, On a measure of the activeness of circuits made of functional elements (Russian). Mathematical problems in cybernetics, 4:218–228, Nauka, Moscow, see Math. Reviews MR1217502 (94c:94019) (1992)Google Scholar
- 11.Reif, J.H., Tyagi, A.: Energy complexity of optical computations. In: Proceedings of the Second IEEE Symposium on Parallel and Distributed Processing (December 1990), pp. 14–21 (1990)Google Scholar
- 14.Weinzweig, M.N.: On the power of networks of functional elements. Dokl.Akad.Nauk SSSR 139, 545–547 (1961) (Russian); in English: Sov.Phys.Dokl, 6, 545–547, see Math. Reviews MR0134413 (24 #B466) Google Scholar