A Limited-Interconnect, Highly Layered Synthetic Neural Architecture

Abstract

Recent encouraging results have occurred in the application of neuromorphic, ie. neural network inspired, software simulations of speech synthesis, word recognition, and image processing. Hardware implementations of neuromorphic systems are required for real-time applications such as control and signal processing. Two disparate groups of workers are interested in VLSI hardware implementations of neural networks. The first is interested in electronic-based implementations of neural networks and use standard or custom VLSI chips for the design. The second group wants to build fault tolerant adaptive VLSI chips and are much less concerned with whether the design rigorously duplicates the neural models. In either case, the central issue in construction of a electronic neural network is that the design constraints of VLSI differ from those of biology (Walker and Akers 1988). In particular, the high fan-in/fan-outs of biology impose connectivity requirements such that the electronic implementation of a highly interconnected biological neural networks of just a few thousand neurons would require a level of connectivity which exceeds the current or even projected interconnection density of ULSI systems. Fortunately, highly-layered limited interconnected networks can be formed that are functionally equivalent to highly connected systems (Akers et al. 1988). Highly layered, limited-interconnected architectures are especially well suited for VLSI implementations. The objective of our work is to design highly layered, limited-interconnect synthetic neural architectures and develop training algorithms for systems made from these chips. These networks are specifically designed to scale to tens of thousands of processing elements on current production size dies.