Optical Character Recognition and Neural-Net Chips

Abstract

Neural Network research has always interested hardware designers, theoreticians, and application engineers. But until recently, the common ground between these groups was limited: the neural-net chips were too small to implement any full-size application, and the algorithms were too complicated (or the applications not interesting enough) to be implemented on a chip. The merging of these efforts is now made possible by the simultaneous emergence of powerful chips and successful, real-world applications of neural networks. Here, we discuss how the compute-intensive part of a handwritten digit recognizer, based on a highly structured backpropagation network, can be implemented on a general purpose neural-network chip containing 32k binary synapses. Using techniques based on the second-order properties of the error function, we show that very little accuracy on the weights and states is required in the first layers of the network. Interestingly, the best digit-recognition network is also the easiest to implement on a chip.