A Convolutional Neural Network Tolerant of Synaptic Faults for Low-Power Analog Hardware
Recently, the authors described a training method for a convolutional neural network of threshold neurons. Hidden layers are trained by by clustering, in a feed-forward manner, while the output layer is trained using the supervised Perceptron rule. The system is designed for implementation on an existing low-power analog hardware architecture, exhibiting inherent error sources affecting the computation accuracy in unspecified ways. One key technique is to train the network on-chip, taking possible errors into account without any need to quantify them. For the hidden layers, an on-chip approach has been applied previously. In the present work, a chip-in-the-loop version of the iterative Perceptron rule is introduced for training the output layer. Influences of various types of errors are thoroughly investigated (noisy, deleted, and clamped weights) for all network layers, using the MNIST database of hand-written digits as a benchmark.
KeywordsHide Layer Output Layer Training Image Convolutional Neural Network Threshold Neuron
Unable to display preview. Download preview PDF.
- 1.Fieres, J., Grubl, A., Philipp, S., Meier, K., Schemmel, J., Schürmann, F.: A platform for parallel operation of VLSI neural networks. In: Conference on Brain Inspired Cognitive Systems (BICS 2004), Stirling, Scotland (2004)Google Scholar
- 2.Fieres, J., Schemmel, J., Meier, K.: Training convolutional neural networks of threshold neurons suited for low-power hardware implementation. In: Int. Joint Conference on Neural Networks (IJCNN 2006), Vancouver, CA (accepted, 2006)Google Scholar
- 4.Hohmann, S.G., Fieres, J., Meier, K., Schemmel, J., Schmittz, T., Schürmann, F.: Training Fast Mixed-Signal Neural Networks for Data Classification. In: Proceedings of the 2004 International Joint Conference on Neural Networks (IJCNN 2004), pp. 2647–2652. IEEE Press, Los Alamitos (2004)Google Scholar
- 5.Jang, J.-S.R., Sun, C.-T., Mizutani, E.: Neuro-Fuzzy and Soft Computing. Prentice-Hall, Englewood Cliffs (1997)Google Scholar
- 7.LeCun, Y., Jackel, L.D., Boser, B., Denker, J.S., Graf, H.P., Guyon, I., Henderson, D., Howard, R.E., Hubbard, W.: Handwritten digit recognition: Applications of neural net chips and automatic learning. IEEE Communications Magazine, 41–46 (1989)Google Scholar
- 9.LeCun, Y.: The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist
- 16.Simard, P.Y., Steinkraus, D., Platt, J.C.: Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis. In: Intl. Conf. Document Analysis and Recognition, pp. 958–962 (2003)Google Scholar