Three Conjectures on Neural Network Implementations of Volterra Models (Mappings)
Three conjectures are presented that form the methodological link between Volterra models (mappings) and a popular class of artificial neural networks (multi-layer perceptrons). The first conjecture elucidates the equivalence between these two types of nonlinear mapping and shows how we can achieve a network implementation of a Volterra model by employing proper linear input transformations and polynomial activation functions in the hidden units; while the output unit(s) may be simple adder(s). The second conjecture outlines the trade-offs between general polynomial and fixed sigmoidal activation functions traditionally used in multilayer perceptrons. The former are more flexible in defining nonlinear mappings; while thelatter, being far more restrictive, lead to increased numbers of hidden units and heavier computational burden during training via back-propagation. In general, an infinite number of sigmoidal hidden units is required to represent exactly a Volterra model (mapping) or a network with (finite) polynomial hidden units. The third conjecture extends the results for continous-output models to binary-(or spike-)output models/mappings, often encountered in neural networks. These conjectures collectively point to the potential versatility and efficiency of a class of networks that utilize polynomial activation functions in the hidden units and linear output unit(s) with fixed weights. Practical procedures for optimal use of these networks are currently developed.
KeywordsInput Vector Hide Unit Volterra Model Trigger Zone Hard Threshold
Unable to display preview. Download preview PDF.
- 11.Rumelhart, D.E. and McClelland, J.L. (1986) (Eds.) Parallel Distributed Processing, Vol. I. and II., MIT Press, Cambridge, Massachussetts.Google Scholar