On the Relation between Volterra Models and Feedforward Artificial Neural Networks
This paper explores the basic relations between nonlinear Volterra models (VM) and a broadly used class of artificial neural networks (ANN) utilizing feedforward connections, in order to achieve possible cross-enhancements from combined use of the two approaches. For instance, the study of nonlinear physiological systems using Volterra models is often impeded by lack of data of sufficient length and/or spectral properties or by inability to estimate high-order kernels. These problems may be mitigated by the cooperative use of ANN models trained with the available data and allowing indirect estimation of system kernels of arbitrary order. On the other hand, the use of ANN has been hindered by lack of clear methodological guidance in selecting the number or type of “hidden units” (critical for determining the efficacy of the training process) or by inability to interpret the obtained results. The latter problem is especially important in applications requiring scientific interpretation of the results, or in cases where it is critical to understand why the obtained empirical models perform the desired tasks. Equivalent VM, obtained by the same da ta training the ANN, may assist in interpreting the corresponding ANN and, furthermore, guide in selecting the proper number and type of “hidden units” to optimize the training process. In deriving explicit relations between VM and ANN, the class of “polynomial ANN” was found particularly useful and efficient in training via back-propagation algorithms. The conditions under which the two approaches may yield equivalent representations of the input-output relation are discussed and illustrated with computer simulations. The feasibility of kernel estimation via equivalent ANN training using back-propagation is also demonstrated by computer-simulated examples. It is shown that the kernel estimation accuracy is comparable to the Laguerre expansion technique (Marmarelis, 1993a) and significantly better than the traditional crosscorrelation technique. Application of this approach to actual experimental data is currently under way.
KeywordsHide Unit Volterra Model Volterra System Volterra Kernel Feedforward Connection
Unable to display preview. Download preview PDF.
- 2.Marmarelis, V.Z. (ed.) (1987) Advanced Methods of Physiological System Modeling, Vol. I. University of Southern California, Biomedical Simulations Resource, Los Angeles, California.Google Scholar
- 6.Marmarelis, V.Z. (1993b) Relation between the Volterra-Wiener approach and a class of artificial neural networks. Proc. 15th Ann. Int. Conf. IEEE-EMBS, San Diego, California, pp. 244–245.Google Scholar
- 10.Wray, J. (1992) Theory and Applications of Neural Networks, Ph.D. Thesis. University of Newcastle upon Tyne, United Kingdom.Google Scholar
- 11.Wray, J. and Green, G.G.R. (1991) Analysis of networks that have learnt control problems. Proc. IEE Control-91. pp. 261–265, Herriot-Watt, United Kingdom.Google Scholar