Neural Network Modelling with Input Uncertainty: Theory and Application
- 206 Downloads
It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which accounts for input noise provided that a model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method adds an extra term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable, and sampling this jointly with the network's weights, using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input. This leads to the possibility of training an accurate model of a system using less accurate, or more uncertain, data. This is demonstrated on both the, synthetic, noisy sine wave problem and a real problem of inferring the forward model for a satellite radar backscatter system used to predict sea surface wind vectors.
KeywordsMarkov Chain Monte Carlo Neural Network Modelling Forward Model Noise Process Wind Vector
Unable to display preview. Download preview PDF.
- 1.D.J.C. MacKay, Bayesian Methods for Adaptive Models, PhD Thesis, California Institute of Technology, 1991.Google Scholar
- 4.C.M. Bishop, “Training with Noise is Equivalent to Tikhonov Regularization,” Neural Computation, vol. 7, no.1, 1995, pp. 1085–1095.Google Scholar
- 5.V. Tresp, S. Ahamad, and R. Neuneier, “Training Neural Networks with Deficient Data,” in Neural Information Processing Systems, vol. 6, 1994, pp. 128–135.Google Scholar
- 6.N.W. Townsend and L. Tarassenko, “Estimations of Error Bounds for RBF networks,” in IEE Artificial Neural Networks, 1997, pp. 227–232.Google Scholar
- 8.R.M. Neal, Bayesian Learning for Neural Networks, Springer, 1996. Lecture Notes in Statistics 118.Google Scholar
- 11.A. Gelman, Markov Chain Monte Carlo in Practice, Interdisiplinary Statistics, Chapman and Hall, 1997, ch. 8, pp. 131–143.Google Scholar
- 12.I.T. Nabney, D. Cornford, and C.K.I Williams, “Bayesian Inference for Wind Field Retrieval,” Neurocomputing Letters, vol. 27, 1999, pp. 1013–1018.Google Scholar
- 15.G. Ramage, Neural Networks for Modelling Wind Vectors, Master's Thesis, Aston University, 1998.Google Scholar
- 16.C. Bishop. Neural Networks for Pattern Recognition, Oxford, UK: Oxford University, 1995.Google Scholar