Advertisement

Neural Network Modelling with Input Uncertainty: Theory and Application

  • W.A. Wright
  • G. Ramage
  • D. Cornford
  • I.T. Nabney
Article

Abstract

It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which accounts for input noise provided that a model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method adds an extra term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable, and sampling this jointly with the network's weights, using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input. This leads to the possibility of training an accurate model of a system using less accurate, or more uncertain, data. This is demonstrated on both the, synthetic, noisy sine wave problem and a real problem of inferring the forward model for a satellite radar backscatter system used to predict sea surface wind vectors.

Keywords

Markov Chain Monte Carlo Neural Network Modelling Forward Model Noise Process Wind Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    D.J.C. MacKay, Bayesian Methods for Adaptive Models, PhD Thesis, California Institute of Technology, 1991.Google Scholar
  2. 2.
    J. Sietsma and R.J.F. Dow, “Creating Artificial Neural Networks the Generalize,” Neural Networks, vol. 4, no.1, 1991, pp. 67–80.CrossRefGoogle Scholar
  3. 3.
    K. Matsuoka, “Noise Injection into Inputs in Back-Propagation Learning,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 22, no.3, 1992, pp. 436–440.CrossRefGoogle Scholar
  4. 4.
    C.M. Bishop, “Training with Noise is Equivalent to Tikhonov Regularization,” Neural Computation, vol. 7, no.1, 1995, pp. 1085–1095.Google Scholar
  5. 5.
    V. Tresp, S. Ahamad, and R. Neuneier, “Training Neural Networks with Deficient Data,” in Neural Information Processing Systems, vol. 6, 1994, pp. 128–135.Google Scholar
  6. 6.
    N.W. Townsend and L. Tarassenko, “Estimations of Error Bounds for RBF networks,” in IEE Artificial Neural Networks, 1997, pp. 227–232.Google Scholar
  7. 7.
    D.J.C. MacKay, “Bayesian Interpolation,” Neural Computation, vol. 4, no.3, 1992, pp. 415–447.CrossRefGoogle Scholar
  8. 8.
    R.M. Neal, Bayesian Learning for Neural Networks, Springer, 1996. Lecture Notes in Statistics 118.Google Scholar
  9. 9.
    S.A. Dellaporta and P. Stephens, “Bayesian Analysis of Error in Variable Regression Models,” Biometrics, vol. 51, 1995, pp. 1085–1095.CrossRefGoogle Scholar
  10. 10.
    N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller, “Equation of State Calculations by Fast Computing Machines,” Journal of Chemical Physics, vol. 21, no.6, 1953, pp. 1087–1092.CrossRefGoogle Scholar
  11. 11.
    A. Gelman, Markov Chain Monte Carlo in Practice, Interdisiplinary Statistics, Chapman and Hall, 1997, ch. 8, pp. 131–143.Google Scholar
  12. 12.
    I.T. Nabney, D. Cornford, and C.K.I Williams, “Bayesian Inference for Wind Field Retrieval,” Neurocomputing Letters, vol. 27, 1999, pp. 1013–1018.Google Scholar
  13. 13.
    D. Offiler, “The Calibration of ERS-1 Satellite Scatterometer Winds,” Journal of Atmospheric and Oceanic Technology, vol. 11, 1994, pp. 1002–1017.CrossRefGoogle Scholar
  14. 14.
    A. Stoffelen and D. Anderson, “Scatterometer Data Interpretation: Estimation and Validation of the Transfer Function CMOD4,” Journal of Geophisical Research, vol. 102, 1997, pp. 5767–5780.CrossRefGoogle Scholar
  15. 15.
    G. Ramage, Neural Networks for Modelling Wind Vectors, Master's Thesis, Aston University, 1998.Google Scholar
  16. 16.
    C. Bishop. Neural Networks for Pattern Recognition, Oxford, UK: Oxford University, 1995.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • W.A. Wright
    • 1
  • G. Ramage
    • 2
  • D. Cornford
    • 2
  • I.T. Nabney
    • 2
  1. 1.Sowerby Research CentreBAE SYSTEMSBristolEngland
  2. 2.Neural Computing Research GroupAston UniversityBirminghamEngland

Personalised recommendations