Advertisement

Hessian Corrected Input Noise Models

  • Botond Attila Bócsi
  • Lehel Csató
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8131)

Abstract

When the inputs of a regression problem are corrupted with noise, integrating out the noise process leads to biased estimates. We present a method that corrects the bias caused by the integration. The correction is proportional to the Hessian of the learned model and to the variance of the input noise. The method works for arbitrary regression models, the only requirement is two times differentiability of the respective model. The conducted experiments suggest that significant improvement can be gained using the proposed method. Nevertheless, experiments on high dimensional data highlight the limitations of the algorithm.

Keywords

Mean Square Error High Dimensional Data Input Noise Output Noise Gaussian Process Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bishop, C.M.: Training with noise is equivalent to Tikhonov regularization. Neural Computation 7(1), 108–116 (1995)CrossRefGoogle Scholar
  2. 2.
    Dallaire, P., Besse, C., Chaib-draa, B.: An approximate inference with Gaussian process to latent functions from uncertain data. Neuroc. 74(11), 1945–1955 (2011)CrossRefGoogle Scholar
  3. 3.
    Evans, L.: Partial Differential Equations. Graduate Studies in Mathematics. American Mathematical Society (2010)Google Scholar
  4. 4.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
  5. 5.
    Girard, A., Murray-Smith, R.: Learning a Gaussian process model with uncertain inputs. Tech. rep., University of Glasgow, Department of Computing Science (2003)Google Scholar
  6. 6.
    Goldberg, P.W., Williams, C.K.I., Bishop, C.M.: Regression with input-dependent noise: A Gaussian process treatment. In: NIPS (1997)Google Scholar
  7. 7.
    Kersting, K., Plagemann, C., Pfaff, P., Burgard, W.: Most likely heteroscedastic Gaussian process regression. In: ICML, pp. 393–400. ACM, New York (2007)CrossRefGoogle Scholar
  8. 8.
    Lzaro-gredilla, M., Titsias, M.K.: Variational heteroscedastic Gaussian process regression. In: ICML, pp. 841–848. ACM (2011)Google Scholar
  9. 9.
    McHutchon, A., Rasmussen, C.E.: Gaussian process training with input noise. In: NIPS, pp. 1341–1349 (2011)Google Scholar
  10. 10.
    Nguyen-Tuong, D., Peters, J.: Incremental online sparsification for model learning in real-time robot control. Neurocomputing 74(11), 1859–1867 (2011)CrossRefGoogle Scholar
  11. 11.
    Rifai, S., Glorot, X., Bengio, Y., Vincent, P.: Adding noise to the input of a model trained with a regularized objective. CoRR abs/1104.3250 (2011)Google Scholar
  12. 12.
    Tresp, V., Ahmad, S., Neuneier, R.: Training neural networks with deficient data. In: NIPS, pp. 128–135. Morgan Kaufman Publishers (1994)Google Scholar
  13. 13.
    Wright, W.: Neural network regression with input uncertainty. In: Neural Networks for Signal Processing VIII, pp. 284–293. IEEE (1998)Google Scholar
  14. 14.
    Yeh, I.C.: Modeling of strength of high performance concrete using artificial neural networks. Cement and Concrete Research 28(12), 1797–1808 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Botond Attila Bócsi
    • 1
  • Lehel Csató
    • 1
  1. 1.Faculty of Mathematics and InformaticsBabeş-Bolyai UniversityRomania

Personalised recommendations