Convergence in Noisy Training

  • Petri Koistinen
Chapter
Part of the Operations Research/Computer Science Interfaces Series book series (ORCS, volume 8)

Abstract

A minimization estimator minimizes the empirical risk associated with a given sample. Sometimes one calculates such an estimator based not on the original sample but on a pseudo sample obtained by adding noise to the original sample points. This may improve the generalization performance of the estimator. We consider the convergence properties (consistency and asymptotic distribution) of such an estimation procedure. Subject classification: AMS(MOS)62F12

Keywords

Random Vector Asymptotic Distribution Smoothing Parameter Generalization Performance Empirical Measure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    C.M. Bishop, Training with noise is equivalent to Tikhonov regularization, Neural Computation, Vol. 7 (1995), pp108–116.CrossRefGoogle Scholar
  2. [2]
    R.M. Dudley, Real Analysis and Probability, Wadsworth & Brooks/Cole, Pacific Grove, CA, (1989).Google Scholar
  3. [3]
    Y. Grandvalet and S. Canu, Comments on “noise injection into inputs in back propagation learning”, IEEE Trans. Systems, Man, and Cybernetics, Vol. 25 (1995), pp678–681.CrossRefGoogle Scholar
  4. [4]
    L. Holmström and P. Koistinen, Using additive noise in back-propagation training, IEEE Trans. Neural Networks, Vol. 3 (1992), pp24–38.CrossRefGoogle Scholar
  5. [5]
    P. Koistinen, Convergence of minimization estimators trained under additive noise, Research reports A 12, Rolf Nevanlinna Institute, Helsinki University of Technology (1995), (Doctoral thesis).Google Scholar
  6. [6]
    K. Matsuoka, Noise injection into inputs in back-propagation learning, IEEE Trans. Systems, Man, and Cybernetics, Vol. 22 (1992), pp436–440.CrossRefGoogle Scholar
  7. [7]
    R. Reed, R.J. Marks II, and S. Oh, Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitter, IEEE Trans. Neural Networks, Vol. 6 (1995), pp529–538.CrossRefGoogle Scholar
  8. [8]
    H. White, Learning in artificial neural networks: A statistical perspective, Neural Computation, Vol. 1 (1989), pp425–464.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1997

Authors and Affiliations

  • Petri Koistinen
    • 1
  1. 1.Rolf Nevanlinna InstituteUniversity of HelsinkiFinland

Personalised recommendations