Advertisement

Adding Noise During Training as a Method to Increase Resilience of Neural Network Solution of Inverse Problems: Test on the Data of Magnetotelluric Sounding Problem

  • Igor Isaev
  • Sergey DolenkoEmail author
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 736)

Abstract

In their previous studies, the authors proposed to use the approach associated with adding noise to the training set when training multilayer perceptron type neural networks to solve inverse problems. For a model inverse problem it was shown that this allows increasing the resilience of neural network solution to noise in the input data with different distributions and various intensity of noise. In the present study, the observed effect was confirmed on the data of the problem of magnetotelluric sounding. Also, maximum noise resilience (maximum quality of the solution) is generally achieved when the level of the noise in the training data set coincides with the level of noise during network application (in the test dataset). Thus, increasing noise resilience of a network when noise is added during its training is associated with the fundamental properties of multilayer perceptron neural networks and not with the properties of the data. So this method can be used solving other multi-parameter inverse problems.

Keywords

Neural networks Inverse problems Noise resilience Training with noise Regularization 

Notes

Acknowledgement

The authors would like to thank E.A. Obornev, I.E. Obornev, and M.I. Shimelevich for providing the data on which this study has been performed.

References

  1. 1.
    Zhdanov, M.: Inverse Theory and Applications in Geophysics, 2nd edn. Elsevier, Amsterdam (2015)Google Scholar
  2. 2.
    Yagola, A., Kochikov, I., Kuramshina, G.: Inverse Problems of Vibrational Spectroscopy. De Gruyter, Berlin (1999)CrossRefzbMATHGoogle Scholar
  3. 3.
    Mohammad-Djafari, A. (ed.): Inverse Problems in Vision and 3D Tomography. Wiley, New York (2010)Google Scholar
  4. 4.
    Spichak, V.V. (ed.): Electromagnetic Sounding of the Earth’s Interior. Methods in Geochemistry and Geophysics, vol. 40. Elsevier, Amsterdam (2006)Google Scholar
  5. 5.
    Zhdanov, M.S.: Geophysical Electromagnetic Theory and Methods. Methods in Geochemistry and Geophysics, vol. 43. Elsevier, Amsterdam (2009)Google Scholar
  6. 6.
    Spichak, V., Popova, I.: Artificial neural network inversion of magnetotelluric data in terms of three-dimensional earth macroparameters. Geophys. J. Int. 142(1), 15–26 (2000)CrossRefGoogle Scholar
  7. 7.
    Li, M., Verma, B., Fan, X., Tickle, K.: RBF neural networks for solving the inverse problem of backscattering spectra. Neural Comput. Appl. 17(4), 391–397 (2008)CrossRefGoogle Scholar
  8. 8.
    Yang, H., Xu, M.: Solving inverse bimodular problems via artificial neural network. Inverse Probl. Sci. Eng. 17(8), 999–1017 (2009)CrossRefzbMATHGoogle Scholar
  9. 9.
    Holmstrom, L., Koistinen, P.: Using additive noise in back-propagation training. Proc. IEEE Trans. Neural Netw. 3(1), 24–38 (1992)CrossRefGoogle Scholar
  10. 10.
    Matsuoka, K.: Noise injection into inputs in back-propagation learning. Proc. IEEE Trans. Syst. Man Cybern. 22(3), 436–440 (1992)CrossRefGoogle Scholar
  11. 11.
    An, G.: The effects of adding noise during backpropagation training on a generalization performance. Neural Comput. 8(3), 643–674 (1996)CrossRefGoogle Scholar
  12. 12.
    Zur, R.M., Jiang, Y., Pesce, L.L., Drukker, K.: Noise injection for training artificial neural networks: a comparison with weight decay and early stopping. Med. Phys. 36(10), 4810–4818 (2009)CrossRefGoogle Scholar
  13. 13.
    Piotrowski, A.P., Napiorkowski, J.J.: A comparison of methods to avoid overfitting in neural networks training in the case of catchment runoff modeling. J. Hydrol. 476, 97–111 (2013)CrossRefGoogle Scholar
  14. 14.
    Bishop, C.M.: Training with noise is equivalent to Tikhonov regularization. Neural Comput. 7(1), 108–116 (1995)CrossRefGoogle Scholar
  15. 15.
    Wang, C., Principe, J.C.: Training neural networks with additive noise in the desired signal. Proc. IEEE Trans. Neural Netw. 10(6), 1511–1517 (1999)CrossRefGoogle Scholar
  16. 16.
    Yin, S., Liu, C., Zhang, Z., Lin, Y., Wang, D., Tejedor, J., Zheng, T.F., Li, Y.: Noisy training for deep neural networks in speech recognition. Proc. EURASIP J. Audio Speech Music Process. 2015(2), 1–14 (2015)Google Scholar
  17. 17.
    Fadeev, V.V., Dolenko, S.A., Dolenko, T.A., Uvenkov, Ya.V., Filippova, E.M., Chubarov, V.V.: Laser diagnostics of complicated organic compounds and complexes by saturation fluorimetry. Quantum Electron. 27(6), 556–559 (1997)Google Scholar
  18. 18.
    Dolenko, S.A., Dolenko, T.A., Kozyreva, O.V., Persiantsev, I.G., Fadeev, V.V., Filippova, E.M.: Solution of inverse problem in nonlinear laser fluorimetry of organic compounds with the use of artificial neural networks. Pattern Recognit. Image Anal. 9(3), 510–515 (1999)Google Scholar
  19. 19.
    Gerdova, I.V., Dolenko, S.A., Dolenko, T.A., Persiantsev, I.G., Fadeev, V.V., Churina, I.V.: New opportunity solutions to inverse problems in laser spectroscopy involving artificial neural networks. Izvestiya Akademii Nauk. Ser. Fizicheskaya 66(8), 1116–1125 (2002)Google Scholar
  20. 20.
    Isaev, I.V., Dolenko, S.A.: Training with noise as a method to increase noise resilience of neural network solution of inverse problems. Opt. Mem. Neural Netw. 25(3), 142–148 (2016)CrossRefGoogle Scholar
  21. 21.
    Dolenko, S., Isaev, I., Obornev, E., Persiantsev, I., Shimelevich, M.: Study of influence of parameter grouping on the error of neural network solution of the inverse problem of electrical prospecting. Commun. Comput. Inf. Sci. 383, 81–90 (2013)Google Scholar
  22. 22.
    Dolenko, S., Guzhva, A., Obornev, E., Persiantsev, I., Shimelevich, M.: Comparison of adaptive algorithms for significant feature selection in neural network based solution of the inverse problem of electrical prospecting. In: Alippi, C. (ed.) ICANN 2009, Part II. Springer-Verlag, Heidelberg (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.D.V. Skobeltsyn Institute of Nuclear PhysicsM.V. Lomonosov Moscow State UniversityMoscowRussia

Personalised recommendations