Skip to main content
SpringerLink
Log in

Elimination of Negative Circuits in Certain Neural Network Structures to Achieve Stable Solutions

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

The problem of improving structural properties of artificial discrete neural networks is investigated. The structure of the neural network is regarded as a theoretical graph. Cyclical substructures can occur in this structure under certain conditions, e.g., when there is feedback among neural network layers. Some properties of cycles in the graphs corresponding to neural networks have a significant effect not only on the rate of convergence to the (stable) solutions of the problems posed by network users, but also on the very possibility of obtaining these solutions. These properties include the negativity of some circuits (simple cycles) in neural network graphs. We propose an algorithm that makes it possible to eliminate negative circuits from neural network graphs under certain constraints formulated in this paper. It increases the chances of finding correct solutions to the problems for which neural networks were developed. An illustrative example is presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.

Similar content being viewed by others

REFERENCES

  1. Cireşan, D., Meier, U., Masci, J., and Schmidhuber, J., Multi-column deep neural network for traffic sign classification, Neural Networks, 2012.

  2. Talbot, D., CES 2015: Nvidia demos a car computer trained with “deep learning,” MIT Technol. Rev., 2015.

    Google Scholar 

  3. Roth, S., Shrinkage fields for effective image restoration, Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2014.

  4. Deng, L. and Yu, D., Deep learning: Methods and applications, Found. Trends Signal Process., 2014, vol. 7, nos. 3–4, pp. 1–19.

    Article  MathSciNet  Google Scholar 

  5. Rosenblatt, F., Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Washington: Spartan Books, 1961.

    Book  Google Scholar 

  6. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundation, Rumelhart, D.E., McClelland, J.L., and PDP research group, Eds., MIT Press, 1986.

  7. Hopfield, J.J., Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA, 1982, vol. 79, no. 8, pp. 2554–2558.

    Article  MathSciNet  Google Scholar 

  8. Ackley, D.H., Hinton, G.E., and Sejnowski, T.J., A learning algorithm for Boltzmann machines, Cognit. Sci., 1985, vol. 9, no. 1, pp. 147–169.

    Article  Google Scholar 

  9. Kohonen, T., Self-organized formation of topologically correct feature maps, Biol. Cybern., 1982, vol. 43, no. 1, pp. 59–69.

    Article  MathSciNet  Google Scholar 

  10. Elman, J.L., Finding structure in time, Cognit. Sci., 1990, vol. 14, no. 2, pp. 79–211.

    Article  Google Scholar 

  11. Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P., Gradient-based learning applied to document recognition, Proc. IEEE, 1998, vol. 86, no. 11, pp. 2278–2324.

    Article  Google Scholar 

  12. Graves, A., Greg, W., and Ivo, D., Neural Turing machines, 2014.

  13. Ritter, G.X. and Sussner, P., An introduction to morphological neural networks, Proc. 13th Int. Conf. Pattern Recognition, Vienna, 1996, pp. 709–717.

  14. Karpov, Yu.L., Karpov, L.E., and Smetanin, Yu.G., Adaptation of general concepts of software testing to neural networks, Program. Comput. Software, 2018, vol. 44, no. 5, pp. 324–334.

    Article  MathSciNet  Google Scholar 

  15. IEEE 829 Standard for Software Test Documentation and IEEE 1008 Standard for Software Unit Testing. https://www.twirpx.com/file/1615980.

  16. ISO/MEK 12119. Information technology. Software packages. Quality requirements and testing. http://docs.cntd.ru/document/1200025075.

  17. GOST R 56920-2016, GOST R 56921–2016, and GOST R 56922–2016. https://allgosts.ru.

  18. ISO/IEC 29119-2013 1-5. Software testing. http://files.stroyinf.ru/Data2/1/4293754/ 4293754866.pdf.

  19. GOST R 12207-2010 and ISO/IEC 12207:2008. http://docs.cntd.ru/document/1200082859.

  20. Aracena, J., Demongeot, J.D., and Goles, E., Positive and negative circuits in discrete neural networks, IEEE Trans. Neural Networks, 2004, vol. 15, no. 1.

    Article  Google Scholar 

  21. Prim, R.C., Shortest connection networks and some generalizations, Bell Syst. Tech. J., 1957, vol. 36, no. 6, pp. 1389–1401.

    Article  Google Scholar 

  22. Cormen, T.H., Charles, E., Leiserson, C.E., Rivest, R.L., and Stein, C., Introduction to Algorithms, MIT Press, McGraw-Hill, 2002, 2nd ed.

    MATH  Google Scholar 

Download references

Funding

This work was supported by the Russian Foundation for Basic Research, project nos. 18-07-00697-а, 18-07-01211-а, 19-07-00321-а, and 19-07-00493-а.

Author information

Authors and Affiliations

  1. Luxoft Professional LLC, 1-i Volokolamskii proezd 10, 123060, Moscow, Russia

    Yu. L. Karpov

  2. Ivannikov Institute for System Programming, Russian Academy of Sciences, ul. Solzhenitsyna 25, 109004, Moscow, Russia

    L. E. Karpov

  3. Moscow State University, 119991, Moscow, Russia

    L. E. Karpov

  4. Federal Research Center Computer Science and Control, Russian Academy of Sciences, ul. Vavilova 44/2, 119333, Moscow, Russia

    Yu. G. Smetanin

  5. Moscow Institute of Physics and Technology, Institutskii per. 9, 141701, Dolgoprudnyi, Moscow oblast, Russia

    Yu. G. Smetanin

Authors
  1. Yu. L. Karpov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. E. Karpov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu. G. Smetanin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yu. L. Karpov, L. E. Karpov or Yu. G. Smetanin.

Additional information

Translated by Yu. Kornienko

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karpov, Y.L., Karpov, L.E. & Smetanin, Y.G. Elimination of Negative Circuits in Certain Neural Network Structures to Achieve Stable Solutions. Program Comput Soft 45, 241–250 (2019). https://doi.org/10.1134/S0361768819050025

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768819050025

Search

Navigation