Skip to main content
SpringerLink
Log in

Higher-order and long-range synchronization effects for classification and computing in oscillator-based spiking neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In the circuit of two thermally coupled VO2 oscillators, we studied a higher-order synchronization effect, which can be used in object classification techniques to increase the number of possible synchronous states of the oscillator system. We developed the phase-locking estimation method to determine the values of subharmonic ratio and synchronization effectiveness. In our experiment, the number of possible synchronous states of the oscillator system was twelve, and subharmonic ratio distributions were shaped as Arnold’s tongues. In the model, the number of states may reach the maximum value of 150 at certain levels of coupling strength and noise. The long-range synchronization effect in a one-dimensional chain of oscillators occurs even at low values of synchronization effectiveness for intermediate links. We demonstrate a technique for storing and recognizing vector images, which can used for reservoir computing. In addition, we present the implementation of analog operation of multiplication, the synchronization-based logic for binary computations and the possibility to develop the interface between spike neural network and a computer. Based on the universal physical effects, the high-order synchronization can be applied to any spiking oscillators with any coupling type, enhancing the practical value of the presented results to expand spike neural network capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Yu Q, Tang H, Hu J, Tan Chen K (2017) Neuromorphic cognitive systems. Springer, Berlin. https://doi.org/10.1007/978-3-319-55310-8

    Book  MATH  Google Scholar 

  2. Rubio JDJ, Garcia E, Ochoa G, Elias I, Cruz DR, Balcazar R, Lopez J, Novoa JF (2019) Unscented Kalman filter for learning of a solar dryer and a greenhouse. J Intell Fuzzy Syst 37:6731–6741. https://doi.org/10.3233/JIFS-190216

    Article  Google Scholar 

  3. Aquino G, Rubio JDJ, Pacheco J, Gutierrez GJ, Ochoa G, Balcazar R, Cruz DR, Garcia E, Novoa JF, Zacarias A (2020) Novel nonlinear hypothesis for the delta parallel robot modeling. IEEE Access 8:46324–46334. https://doi.org/10.1109/ACCESS.2020.2979141

    Article  Google Scholar 

  4. Rubio JDJ (2009) SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17:1296–1309. https://doi.org/10.1109/TFUZZ.2009.2029569

    Article  Google Scholar 

  5. Meda-Campaña JA (2018) On the estimation and control of nonlinear systems with parametric uncertainties and noisy outputs. IEEE Access 6:31968–31973. https://doi.org/10.1109/ACCESS.2018.2846483

    Article  Google Scholar 

  6. Volgushev M, Chauvette S, Mukovski M, Timofeev I (2006) Precise long-range synchronization of activity and silence in neocortical neurons during slow-wave sleep. J Neurosci 26:5665–5672. https://doi.org/10.1523/JNEUROSCI.0279-06.2006

    Article  Google Scholar 

  7. Lachaux JP, Rodriguez E, Martinerie J, Varela FJ (1999) Measuring phase synchrony in brain signals. Hum Brain Mapp 8:194–208

    Article  Google Scholar 

  8. Barraza P, Gómez DM, Oyarzún F, Dartnell P (2014) Long-distance neural synchrony correlates with processing strategies to compare fractions. Neurosci Lett 567:40–44. https://doi.org/10.1016/J.NEULET.2014.03.021

    Article  Google Scholar 

  9. Ponulak F, Kasinski A (2011) Introduction to spiking neural networks: information processing, learning and applications. Acta Neurobiol Exp (Wars) 71:409–433

    MATH  Google Scholar 

  10. Belyaev M, Velichko A (2019) A spiking neural network based on the model of VO2—neuron. Electronics 8:1065. https://doi.org/10.3390/electronics8101065

    Article  Google Scholar 

  11. Velichko A, Belyaev M, Putrolaynen V, Boriskov P (2018) A new method of the pattern storage and recognition in oscillatory neural networks based on resistive switches. Electronics 7:266. https://doi.org/10.3390/electronics7100266

    Article  Google Scholar 

  12. Velichko A, Belyaev M, Boriskov P (2019) A model of an oscillatory neural network with multilevel neurons for pattern recognition and computing. Electronics 8:75. https://doi.org/10.3390/electronics8010075

    Article  Google Scholar 

  13. Velichko A (2019) A method for evaluating chimeric synchronization of coupled oscillators and its application for creating a neural network information converter. Electronics 8:756. https://doi.org/10.3390/electronics8070756

    Article  Google Scholar 

  14. Pikovsky A, Rosenblum M, Kurths J (2001) Synchronization : a universal concept in nonlinear sciences. Cambridge University Press, Cambridge

    Book  Google Scholar 

  15. Fang Y, Yashin VV, Levitan SP, Balazs AC (2016) Pattern recognition with “materials that compute”. Sci Adv 2:e1601114. https://doi.org/10.1126/sciadv.1601114

    Article  Google Scholar 

  16. Maffezzoni P, Bahr B, Zhang Z, Daniel L (2015) Oscillator array models for associative memory and pattern recognition. IEEE Trans Circuits Syst I Regul Pap 62:1591–1598. https://doi.org/10.1109/TCSI.2015.2418851

    Article  MathSciNet  Google Scholar 

  17. Nikonov DE, Csaba G, Porod W, Shibata T, Voils D, Hammerstrom D, Young IA, Bourianoff GI (2015) Coupled-oscillator associative memory array operation for pattern recognition. IEEE J Explor Solid-State Comput Devices Circuits 1:85–93. https://doi.org/10.1109/JXCDC.2015.2504049

    Article  Google Scholar 

  18. Vodenicarevic D, Locatelli N, Abreu Araujo F, Grollier J, Querlioz D (2017) A nanotechnology-ready computing scheme based on a weakly coupled oscillator network. Sci Rep 7:44772. https://doi.org/10.1038/srep44772

    Article  Google Scholar 

  19. Meier M, Haschke R, Ritter HJ (2014) Perceptual grouping through competition in coupled oscillator networks. Neurocomputing 141:76–83. https://doi.org/10.1016/J.NEUCOM.2014.02.011

    Article  Google Scholar 

  20. Benicasa AX, Quiles MG, Silva TC, Zhao L, Romero RAF (2016) An object-based visual selection framework. Neurocomputing 180:35–54. https://doi.org/10.1016/J.NEUCOM.2015.10.111

    Article  Google Scholar 

  21. Yogendra K, Fan D, Shim Y, Koo M, Roy K (2016) Computing with coupled spin torque nano oscillators. In: 2016 21st Asia South Pacific design automation conference. IEEE, pp 312–317. https://doi.org/10.1109/ASPDAC.2016.7428030

  22. Shukla N, Tsai W-Y, Jerry M, Barth M, Narayanan V, Datta S (2016) Ultra low power coupled oscillator arrays for computer vision applications. In: 2016 IEEE symposium on VLSI technology. IEEE, pp 1–2. https://doi.org/10.1109/VLSIT.2016.7573439

  23. Lynch S (2014) Binary oscillator computing. In: Lynch S (ed) Dynamical systems with applications using MATLAB®. Springer, Cham, pp 435–455. https://doi.org/10.1007/978-3-319-06820-6_20

    Chapter  MATH  Google Scholar 

  24. Coulombe JC, York MCA, Sylvestre J (2017) Computing with networks of nonlinear mechanical oscillators. PloS One 12:e0178663. https://doi.org/10.1371/journal.pone.0178663

    Article  Google Scholar 

  25. Shukla N, Parihar A, Freeman E, Paik H, Stone G, Narayanan V, Wen H, Cai Z, Gopalan V, Engel-Herbert R, Schlom DG, Raychowdhury A, Datta S (2015) Synchronized charge oscillations in correlated electron systems. Sci Rep 4:4964. https://doi.org/10.1038/srep04964

    Article  Google Scholar 

  26. Parihar A, Shukla N, Datta S, Raychowdhury A (2016) Computing with dynamical systems in the post-CMOS era. In: 2016 IEEE photonics society summer topicals meeting series. IEEE, pp 110–111. https://doi.org/10.1109/PHOSST.2016.7548777

  27. Sharma AA, Bain JA, Weldon JA (2015) Phase coupling and control of oxide-based oscillators for neuromorphic computing. IEEE J Explor Solid-State Comput Devices Circuits 1:58–66. https://doi.org/10.1109/JXCDC.2015.2448417

    Article  Google Scholar 

  28. Li S, Liu X, Nandi SK, Venkatachalam DK, Elliman RG (2017) Coupling dynamics of Nb/Nb2O5 relaxation oscillators. Nanotechnology 28:125201. https://doi.org/10.1088/1361-6528/aa5de0

    Article  Google Scholar 

  29. Kumar A, Mohanty P (2017) Autoassociative memory and pattern recognition in micromechanical oscillator network. Sci Rep 7:411. https://doi.org/10.1038/s41598-017-00442-y

    Article  Google Scholar 

  30. Flovik V, Macià F, Wahlström E (2016) Describing synchronization and topological excitations in arrays of magnetic spin torque oscillators through the Kuramoto model. Sci Rep 6:32528. https://doi.org/10.1038/srep32528

    Article  Google Scholar 

  31. Csaba G, Porod W (2013) Computational study of spin-torque oscillator interactions for non-boolean computing applications. IEEE Trans Magn 49:4447–4451. https://doi.org/10.1109/TMAG.2013.2244202

    Article  Google Scholar 

  32. Lebrun R, Tsunegi S, Bortolotti P, Kubota H, Jenkins AS, Romera M, Yakushiji K, Fukushima A, Grollier J, Yuasa S, Cros V (2017) Mutual synchronization of spin torque nano-oscillators through a long-range and tunable electrical coupling scheme. Nat Commun 8:15825. https://doi.org/10.1038/ncomms15825

    Article  Google Scholar 

  33. Locatelli N, Hamadeh A, Abreu Araujo F, Belanovsky AD, Skirdkov PN, Lebrun R, Naletov VV, Zvezdin KA, Muñoz M, Grollier J, Klein O, Cros V, de Loubens G (2015) Efficient synchronization of dipolarly coupled vortex-based spin transfer nano-oscillators. Sci Rep 5:17039. https://doi.org/10.1038/srep17039

    Article  Google Scholar 

  34. Erzgräber H, Wieczorek S, Krauskopf B (2009) Locking behavior of three coupled laser oscillators. Phys Rev E 80:026212. https://doi.org/10.1103/PhysRevE.80.026212

    Article  Google Scholar 

  35. Yoshihara F, Fuse T, Ashhab S, Kakuyanagi K, Saito S, Semba K (2016) Superconducting qubit–oscillator circuit beyond the ultrastrong-coupling regime. Nat Phys 13:44–47. https://doi.org/10.1038/nphys3906

    Article  Google Scholar 

  36. Kemeth FP, Haugland SW, Schmidt L, Kevrekidis IG, Krischer K (2016) A classification scheme for chimera states. Chaos Interdiscip J Nonlinear Sci 26:094815. https://doi.org/10.1063/1.4959804

    Article  Google Scholar 

  37. Heltberg ML, Jensen MH (2019) Locked body clocks. Nat Phys. https://doi.org/10.1038/s41567-019-0617-2

    Article  Google Scholar 

  38. Heltberg M, Kellogg RA, Krishna S, Tay S, Jensen MH (2016) Noise induces hopping between NF-κB entrainment modes. Cell Syst 3:532.e3–539.e3. https://doi.org/10.1016/j.cels.2016.11.014

    Article  Google Scholar 

  39. Jerry M, Tsai W, Xie B, Li X, Narayanan V, Raychowdhury A, Datta S (2016) Phase transition oxide neuron for spiking neural networks. In: 2016 74th annual device research conference. IEEE, pp 1–2. https://doi.org/10.1109/DRC.2016.7548503

  40. Pickett MD, Medeiros-Ribeiro G, Williams RS (2013) A scalable neuristor built with Mott memristors. Nat Mater 12:114–117. https://doi.org/10.1038/nmat3510

    Article  Google Scholar 

  41. Boriskov P, Velichko A (2019) Switch elements with S-shaped current–voltage characteristic in models of neural oscillators. Electronics 8:922. https://doi.org/10.3390/electronics8090922

    Article  Google Scholar 

  42. Velichko AA, Stefanovich GB, Pergament AL, Boriskov PP (2003) Deterministic noise in vanadium dioxide based structures. Tech Phys Lett 29:435–437. https://doi.org/10.1134/1.1579818

    Article  Google Scholar 

  43. Jerry M, Parihar A, Raychowdhury A, Datta S (2017) A random number generator based on insulator-to-metal electronic phase transitions. In: 2017 75th annual device research conference. IEEE, pp 1–2. https://doi.org/10.1109/DRC.2017.7999423

  44. Parihar A, Jerry M, Datta S, Raychowdhury A (2018) Stochastic IMT (insulator–metal-transition) neurons: an interplay of thermal and threshold noise at bifurcation. Front Neurosci 12:210. https://doi.org/10.3389/fnins.2018.00210

    Article  Google Scholar 

  45. Jerry M, Parihar A, Grisafe B, Raychowdhury A, Datta S (2017) Ultra-low power probabilistic IMT neurons for stochastic sampling machines. In: 2017 symposium on VLSI technology. IEEE, pp T186–T187. https://doi.org/10.23919/VLSIT.2017.7998148

  46. Czanner G, Sarma SV, Ba D, Eden UT, Wu W, Eskandar E, Lim HH, Temereanca S, Suzuki WA, Brown EN (2015) Measuring the signal-to-noise ratio of a neuron. Proc Natl Acad Sci U S A 112:7141–7146. https://doi.org/10.1073/pnas.1505545112

    Article  Google Scholar 

  47. Faisal AA, Selen LPJ, Wolpert DM (2008) Noise in the nervous system. Nat Rev Neurosci 9:292–303. https://doi.org/10.1038/nrn2258

    Article  Google Scholar 

  48. Branco T, Staras K (2009) The probability of neurotransmitter release: variability and feedback control at single synapses. Nat Rev Neurosci 10:373–383. https://doi.org/10.1038/nrn2634

    Article  Google Scholar 

  49. Velichko A, Belyaev M, Putrolaynen V, Perminov V, Pergament A (2018) Modeling of thermal coupling in VO2-based oscillatory neural networks. Solid State Electron 139:8–14. https://doi.org/10.1016/j.sse.2017.09.014

    Article  Google Scholar 

  50. Velichko A, Belyaev M, Putrolaynen V, Perminov V, Pergament A (2018) Thermal coupling and effect of subharmonic synchronization in a system of two VO2 based oscillators. Solid State Electron 141:40–49. https://doi.org/10.1016/j.sse.2017.12.003

    Article  Google Scholar 

  51. Velichko A, Belyaev M, Putrolaynen V, Pergament A, Perminov V (2017) Switching dynamics of single and coupled VO2-based oscillators as elements of neural networks. Int J Mod Phys B 31:1650261. https://doi.org/10.1142/S0217979216502611

    Article  Google Scholar 

  52. Lowet E, Roberts MJ, Bonizzi P, Karel J, De Weerd P (2016) Quantifying neural oscillatory synchronization: a comparison between spectral coherence and phase-locking value approaches. PloS One 11:e0146443. https://doi.org/10.1371/journal.pone.0146443

    Article  Google Scholar 

  53. Parihar A, Shukla N, Jerry M, Datta S, Raychowdhury A (2017) Vertex coloring of graphs via phase dynamics of coupled oscillatory networks. Sci Rep 7:911. https://doi.org/10.1038/s41598-017-00825-1

    Article  Google Scholar 

  54. Shukla N, Parihar A, Cotter M, Barth M, Li X, Chandramoorthy N, Paik H, Schlom DG, Narayanan V, Raychowdhury A, Datta S (2014) Pairwise coupled hybrid vanadium dioxide-MOSFET (HVFET) oscillators for non-boolean associative computing. In: 2014 IEEE international electron devices meeting. IEEE, pp 28.7.1–28.7.4. https://doi.org/10.1109/IEDM.2014.7047129

  55. Von NJ (1954) Non-linear capacitance or inductance switching, amplifying, and memory organs. US2815488A. https://patents.google.com/patent/US2815488. Accessed 19 Mar 2018

  56. Roychowdhury J (2015) Boolean computation using self-sustaining nonlinear oscillators. Proc IEEE 103:1958–1969. https://doi.org/10.1109/JPROC.2015.2483061

    Article  Google Scholar 

  57. Pergament A, Velichko A, Belyaev M, Putrolaynen V (2018) Electrical switching and oscillations in vanadium dioxide. Phys B Condens Matter 536:239–248. https://doi.org/10.1016/j.physb.2017.10.123

    Article  Google Scholar 

  58. Belyaev MA, Boriskov PP, Velichko AA, Pergament AL, Putrolainen VV, Ryabokon’ DV, Stefanovich GB, Sysun VI, Khanin SD (2018) Switching channel development dynamics in planar structures on the basis of vanadium dioxide. Phys Solid State 60:447–456. https://doi.org/10.1134/S1063783418030046

    Article  Google Scholar 

  59. Pergament A, Stefanovich G, Velichko A (2017) Relaxation oscillations in circuits containing sandwich switches based on vanadium dioxide. Phase Transit 90:351–361. https://doi.org/10.1080/01411594.2016.1201818

    Article  Google Scholar 

  60. Belyaev M, Velichko A, Putrolaynen V, Perminov V, Pergament A (2017) Electron beam modification of vanadium dioxide oscillators. Phys Status Solidi Curr Top Solid State Phys. https://doi.org/10.1002/pssc.201600236

    Article  Google Scholar 

  61. Jerry M, Ni K, Parihar A, Raychowdhury A, Datta S (2018) Stochastic insulator-to-metal phase transition-based true random number generator. IEEE Electron Device Lett 39:139–142. https://doi.org/10.1109/LED.2017.2771812

    Article  Google Scholar 

  62. Somers D, Kopell N (1993) Rapid synchronization through fast threshold modulation. Biol Cybern 68:393–407. https://doi.org/10.1007/BF00198772

    Article  Google Scholar 

  63. Yamamoto H, Matsumura R, Takaoki H, Katsurabayashi S, Hirano-Iwata A, Niwano M (2016) Unidirectional signal propagation in primary neurons micropatterned at a single-cell resolution. Appl Phys Lett 109:043703. https://doi.org/10.1063/1.4959836

    Article  Google Scholar 

  64. Tanaka G, Yamane T, Héroux JB, Nakane R, Kanazawa N, Takeda S, Numata H, Nakano D, Hirose A (2019) Recent advances in physical reservoir computing: a review. Neural Netw 115:100–123. https://doi.org/10.1016/j.neunet.2019.03.005

    Article  Google Scholar 

  65. Zhou W, Goh WL, Gao Y (2019) A 3-MHz 17.3-μW 0.015% period jitter relaxation oscillator with energy efficient swing boosting. IEEE Trans Circuits Syst II Express Briefs 1:1. https://doi.org/10.1109/tcsii.2019.2948032

    Article  Google Scholar 

  66. Callan R (1998) Essence of neural networks. Prentice Hall PTR, Upper Saddle River

    Google Scholar 

  67. Kernel method—Wikipedia, (2005). https://en.wikipedia.org/wiki/Kernel_method. Accessed 29 Mar 2020

  68. Lukoševičius M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3:127–149. https://doi.org/10.1016/j.cosrev.2009.03.005

    Article  MATH  Google Scholar 

  69. Hoppensteadt FCFC, Izhikevich EMEM (2000) Pattern recognition via synchronization in phase-locked loop neural networks. IEEE Trans Neural Netw 11:734–738. https://doi.org/10.1109/72.846744

    Article  Google Scholar 

  70. Romera M, Talatchian P, Tsunegi S, Abreu Araujo F, Cros V, Bortolotti P, Trastoy J, Yakushiji K, Fukushima A, Kubota H, Yuasa S, Ernoult M, Vodenicarevic D, Hirtzlin T, Locatelli N, Querlioz D, Grollier J (2018) Vowel recognition with four coupled spin-torque nano-oscillators. Nature 563:230–234. https://doi.org/10.1038/s41586-018-0632-y

    Article  Google Scholar 

  71. Jerominek H, Renaud M, Swart NR, Picard F, Pope TD, Levesque M, Lehoux M, Bilodeau G, Pelletier M, Audet D, Lambert P (1996) Micromachined VO2-based uncooled IR bolometric detector arrays with integrated CMOS readout electronics. In: Micromachined devices components II, SPIE, pp 111–121. https://doi.org/10.1117/12.250694

  72. Kumar S, Strachan JP, Williams RS (2017) Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing. Nature 548:318–321. https://doi.org/10.1038/nature23307

    Article  Google Scholar 

  73. Wang L, Ren W, Wen J, Xiong B (2018) Overview of phase-change electrical probe memory. Nanomaterials. https://doi.org/10.3390/nano8100772

    Article  Google Scholar 

  74. Foong A, Hady F (2016) Storage as fast as rest of the system. In: 2016 IEEE 8th international memory work, IMW 2016. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/IMW.2016.7495289

Download references

Acknowledgements

This research was supported by Russian Science Foundation (Grant no. 16-19-00135). The authors express their gratitude to Dr. Andrei Rikkiev for the valuable comments in the course of the article translation and revision.

Author information

Authors and Affiliations

  1. Petrozavodsk State University, Petrozavodsk, Russia, 185910

    Andrei Velichko, Vadim Putrolaynen & Maksim Belyaev

Authors
  1. Andrei Velichko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vadim Putrolaynen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maksim Belyaev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrei Velichko.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Velichko, A., Putrolaynen, V. & Belyaev, M. Higher-order and long-range synchronization effects for classification and computing in oscillator-based spiking neural networks. Neural Comput & Applic 33, 3113–3131 (2021). https://doi.org/10.1007/s00521-020-05177-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05177-y

Keywords

Search

Navigation