Skip to main content
SpringerLink
Log in

A Nonlinear Fokker-Planck Description of Continuous Neural Network Dynamics

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation (ICANN 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11727))

Included in the following conference series:

Abstract

The nonextensive thermostatistical formalism has been increasingly applied to the description of many complex systems in physics, biology, psychology, economics, and other fields. The q-Maximum Entropy (q-MaxEnt) distributions, which optimize the \(S_q\), power-law entropic functionals, are central to this formalism. We have done previous work regarding computational neural models of associative memory functioning, for mental phenomena such as neurosis, creativity, and the interplay between consciousness and unconsciousness, which suggest that q-MaxEnt distributions may be relevant for the development of neural models for these processes. Power-law behavior has also been experimentally observed in brain functioning. We propose here a nonlinear Fokker-Planck model, associated with the continuous-time evolution equations for interconnected neurons of the Hopfield model. The equation which characterizes the model has stationary solutions of the q-MaxEnt type and is associated with a free energy like quantity that decreases during the time-evolution of the system. This framework elucidates a possible dynamical mechanism which can generate q-MaxEnt distributions in Hopfield memory neural networks. It also provides a theoretical framework that supports the choice of different entropic measures for modelling and simulating complex networks such as the brain, as well as other artificial neural networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hopfield, J.J.: Neurons with graded responses have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. 81, 3088–3092 (1984). https://doi.org/10.1073/pnas.81.10.3088

    Article  MATH  Google Scholar 

  2. Hertz, J.A., Krogh, A., Palmer, R.G. (eds.): Introduction to the Theory of Neural Computation. Lecture Notes, vol. 1. Perseus Books, Cambridge (1991)

    Google Scholar 

  3. Wedemann, R.S., Donangelo, R., Carvalho, L.A.V.: Generalized memory associativity in a network model for the neuroses. Chaos 19, 015116 (2009). https://doi.org/10.1063/1.3099608

    Article  MathSciNet  Google Scholar 

  4. Kandel, E.: Psychiatry, Psychoanalysis, and the New Biology of Mind. American Psychiatric Publishing Inc., Washington (2005)

    Google Scholar 

  5. Cleeremans, A., Timmermans, B., Pasquali, A.: Consciousness and metarepresentation: a computational sketch. Neural Netw. 20, 1032–1039 (2007). https://doi.org/10.1016/j.neunet.2007.09.011

    Article  Google Scholar 

  6. Taylor, J.G.: A neural model of the loss of self in schizophrenia. Schizophr. Bull. 37(6), 1229–1247 (2011). https://doi.org/10.1093/schbul/sbq033

    Article  Google Scholar 

  7. Edalat, A., Mancinelli, F.: Strong attractors of Hopfield neural networks to model attachment types and behavioural patterns. In: Angelov, P., Levine, D., Apolloni, B. (eds.) Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN), pp. 14027190–1-10. IEEE, Red Hook (2013). https://doi.org/10.1109/IJCNN.2013.6706924

  8. Edalat, A.: Self-attachment: a holistic approach to computational psychiatry. In: Érdi, P., Sen Bhattacharya, B., Cochran, A.L. (eds.) Computational Neurology and Psychiatry. SSB, vol. 6, pp. 273–314. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49959-8_10

    Chapter  Google Scholar 

  9. Taylor, J.G., Villa, A.E.P.: The “Conscious I”: a neuroheuristic approach to the mind. In: Baltimore, D., Dulbecco, R., Francois, J., Levi-Montalcini, R. (eds.) Frontiers of Life, vol. 3, pp. 349–368. Academic Press, Cambridge (2001)

    Google Scholar 

  10. de Carvalho, L.A.V., Mendes, D.Q., Wedemann, R.S.: Creativity and delusions: the dopaminergic modulation of cortical maps. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Dongarra, J.J., Zomaya, A.Y., Gorbachev, Y.E. (eds.) ICCS 2003. LNCS, vol. 2657, pp. 511–520. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-44860-8_53

    Chapter  Google Scholar 

  11. Carhart-Harris, R.L., Friston, K.J.: Free-energy and Freud: an update. In: Fotopoulou, A., Pfaff, D., Conway, M.A. (eds.) From the Couch to the Lab: Trends in Psychodynamic Neuroscience, pp. 219–229. Oxford University Press, Oxford (2012). https://doi.org/10.1093/med/9780199600526.003.0013

    Chapter  Google Scholar 

  12. Carhart-Harris, R.L., Friston, K.J.: The default-mode, ego-functions and free-energy: a neurobiological account of Freudian ideas. Brain 133(4), 1265–1283 (2010). https://doi.org/10.1093/brain/awq010

    Article  Google Scholar 

  13. Carhart-Harris, R., et al.: The entropic brain: a theory of conscious states informed by neuroimaging research with psychedelic drugs. Front. Hum. Neurosci. 8, 20 (2014). https://doi.org/10.3389/fnhum.2014.00020

    Article  Google Scholar 

  14. Badcock, P.B., Friston, K.J., Ramstead, M.J.D.: The hierarchically mechanistic mind: a free-energy formulation of the human psyche. Phys. Life Rev. (2019, in Press). https://doi.org/10.1016/j.plrev.2018.10.002

  15. Beggs, J.M., Plenz, D.: Neuronal avalanches in neocortical circuits. J. Neurosci. 23, 11167–11177 (2003). https://doi.org/10.1523/JNEUROSCI.23-35-11167.2003

    Article  Google Scholar 

  16. Tagliazucchi, E., Balenzuela, P., Fraiman, D., Chialvo, D.R.: Criticality in large-scale brain fMRI dynamics unveiled by a novel point process analysis. Front. Physiol.|Fract. Physiol. 3, 15 (2012). https://doi.org/10.3389/fphys.2012.00015

    Article  Google Scholar 

  17. Wedemann, R.S., Carvalho, L.A.V., Donangelo, R.: Access to symbolization and associativity mechanisms in a model of conscious and unconscious processes. In: Samsonovich, A.V., Jóhannsdóttir, K.R. (eds.) Biologically Inspired Cognitive Architectures 2011. Frontiers in Artificial Intelligence and Applications, vol. 233, pp. 444–449. IOS Press, Amsterdam (2011). https://doi.org/10.3233/978-1-60750-959-2-444

    Chapter  Google Scholar 

  18. Wedemann, R.S., de Carvalho, L.A.V.: Some things psychopathologies can tell us about consciousness. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012. LNCS, vol. 7552, pp. 379–386. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33269-2_48

    Chapter  Google Scholar 

  19. Siddiqui, M., Wedemann, R.S., Jensen, H.J.: Avalanches and generalized memory associativity in a network model for conscious and unconscious mental functioning. Phys. A 490, 127–138 (2018). https://doi.org/10.1016/j.physa.2017.08.011

    Article  MathSciNet  Google Scholar 

  20. Tsallis, C., Stariolo, D.A.: Generalized simulated annealing. Phys. A 233, 395–406 (1996). https://doi.org/10.1016/S0378-4371(96)00271-3

    Article  Google Scholar 

  21. Wedemann, R.S., Plastino, A.R.: Asymmetries in synaptic connections and the nonlinear Fokker-Planck formalism. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016. LNCS, vol. 9886, pp. 19–27. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44778-0_3

    Chapter  Google Scholar 

  22. Wedemann, R.S., Plastino, A.R., Tsallis, C.: Curl forces and the nonlinear Fokker-Planck equation. Phys. Rev. E 94(6), 062105 (2016). https://doi.org/10.1103/PhysRevE.94.062105

    Article  MathSciNet  Google Scholar 

  23. Wedemann, R.S., Plastino, A.R.: q-maximum entropy distributions and memory neural networks. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 300–308. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_35

    Chapter  Google Scholar 

  24. de Luca, V.T.F., Wedemann, R.S., Plastino, A.R.: Neuronal asymmetries and Fokker-Planck dynamics. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 703–713. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_69

    Chapter  Google Scholar 

  25. Cohen, M.A., Grossberg, S.: Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Syst. Man Cybern. 13, 815–826 (1983). https://doi.org/10.1109/TSMC.1983.6313075

    Article  MathSciNet  MATH  Google Scholar 

  26. Martinez, S., Plastino, A.R., Plastino, A.: Nonlinear Fokker-Planck equations and generalized entropies. Phys. A 259(1–2), 183–192 (1998). https://doi.org/10.1016/S0378-4371(98)00277-5

    Article  Google Scholar 

  27. Franck, T.D.: Nonlinear Fokker-Planck Equations: Fundamentals and Applications. Springer, Heidelberg (2005). https://doi.org/10.1007/b137680

    Book  Google Scholar 

  28. Tsallis, C.: The nonadditive entropy \(S_q\) and its applications in physics and elsewhere: some remarks. Entropy 13, 1765–1804 (2011). https://doi.org/10.3390/e13101765

    Article  MathSciNet  MATH  Google Scholar 

  29. Papa, A.R.R., da Silva, L.: Earthquakes in the brain. Theory Biosci. 116, 321–327 (1997)

    Google Scholar 

  30. Tsallis, C.: Introduction to Nonextensive Statistical Mechanics, Approaching a Complex World. Springer, New York (2009). https://doi.org/10.1007/978-0-387-85359-8

    Book  MATH  Google Scholar 

  31. Yan, H., Zhao, L., Hu, L., Wang, X., Wang, E., Wang, J.: Nonequilibrium landscape theory of neural networks. Proc. Natl. Acad. Sci. 110(45), E4185–E4194 (2013). https://doi.org/10.1073/pnas.1310692110

    Article  Google Scholar 

  32. Gajowniczek, K., Orłowski, A., Ząbkowski, T.: Simulation study on the application of the generalized entropy concept in artificial neural networks. Entropy 20(4), 1–17 (2018). https://doi.org/10.3390/e20040249

    Article  Google Scholar 

  33. Brito, S., da Silva, L.R., Tsallis, C.: Role of dimensionality in complex networks. Nature Sci. Rep. 6, 27992 (2016). https://doi.org/10.1038/srep27992

    Article  Google Scholar 

  34. Tirnakli, U., Borges, E.P.: The standard map: from Boltzmann-Gibbs statistics to Tsallis statistics. Nature Sci. Rep. 6, 23644 (2016). https://doi.org/10.1038/srep23644

    Article  Google Scholar 

  35. Malacarne, L.C., Mendes, R.S., Pedron, I.T., Lenzi, E.K.: \(N\)-dimensional nonlinear Fokker-Planck equation with time-dependent coefficients. Phys. Rev. E 65(5), 052101 (2002). https://doi.org/10.1103/PhysRevE.65.052101

    Article  Google Scholar 

  36. Schwämmle, V., Nobre, F.D., Curado, E.M.F.: Consequences of the \({H}\) theorem from nonlinear Fokker-Planck equations. Phys. Rev. E 76(4), 041123 (2007). https://doi.org/10.1103/PhysRevE.76.041123

    Article  Google Scholar 

  37. Andrade, J.S., da Silva, G.F.T., Moreira, A.A., Nobre, F., Curado, E.M.F.: Thermostatistics of overdamped motion of interacting particles. Phys. Rev. Lett. 105(26), 260601 (2010). https://doi.org/10.1103/PhysRevLett.105.260601

    Article  Google Scholar 

  38. Ribeiro, M.S., Nobre, F.D., Curado, E.M.F.: Classes of N-dimensional nonlinear Fokker-Planck equations associated to Tsallis entropy. Entropy 13(11), 1928–1944 (2011). https://doi.org/10.3390/e13111928

    Article  MathSciNet  MATH  Google Scholar 

  39. Plastino, A.R., Plastino, A.: Non-extensive statistical mechanics and generalized Fokker-Planck equation. Phys. A 222(1), 347–354 (1995). https://doi.org/10.1016/0378-4371(95)00211-1

    Article  MathSciNet  Google Scholar 

  40. Tsallis, C., Bukman, D.J.: Anomalous diffusion in the presence of external forces: exact time-dependent solutions and their thermostatistical basis. Phys. Rev. E 54(3), R2197–R2200 (1996). https://doi.org/10.1103/PhysRevE.54.R2197

    Article  Google Scholar 

Download references

Acknowledgments

We acknowledge financial support from the Brazilian National Research Council (CNPq), the Brazilian agency which funds graduate studies (CAPES) and from the Rio de Janeiro State Research Foundation (FAPERJ).

Author information

Authors and Affiliations

  1. Instituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rua São Francisco Xavier 524, Rio de Janeiro, RJ, 20550-900, Brazil

    Roseli S. Wedemann

  2. CeBio y Departamento de Ciencias Básicas, Universidad Nacional del Noroeste de la Provincia de Buenos Aires, UNNOBA, Conicet, Roque Saenz Peña 456, Junin, Argentina

    Angel R. Plastino

Authors
  1. Roseli S. Wedemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angel R. Plastino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roseli S. Wedemann .

Editor information

Editors and Affiliations

  1. Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH), Neuherberg, Germany

    Igor V. Tetko

  2. Institute of Computer Science, Czech Academy of Sciences, Prague 8, Czech Republic

    Věra Kůrková

  3. Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH), Neuherberg, Germany

    Pavel Karpov

  4. Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH), Neuherberg, Germany

    Fabian Theis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wedemann, R.S., Plastino, A.R. (2019). A Nonlinear Fokker-Planck Description of Continuous Neural Network Dynamics. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation. ICANN 2019. Lecture Notes in Computer Science(), vol 11727. Springer, Cham. https://doi.org/10.1007/978-3-030-30487-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30487-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30486-7

  • Online ISBN: 978-3-030-30487-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation