Abstract
Modeling the neuronal processes underlying short-term working memory remains the focus of many theoretical studies in neuroscience. In this paper, we propose a mathematical model of a spiking neural network (SNN) which simulates the way a fragment of information is maintained as a robust activity pattern for several seconds and the way it completely disappears if no other stimuli are fed to the system. Such short-term memory traces are preserved due to the activation of astrocytes accompanying the SNN. The astrocytes exhibit calcium transients at a time scale of seconds. These transients further modulate the efficiency of synaptic transmission and, hence, the firing rate of neighboring neurons at diverse timescales through gliotransmitter release. We demonstrate how such transients continuously encode frequencies of neuronal discharges and provide robust short-term storage of analogous information. This kind of short-term memory can store relevant information for seconds and then completely forget it to avoid overlapping with forthcoming patterns. The SNN is inter-connected with the astrocytic layer by local inter-cellular diffusive connections. The astrocytes are activated only when the neighboring neurons fire synchronously, e.g., when an information pattern is loaded. For illustration, we took grayscale photographs of people’s faces where the shades of gray correspond to the level of applied current which stimulates the neurons. The astrocyte feedback modulates (facilitates) synaptic transmission by varying the frequency of neuronal firing. We show how arbitrary patterns can be loaded, then stored for a certain interval of time, and retrieved if the appropriate clue pattern is applied to the input.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Understanding principles of information processing in the brain remains one of the primary challenges of neuroscience [1, 2]. In theory, there should be a gap between molecular and cellular levels of implementation and its functionality at the cognitive level. Scholars proposed a variety of conceptual, mathematical, and computational models of neuronal networks pretending to implement cognitive functions, such as learning and memory [3,4,5,6,7,8,9]. Systems neuroscience views memory as a substantially complicated paradigm involving different types and forms. Working memory represents one of these types [10]. Just like RAM in computers, it can store several patterns for several seconds to be used ‘on the spot.’ After that, some patterns can be stored in a long-term memory, while others will be completely erased. Working memory is believed to be “encoded” by changes in the strengths of synaptic connections, e.g., synaptic plasticity [4, 11, 12]. These changes determine, which particular neuronal clusters or signal transmission pathways that encode the information should be memorized. When an appropriate clue is applied, the information is retrieved in the form of a spatio-temporal neuronal firing pattern reproducing original information. In modeling, the design of an adequate mathematical model that can possess both biological plausibility and processing functionality is still an open question [13, 14].
Studies conducted in the last decade reveal increasingly more aspects related to the implementation of information functions of the CNS. The list of functions handled by the astrocytic cells keeps getting updated and revised quite frequently [15,16,17,18,19,20]. Several studies discuss the role of astrocytes in the perception of sensory stimuli [21,22,23,24], spatio-temporal coordination of neural network signaling [25,26,27,28,29,30], information processing, and cognitive functions [31,32,33]. A growing number of arguments are accumulating in favor of the theory of continuity and joint coordinated activity of neuron–astrocyte functional networks [34,35,36]. By modulating synaptic transmission, astrocytes act as the third part of the so-called tripartite synapses [37, 38].
1.1 Related works
The morphological and functional connectivity of spiking neural networks (SNN) enhanced by astrocytes determines the features of the biologically motivated computational frameworks. For example, two recent studies demonstrate how a neuron–astrocyte network can improve pattern recognition performed by cortical SNN without retraining [39] and illustrate the self-repairing capability of distributed SNN accompanied by astrocytes in a robotic application [40]. Several attempts to digitally simulate the astrocytic dynamics [41] and neuron–astrocyte interaction [42, 43] indicate that astrocytes could, indeed, be used to solve neurocomputing tasks, which opens a novel fundamental field of research.
A biologically plausible computational model of working memory implemented via SNN interaction with a network of astrocytes was first proposed in our recent work [44]. The astrocytes operate via calcium transients at a much slower time scale of a few seconds by releasing gliotransmitters that modulate synaptic transmission in neurons and, hence, their firing rate. The working memory is associated with item-specific patterns of astrocyte-induced enhancement of synaptic transmission in neuronal networks.
Our work [44], as the majority of conceptual and mathematical models of neuronal memory, operates with binary information. However, the real-world data are analogous, not binary. Initially, the exploitation of binary information patterns in neural networks was the consequence of “digitizing” neuronal signals. These signals are naturally continuous and possess analogous and gradually changing characteristics, such as firing rate, timing, phase. In artificial digital systems, the “black-and-white” (BW) paradigm can be easily enhanced to “colored” (CL) by simple “spatial” scaling—increasing the number of bits. The same cannot be done in the neuronal systems with brain-inspired processing, where such scaling is impossible. The transition from BW to CL dynamics will require conceptual changes in the models. For proper recognition of non-binary (grayscale or color) images, stimuli should be converted into signals of spiking neurons. For example, Kulkarni and Wozniak [45, 46] attempted to do that by stimulating sensory neurons proportional to the intensity of the corresponding pixels. Other studies proposed SNNs for grayscale [47, 48] and color [49] image recognition. However, such SNNs belong to the class of convolutional networks composed of hierarchically stacked convolutional layers. Such networks are trained to contrast the boundaries of objects, which are clearly expressed only in binary images. Thus, the circuit that processes the SNN input signal should contain an algorithm for translating the input image into neural instructions, or the network should have a complex artificial architecture. These factors limit the biological relevance of the models.
Synaptic plasticity represents directed changes of synaptic weights, which either facilitate or depress particular connections. In terms of information encoding, such changes are binary, and their main function is the BW representation of the memorized information. The revealed dependence of the level of calcium elevations generated by astrocytes on neural activity [50] indicates that the astrocytes are involved in the regulation of synaptic transmission [51]. This modulation is gradual and can provide proportional control of the connection efficacy. In other words, astrocytes enable analogous information encoding.
1.2 Problem statement
In this paper, we employ our bioinspired model of SNN accompanied by astrocytes [44] and show how it can reliably store “colored” images for several seconds. To the best of the authors’ knowledge, this is the first time that a spiking neuron–astrocyte network has been shown to be able to simulate a robust analogous memory that can be used in brain-inspired artificial intelligence frameworks. For illustration, we take grayscale images as the information patterns and encode them into the level of input currents of the neuronal layer. By interacting with the astrocyte layer, the patterns can be further stored in the network and maintained during the astrocyte activation interval, e.g., several seconds. During this time, the patterns can be retrieved if an appropriate image, e.g., similar to the original, is fed to the system. After that, the pattern completely disappears, and the network becomes ready to store another image.
2 Colored memory and image recognition in the neuron–astrocyte network model
The neuron–astrocyte network has two interconnected layers: the SNN and the astrocytic network. The SNN is composed of random, sparsely connected excitatory Izhikevich’s neurons [52] with non-plastic synapses arranged in a two-dimensional layer. This layer is interconnected with the astrocytic layer modeled by the Ullah model [53] with local inter-cellular diffusive connections. Each astrocyte bidirectionally communicates with ensembles of \(N_a\) neurons. Astrocytes are activated by coordinated activity of the neighboring neurons, e.g., when an input is applied to the neuronal layer. Astrocytic calcium activation induces gliotransmitter release, which modulates the synaptic transmission in neuronal ensemble corresponding to the astrocyte. This astrocyte-induced synaptic regulation forms spatially distributed clusters of synchronized neurons. The temporal and amplitude characteristics of astrocytic feedback are determined by its calcium dynamics. This biologically relevant mechanism of bidirectional coordination between neuronal and astrocytic activities provides loading, storage, and retrieval of information patterns in the proposed model. The period of information storage in the system is unequivocally determined by the duration of the calcium signals in astrocytes. The neuron–astrocyte network architecture is schematically shown in Fig. 1. Detailed description of the model structure and its parameters is found in our previous paper [44]. The key mathematical details are summarized in Appendix A.
We trained the network to memorize grayscale images. The original 8-bit image (Fig. 2a) was converted to the pattern of input current, \(I^{(i,j)}_{\text {app}}\), (Fig. 2b) and fed to the neuronal layer. Description of the stimulation protocol can be found in A.5. In response to these signals, the neurons fire at different rates depending on the amplitude of the input current (Fig. 2c). Differences in the activity of neural ensembles lead to a variety of Ca\(^{2+}\) events in astrocytes that interact with these ensembles. Figure 2d shows the Ca\(^{2+}\) pattern formed in the astrocytic layer. Such sample-specific distribution of Ca\(^{2+}\) concentration in the astrocytic layer lasts for several seconds.
We assessed the learning performance of the proposed neuron–astrocyte network model by giving it an image recognition problem. For this purpose, we used four test images: the sample image distorted by 80% Gaussian noise (Fig. 3a), by 40% “salt and pepper” noise (Fig. 3c), uniform noise (Fig. 3e), and a new image (Fig. 3g). To illustrate the impact of astrocytes in the image classification task performed by the neuron–astrocyte network, we compared two results of the system recall—with astrocytic modulation of synaptic transmission and without it. Figure 3 shows that the neuronal layer working alone can only repeat the input signal without information processing. The results of four tests performed by the full neuron–astrocyte network model are demonstrated in Fig. 4. Figure 4a, c, e, g contains four types of input test images and Fig. 4b, d, f, h represents the system recalls shown as the mean neuronal firing rate distributions. The proposed neuron–astrocyte network model can recognize and effectively restore the distorted test image. In the first and second tests, in which the network was fed the matching noisy image, our system significantly reduced the excessive noise as shown in Fig. 4b, d. Applying noise (Fig. 4e) or a nonmatching test image (Fig. 4g) to the neuron–astrocyte network results in a nonspecific (Fig. 4f) or chimera-like (Fig. 4h) output.
To evaluate the robustness to noise of the proposed neuron–astrocyte network model, we investigated the dependence of the quality of model retrieval on the noise level in the test image. We used two different types of random noise: “salt and pepper” pulsed noise and Gaussian white noise. This allowed us to examine the ability of our model to remove and reduce noise in an image. In the case of the pulsed noise, the noise pixels could be either 1 or 0, which makes them significantly different from image pixels, which is why the neuron firing rates also differ from the neuronal ensemble. When the noise level is not high, the neuronal correlated activity evokes the astrocyte-mediated feedback which can decrease or increase the firing rates of noise neurons. With Gaussian noise, all pixels of the image change their intensity depending on the noise level. In this case, the astrocyte-induced regulation of synaptic weights restores the general level of activity and synchronization in the neural ensemble. We measured the PSNR between the recalled pattern (e.g., Fig. 4b, d) and the ideal sample image (see Sect. A.6) to create a conventional quality metric of image processing systems. Please note that the maximum possible recall PSNR\(_{\mathrm{max}}\) to the response to the ideal image in the system is 18.295 dB (which is not a very high value) because the resolution of our system is determined by the radius of the interaction of astrocytes with neurons. The results of the measurements are provided in Fig. 5 and Table 1. The PSNR in % denotes the PSNR of recalls related to the PSNR\(_{\mathrm{max}}\). We can see that the neuron–astrocyte network can robustly retrieve a memorized image even with a high level of noise. The model significantly improved the PSNR for pulsed noise and high-intensity Gaussian noise (Fig. 5). The high level of pulsed noise destroys coordinated activity in the neural ensembles which prevents astrocyte-mediated synaptic modulation and, as a result, disturbs the retrieval of formation. Calcium patterns in the astrocytic layer are not frozen and their dynamics is determined by the intracellular biophysical mechanisms. Therefore, the astrocyte-induced feedback and the system recall that depends on it will vary in time. To investigate this, we fed a test image to the system at different moments corresponding to different distribution schemes of calcium pattern amplitudes in the astrocytic layer. Figure 6 shows the dependence of the PSNR recall on the astrocytic calcium dynamics. A greater difference between the amplitudes of calcium pulses in astrocytes leads to an increase in the difference between the activity levels of neural ensembles and, thus, to a better recall bit depth and improved recall quality.
3 Discussion
We demonstrated how astrocytes accompanying neuronal synaptic connections can enhance the capacity of the neuronal network to store and retrieve gradual (analogous) information patterns. Grayscale images were used to stimulate our two-layer neuron–astrocyte network. Corresponding synchronous activation of the astrocytic layer allows the system to store images in the form of astrocyte calcium signal levels during the calcium transients. Furthermore, different levels of calcium were associated with different strengths of modulation of the synaptic connections in the neuronal layer. Consequently, in the neuronal layer, the images appeared in the form of activity patterns with different firing rates. During the storage interval, the system maintained the information and could retrieve it if the appropriate clue was shown in the input. We showed that the retrieval was quite effective even if a noisy clue pattern was shown.
These findings raise an important question: how can we guarantee that almost the same image will reach the input during the characteristic time interval of the calcium events? The key phenomenon which needs to be investigated is the structure of the data flow in real life. The data stream is not an i.i.d. sample taken from the general population. All animals learn to survive in the world with significant local correlations. Thus, we can formulate the local recurrence principle: the probability to receive “almost the same” image reaches its peak at a certain moment, after which the image begins to decay. Moreover, we can even trace several characteristic periods in these decays. The mechanism described in our work allows the astrocyte network to extract the correlations in the time scale of calcium events. Of course, not all the images return in the characteristic time. The proposed mechanism highlights the returned images that can be considered as essential elements of the situation. The duration of information storage in the proposed model is determined by the duration of the elevation in the intracellular Ca\(^{2+}\) concentration in astrocytes. Therefore, in this work, testing of information storage in the form of induced recall is carried out during the time interval during which the Ca\(^{2+}\) signal exists. Sequences of several Ca\(^{2+}\) signals and the heterogeneous calcium events allow the neuron–astrocyte network to capture longer and more complex correlations.
The role of the astrocytes in brain information processing has been intensively debated in neuroscience in recent years [35]. By modulating synaptic transmission, astrocytes can be involved in many computational functions of the brain circuits [33, 36]. Today, we have a variety of experimental facts indicating a similar functional role of astrocytes and neurons in perception processes, for example in the processing of visual stimuli. Along with metabolic, homeostatic, and other supporting functions [58], Muller glia cells in the retina provide the delivery of visual information—light—from the anterior surface of the retina to photoreceptors with minimal losses [59]. Muller cells participate in the structural organization of the retina by creating non-overlapping microdomains that integrate through gap junctions [60]. This organization allows glial subnets to communicate over long distances [61]. It was shown that astrocytes, like neurons, generate calcium signals in response to visual stimuli, with distinct spatial receptive fields and sharp tuning to a visual stimulus [25, 62]. Schumer also discovered a significant overlap of the receptive fields of astrocytes and nearby neuronal cells [62]. Interestingly, sensory stimulation was recently found to be able to evoke astrocytic calcium signals with similar temporal dynamics to neurons [22]. Unlike the neuronal activations, the astrocyte calcium transients are gradual in amplitude [63]. These features indicate that the astrocytes can add an analogous component to the digitized neuronal computations, which can significantly increase the computational power of brain circuits.
The presented result indicates that the spiking neuron–astrocyte network can provide robust analogous information encoding via the astrocytic modulation of synaptic transmission mechanisms. This is a small but important step in ongoing research on the development of brain-inspired artificial intelligence. For instance, the performance and accuracy of neuromorphic computing implemented by spiking neural networks are still behind modern deep-learning networks in most learning tasks [64]. Along with the desire to understand how our brains work, the main reason for the intensified ongoing research efforts in designing brain-like hardware systems that implement neuronal and synaptic computations through spike-driven communication is that it can enable energy-efficient machine intelligence [65]. It is believed that the exploitation of spatio-temporal encoding in SNNs could make the exchange of information more efficient. In this regard, the experimentally and theoretically revealed ability of the astrocytes to evoke the local spatial synchronization in neuronal ensembles due to the activity-dependent short-term synaptic plasticity can become a promising additional feature of training algorithms for SNNs. Another important point that should be stressed is that short-term memory implemented by astrocytes is characterized by one-shot learning and is maintained during the interval of slow astrocytic calcium dynamics. Including the astrocyte-mediated synaptic plasticity in SNN learning algorithms can help achieve better results than deep learning, especially for training on limited data sets.
References
Chaudhuri R, Fiete I (2016) Computational principles of memory. Nat Neurosci 19(3):394–403. https://doi.org/10.1038/nn.4237
Benna MK, Fusi S (2016) Computational principles of synaptic memory consolidation. Nat Neurosci 19(12):1697–1706. https://doi.org/10.1038/nn.4401
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci 79(8):2554–2558. https://doi.org/10.1073/pnas.79.8.2554
Mongillo G, Barak O, Tsodyks M (2008) Synaptic theory of working memory. Science 319(5869):1543–1546. https://doi.org/10.1126/science.1150769
Goldman MS (2009) Memory without feedback in a neural network. Neuron 61(4):621–634. https://doi.org/10.1016/j.neuron.2008.12.012
Zenke F, Agnes EJ, Gerstner W (2015) Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nat Commun 6(1):6922. https://doi.org/10.1038/ncomms7922
Lobo JL, Ser JD, Bifet A, Kasabov N (2020) Spiking neural networks and online learning: an overview and perspectives. Neural Netw 121:88–100. https://doi.org/10.1016/j.neunet.2019.09.004
Lobov SA, Zharinov AI, Makarov VA, Kazantsev VB (2021) Spatial memory in a spiking neural network with robot embodiment. Sensors 21(8):2678. https://doi.org/10.3390/s21082678
Gorban AN, Mirkes YM, Wunsch DC (1997) High order orthogonal tensor networks: information capacity and reliability. Proc Int Conf Neural Netw. https://doi.org/10.1109/icnn.1997.616224
Baddeley A (2012) Working memory: theories, models, and controversies. Annu Rev Psychol 63(1):1–29. https://doi.org/10.1146/annurev-psych-120710-100422
Hansel D, Mato G (2013) Short-term plasticity explains irregular persistent activity in working memory tasks. J Neurosci 33(1):133–149. https://doi.org/10.1523/jneurosci.3455-12.2013
Lundqvist M, Herman P, Miller EK (2018) Working memory: delay activity, yes! persistent activity? maybe not. J Neurosci 38(32):7013–7019. https://doi.org/10.1523/jneurosci.2485-17.2018
Fiebig F, Lansner A (2016) A spiking working memory model based on hebbian short-term potentiation. J Neurosci 37(1):83–96. https://doi.org/10.1523/jneurosci.1989-16.2016
Mi Y, Katkov M, Tsodyks M (2017) Synaptic correlates of working memory capacity. Neuron 93(2):323–330. https://doi.org/10.1016/j.neuron.2016.12.004
Perea G (2005) Properties of synaptically evoked astrocyte calcium signal reveal synaptic information processing by astrocytes. J Neurosci 25(9):2192–2203. https://doi.org/10.1523/jneurosci.3965-04.2005
Kimelberg HK, Nedergaard M (2010) Functions of astrocytes and their potential as therapeutic targets. Neurotherapeutics 7(4):338–353. https://doi.org/10.1016/j.nurt.2010.07.006
Fields RD, Araque A, Johansen-Berg H, Lim S-S, Lynch G, Nave K-A, Nedergaard M, Perez R, Sejnowski T, Wake H (2013) Glial biology in learning and cognition. Neuroscientist 20(5):426–431. https://doi.org/10.1177/1073858413504465
Rusakov DA, Bard L, Stewart MG, Henneberger C (2014) Diversity of astroglial functions alludes to subcellular specialisation. Trends Neurosci 37(4):228–242. https://doi.org/10.1016/j.tins.2014.02.008
López-Hidalgo M, Schummers J (2014) Cortical maps: a role for astrocytes? Curr Opin Neurobiol 24:176–189. https://doi.org/10.1016/j.conb.2013.11.001
Vasile F, Dossi E, Rouach N (2017) Human astrocytes: structure and functions in the healthy brain. Brain Struct Funct 222(5):2017–2029. https://doi.org/10.1007/s00429-017-1383-5
Lines J, Martin ED, Kofuji P, Aguilar J, Araque A (2020) Astrocytes modulate sensory-evoked neuronal network activity. Nat Commun 11(1):3689. https://doi.org/10.1038/s41467-020-17536-3
Stobart JL, Ferrari KD, Barrett MJP, Glück C, Stobart MJ, Zuend M, Weber B (2018) Cortical circuit activity evokes rapid astrocyte calcium signals on a similar timescale to neurons. Neuron 98(4):726–7354. https://doi.org/10.1016/j.neuron.2018.03.050
Reynolds JP, Zheng K, Rusakov DA (2019) Multiplexed calcium imaging of single-synapse activity and astroglial responses in the intact brain. Neurosci Lett 689:26–32. https://doi.org/10.1016/j.neulet.2018.06.024
Chen N, Sugihara H, Sharma J, Perea G, Petravicz J, Le C, Sur M (2012) Nucleus basalis-enabled stimulus-specific plasticity in the visual cortex is mediated by astrocytes. Proc Natl Acad Sci 109(41):2832–2841. https://doi.org/10.1073/pnas.1206557109
Sonoda K, Matsui T, Bito H, Ohki K (2018) Astrocytes in the mouse visual cortex reliably respond to visual stimulation. Biochem Biophys Res Commun 505(4):1216–1222. https://doi.org/10.1016/j.bbrc.2018.10.027
Gordleeva SY, Lebedev SA, Rumyantseva MA, Kazantsev VB (2018) Astrocyte as a detector of synchronous events of a neural network. JETP Lett 107(7):440–445. https://doi.org/10.1134/s0021364018070032
Gordleeva SY, Ermolaeva AV, Kastalskiy IA, Kazantsev VB (2019) Astrocyte as spatiotemporal integrating detector of neuronal activity. Front Physiol 10:294. https://doi.org/10.3389/fphys.2019.00294
Kanakov O, Gordleeva S, Ermolaeva A, Jalan S, Zaikin A (2019) Astrocyte-induced positive integrated information in neuron-astrocyte ensembles. Phys Rev E 99(1):012418. https://doi.org/10.1103/physreve.99.012418
Abrego L, Gordleeva S, Kanakov O, Krivonosov M, Zaikin A (2021) Estimating integrated information in bidirectional neuron-astrocyte communication. Phys Rev E 103:022410. https://doi.org/10.1103/PhysRevE.103.022410
Kanakov O, Gordleeva S, Zaikin A (2020) Integrated information in the spiking-bursting stochastic model. Entropy 22(12):1334. https://doi.org/10.3390/e22121334
Oliveira JF, Sardinha VM, Guerra-Gomes S, Araque A, Sousa N (2015) Do stars govern our actions? Astrocyte involvement in rodent behavior. Trends Neurosci 38(9):535–549. https://doi.org/10.1016/j.tins.2015.07.006
Paukert M, Agarwal A, Cha J, Doze VA, Kang JU, Bergles DE (2014) Norepinephrine controls astroglial responsiveness to local circuit activity. Neuron 82(6):1263–1270. https://doi.org/10.1016/j.neuron.2014.04.038
Santello M, Toni N, Volterra A (2019) Astrocyte function from information processing to cognition and cognitive impairment. Nat Neurosci 22(2):154–166. https://doi.org/10.1038/s41593-018-0325-8
Perea G, Sur M, Araque A (2014) Neuron-glia networks: integral gear of brain function. Front Cell Neurosci 8:378. https://doi.org/10.3389/fncel.2014.00378
Kastanenka KV, Moreno-Bote R, DePittà M, Perea G, Eraso-Pichot A, Masgrau R, Poskanzer KE, Galea E (2019) A roadmap to integrate astrocytes into systems neuroscience. Glia 68(1):5–26. https://doi.org/10.1002/glia.23632
Kofuji P, Araque A (2021) Astrocytes and behavior. Annu Rev Neurosci 44(1):49–67. https://doi.org/10.1146/annurev-neuro-101920-112225
Halassa MM, Fellin T, Haydon PG (2007) The tripartite synapse: roles for gliotransmission in health and disease. Trends Mol Med 13(2):54–63. https://doi.org/10.1016/j.molmed.2006.12.005
Perea G, Navarrete M, Araque A (2009) Tripartite synapses: astrocytes process and control synaptic information. Trends Neurosci 32(8):421–431. https://doi.org/10.1016/j.tins.2009.05.001
Nazari S, Amiri M, Faez K, Hulle MMV (2019) Information transmitted from bioinspired neuron-astrocyte network improves cortical spiking network’s pattern recognition performance. IEEE Trans Neural Netw Learn Syst 31(2):464–474. https://doi.org/10.1109/tnnls.2019.2905003
Liu J, Mcdaid LJ, Harkin J, Karim S, Johnson AP, Millard AG, Hilder J, Halliday DM, Tyrrell AM, Timmis J (2018) Exploring self-repair in a coupled spiking astrocyte neural network. IEEE Trans Neural Netw Learn Syst 30(3):865–875. https://doi.org/10.1109/tnnls.2018.2854291
Soleimani H, Bavandpour M, Ahmadi A, Abbott D (2015) Digital implementation of a biological astrocyte model and its application. IEEE Trans Neural Netw Learn Syst 26(1):127–139. https://doi.org/10.1109/tnnls.2014.2311839
Nazari S, Faez K, Amiri M, Karami E (2015) A digital implementation of neuron-astrocyte interaction for neuromorphic applications. Neural Netw 66:79–90. https://doi.org/10.1016/j.neunet.2015.01.005
Hayati M, Nouri M, Haghiri S, Abbott D (2016) A digital realization of astrocyte and neural glial interactions. IEEE Trans Biomed Circuits Syst 10(2):518–529. https://doi.org/10.1109/tbcas.2015.2450837
Gordleeva SY, Tsybina YA, Krivonosov MI, Ivanchenko MV, Zaikin AA, Kazantsev VB, Gorban AN (2021) Modeling working memory in a spiking neuron network accompanied by astrocytes. Front Cell Neurosci 15:631485. https://doi.org/10.3389/fncel.2021.631485
Kulkarni SR, Rajendran B (2018) Spiking neural networks for handwritten digit recognition—supervised learning and network optimization. Neural Netw 103:118–127. https://doi.org/10.1016/j.neunet.2018.03.019
Woźniak S, Pantazi A, Bohnstingl T, Eleftheriou E (2020) Deep learning incorporating biologically inspired neural dynamics and in-memory computing. Nat Mach Intell 2(6):325–336. https://doi.org/10.1038/s42256-020-0187-0
Lee C, Srinivasan G, Panda P, Roy K (2018) Deep spiking convolutional neural network trained with unsupervised spike-timing-dependent plasticity. IEEE Trans Cogn Dev Syst 11(3):384–394. https://doi.org/10.1109/tcds.2018.2833071
Yu Q, Song S, Ma C, Wei J, Chen S, Tan KC (2021) Temporal encoding and multispike learning framework for efficient recognition of visual patterns. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2021.3052804
Cao Y, Chen Y, Khosla D (2014) Spiking deep convolutional neural networks for energy-efficient object recognition. Int J Comput Vision 113(1):54–66. https://doi.org/10.1007/s11263-014-0788-3
Bindocci E, Savtchouk I, Liaudet N, Becker D, Carriero G, Volterra A (2017) Three-dimensional CA2\(+\) imaging advances understanding of astrocyte biology. Science 356(6339):8185. https://doi.org/10.1126/science.aai8185
Araque A, Carmignoto G, Haydon PG, Oliet SHR, Robitaille R, Volterra A (2014) Gliotransmitters travel in time and space. Neuron 81(4):728–739. https://doi.org/10.1016/j.neuron.2014.02.007
Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Networks 14(6):1569–1572. https://doi.org/10.1109/tnn.2003.820440
Ullah G, Jung P, Cornell-Bell A (2006) Anti-phase calcium oscillations in astrocytes via inositol (1, 4, 5)-trisphosphate regeneration. Cell Calcium 39(3):197–208. https://doi.org/10.1016/j.ceca.2005.10.009
Fellin T, Pascual O, Gobbo S, Pozzan T, Haydon PG, Carmignoto G (2004) Neuronal synchrony mediated by astrocytic glutamate through activation of extrasynaptic NMDA receptors. Neuron 43(5):729–743. https://doi.org/10.1016/j.neuron.2004.08.011
Perea G, Araque A (2007) Astrocytes potentiate transmitter release at single hippocampal synapses. Science 317(5841):1083–1086. https://doi.org/10.1126/science.1144640
Navarrete M, Araque A (2008) Endocannabinoids mediate neuron-astrocyte communication. Neuron 57(6):883–893. https://doi.org/10.1016/j.neuron.2008.01.029
Navarrete M, Araque A (2010) Endocannabinoids potentiate synaptic transmission through stimulation of astrocytes. Neuron 68(1):113–126. https://doi.org/10.1016/j.neuron.2010.08.043
de Hoz R, Rojas B, Ramírez AI, Salazar JJ, Gallego BI, Triviño A, Ramírez JM (2016) Retinal macroglial responses in health and disease. Biomed Res Int 2016:1–13. https://doi.org/10.1155/2016/2954721
Franze K, Grosche J, Skatchkov SN, Schinkinger S, Foja C, Schild D, Uckermann O, Travis K, Reichenbach A, Guck J (2007) Muller cells are living optical fibers in the vertebrate retina. Proc Natl Acad Sci 104(20):8287–8292. https://doi.org/10.1073/pnas.0611180104
Ramírez JM, Triviño A, Ramírez AI, Salazar JJ, García-Sanchez J (1996) Structural specializations of human retinal glial cells. Vision Res 36(14):2029–2036. https://doi.org/10.1016/0042-6989(95)00322-3
Oberheim NA, Takano T, Han X, He W, Lin JHC, Wang F, Xu Q, Wyatt JD, Pilcher W, Ojemann JG, Ransom BR, Goldman SA, Nedergaard M (2009) Uniquely hominid features of adult human astrocytes. J Neurosci 29(10):3276–3287. https://doi.org/10.1523/jneurosci.4707-08.2009
Schummers J, Yu H, Sur M (2008) Tuned responses of astrocytes and their influence on hemodynamic signals in the visual cortex. Science 320(5883):1638–1643. https://doi.org/10.1126/science.1156120
Semyanov A, Henneberger C, Agarwal A (2020) Making sense of astrocytic calcium signals – from acquisition to interpretation. Nat Rev Neurosci 21(10):551–564. https://doi.org/10.1038/s41583-020-0361-8
Shakirov VV, Solovyeva KP, Dunin-Barkowski WL (2018) Review of state-of-the-art in deep learning artificial intelligence. Opt Memory Neural Netw 27(2):65–80. https://doi.org/10.3103/s1060992x18020066
Roy K, Jaiswal A, Panda P (2019) Towards spike-based machine intelligence with neuromorphic computing. Nature 575(7784):607–617. https://doi.org/10.1038/s41586-019-1677-2
Kazantsev VB, Asatryan SY (2011) Bistability induces episodic spike communication by inhibitory neurons in neuronal networks. Phys Rev E 84(3):031913. https://doi.org/10.1103/physreve.84.031913
Esir PM, Gordleeva SY, Simonov AY, Pisarchik AN, Kazantsev VB (2018) Conduction delays can enhance formation of up and down states in spiking neuronal networks. Phys Rev E 98(5):052401. https://doi.org/10.1103/physreve.98.052401
Yamamoto T, Ochalski A, Hertzberg EL, Nagy JI (1990) On the organization of astrocytic gap junctions in rat brain as suggested by LM and EM immunohistochemistry of connexin43 expression. J Comp Neurol 302(4):853–883. https://doi.org/10.1002/cne.903020414
Nagy JI, Rash JE (2000) Connexins and gap junctions of astrocytes and oligodendrocytes in the CNS. Brain Res Rev 32(1):29–44. https://doi.org/10.1016/s0165-0173(99)00066-1
Nimmerjahn A, Kirchhoff F, Kerr JND, Helmchen F (2004) Sulforhodamine 101 as a specific marker of astroglia in the neocortex in vivo. Nat Methods 1(1):31–37. https://doi.org/10.1038/nmeth706
Mitroshina EV, Krivonosov MI, Burmistrov DE, Savyuk MO, Mishchenko TA, Ivanchenko MV, Vedunova MV (2020) Signatures of the consolidated response of astrocytes to ischemic factors in vitro. Int J Mol Sci 21(21):7952. https://doi.org/10.3390/ijms21217952
Gordleeva SY, Stasenko SV, Semyanov AV, Dityatev AE, Kazantsev VB (2012) Bi-directional astrocytic regulation of neuronal activity within a network. Front Comput Neurosci 6:92. https://doi.org/10.3389/fncom.2012.00092
Pankratova EV, Kalyakulina AI, Stasenko SV, Gordleeva SY, Lazarevich IA, Kazantsev VB (2019) Neuronal synchronization enhanced by neuron-astrocyte interaction. Nonlinear Dyn 97(1):647–662. https://doi.org/10.1007/s11071-019-05004-7
Makovkin SY, Shkerin IV, Gordleeva SY, Ivanchenko MV (2020) Astrocyte-induced intermittent synchronization of neurons in a minimal network. Chaos Solitons Fractals 138:109951. https://doi.org/10.1016/j.chaos.2020.109951
Acknowledgements
This research was supported by the Ministry of Science and Higher Education of the RF (Project No. 0729-2020-0061), and by the Grant of the President of the RF for state support of leading scientific schools (Project No. NSh-2256.2022.1.2). IK acknowledges the RFBR Grant No. 19-32-60051 and the President of Russia Scholarship for young Scientists No. SP-3409.2022.5. YT, IK, and SG thank the RFBR Grant No. 20-32-70081. AZ thanks Medical Research Council grant (MR/R02524X/1). Conceptual work of VK and ANG was supported by the Ministry of Science and Higher Education of the RF (Project No. 075-15-2021-634).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflict of interest to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A model details
Appendix A model details
1.1 Spiking neuron–astrocyte network model
The neuron–astrocyte network consists of two layers: the spiking neural network with dimension \(W\times H\) (\(151\times 151\)) and the astrocytic network. The SNN consists of Izhikevich neurons [52] connected by random excitatory synaptic connections. The astrocytic network is \(M\times N\) (\(50\times 50\)) square lattice with only nearest-neighbor connectivity. The dynamics of the intracellular calcium concentration in each astrocyte is described by the Ullah model [53]. The network models a bidirectional neuron–astrocyte interaction. Each astrocyte interacts with \(N_a=16\) neurons located spatially close to it. A graphical representation of the network topology is shown in Fig. 1. The model was integrated using the 4th order Runge–Kutta method with a time step of 0.1 ms. All parameters used in this computational study are given in Table 2 and our previous paper [44]. The code is available at https://github.com/altergot/neuro-astro-network-grayscale.
1.2 Neuronal network
The Izhikevich neuron [52] was chosen as a model to describe the dynamics of each neuron in our network due to its biological relevance and computational efficiency. This model is described by the following differential equations [52]:
with the auxiliary after-spike resetting:
where i, j \((i = \overline{1,W}, j = \overline{1,H})\) are the neural indices, V is the transmembrane potential, t is the time in ms. \(I^{(i,j)}_{\text {app}}\) is the input signal. \(I^{(i,j)}_{\text {syn}}\) is the total synaptic current from all presynaptic neurons \(N^{(i,j)}_{\text {in}}\), which is calculated as follows [66, 67]:
where the parameter \(g^{(i,j)}_{\text {syn}}\) is the synaptic weight: \(g^{(i,j)}_{\text {syn}}=\eta +\nu ^{(m,n)}_{Ca}\), \(\eta\) is the weight of the synaptic connection, \(\nu ^{(m,n)}_{Ca}\) is the astrocyte-induced modulation of the synaptic weight (see Sect. A4). \(E_{\text {syn}}=0\) is the synaptic reversal potential for excitatory synapses. \(V_{\text {pre}}\) is the membrane potential of the presynaptic neuron, \(k_{\text {syn}}\) is the slope of the synaptic activation function. In this model, we do not take into account synaptic and axonal delays.
The architecture of synaptic connections between neurons is random: for each neuron, the number of output connections is fixed and equal to \(N_{\text {out}}\). Thus, the probabilities of the formation of a local and remote synaptic connection are the same.
First, we tested the functionality of our model with the same weights of synaptic connections between all neurons in the neuron–astrocyte network. Differences in the total synaptic input current resulted in some noise in the firing rate response when the original training image was fed. To reduce this effect, at the beginning of the session, we pre-trained the synaptic connections depending on the shades of the training image I:
where \(I^{(i,j)}\) is the pixel shade value of the training source image I from the interval [0; 255] corresponding to the presynaptic neuron (i,j), \(I^{(i^{*},j^{*})}\) is the pixel shade value of the training source image I corresponding to the postsynaptic neuron \((i^{*},j^{*})\). Thus, a small difference in the shades of the pixels of the original training image corresponding to the presynaptic and postsynaptic neurons corresponds to a strong synaptic connection between this pair of neurons. The greater the difference in pixel shades, the weaker the synaptic connection between the corresponding neurons.
1.3 Astrocytic network
Astrocytic dynamics is determined by changes in the concentration of two main substances: inositol 1,4,5-triphosphate (IP\(_3\)) and intracellular calcium (Ca\(^{2+}\)). The main astrocytic intracellular calcium store is the endoplasmic reticulum (ER). Ca\(^{2+}\) can be released from the ER through the membrane channels into the cytoplasm, which corresponds to an increase in intracellular calcium concentration. Ca\(^{2+}\) flux from the ER to the cytoplasm, \(J_{{ER}}\), is a non-linear function of calcium concentration [Ca\(^{2+}\)] and is controlled by the IP\(_3\) concentration. The rate of this flow is determined by the fraction of channels on the ER membrane that are in the open (non-inactivated) state h. The reverse flow of calcium \(J_{\mathrm{pump}}\) from the cytoplasm to the ER is an active transport that pumps calcium back into the ER and is directed to the concentration gradient.
To describe the dynamics of the intracellular [Ca\(^{2+}\)] in each astrocyte (m,n) of our network, we used the Ullah model [53], which qualitatively reflects the main features of the calcium dynamics of astrocyte (for more details about this model and the biophysical meaning of all flows and parameters, see [53]). This model consists of the following differential equations:
where \(J_{\mathrm{leak}}\) is the leakage flux from the ER to the cytosol. The fluxes \(J_{\mathrm{in}}\) and \(J_{\mathrm{out}}\) describe the exchange of calcium with the extracellular space, m,n (m = 1,..., M, n = 1,...,N) are the astrocyte indices. The parameter [IP\(_3^*\)] denotes the steady-state concentration of IP\(_3\), \(J_{\text {PLC}\delta }\) describes the production of IP\(_3\) by phospholipase C\(\delta\) (PLC\(\delta\)), \(J_{{glu}}\) describes the glutamate-induced IP\(_3\) production in response to neural activity. The fluxes are expressed as follows:
Astrocytes form networks by connecting through gap-junctions Cx43 [68,69,70,71]. Diffusion currents of IP\(_3\) molecules and Ca\(^{2+}\) ions, \(\text {diff}_{Ca}\) and \(\text {diff}_{IP_3}\), can be expressed as follows:
where \(d_{Ca}\) and \(d_{IP_3}\) describe the Ca\(^{2+}\) and IP\(_3\) diffusion rates, respectively. In our model each astrocyte is coupled with only four nearest neighbors. \((\Delta [Ca^{2+}])^{(m,n)}\) and \((\Delta [IP_3])^{(m,n)}\) are the discrete Laplace operators:
1.4 Bidirectional neuron–astrocyte interaction
Each astrocyte in the spiking neuron–astrocyte network interacts with a 4 by 4 ensemble of \(N_a\) neurons overlapping in one row. The spiking activity of neurons leads to the release of the neurotransmitter glutamate G from the presynaptic terminal into the synaptic gap. The amount of G that reached the astrocyte is described by the following equation: [72,73,74]:
where \(\alpha _\text {glu}\) is the glutamate clearance constant, \(k\text {glu}\) is the release efficiency, \(\Theta\) is the Heaviside step function, and \(V^{(i,j)}\) is the membrane potential of a neuron (i,j). Glutamate contacts metabotropic glutamate receptors (mGluR) on the astrocyte membrane and initiates the production of IP\(_3\). The \(J_{{glu}}\) variable in the equation describes glutamate-induced IP\(_3\) production and is modeled as:
where \(G_{\mathrm{thr1}}\) is the threshold for the total amount of glutamate G released by all neurons associated with the astrocyte (m,n). \(G_\text {sum}^{(m,n)}\) is the total glutamate G that reached an astrocyte (m,n):
Higher neuronal activity causes more glutamate to be released. This, in turn, leads to longer duration and greater amplitude of the \(J_{\mathrm{glu}}\) elevation. Differences in the \(J_{\mathrm{glu}}\) elevations initiated by the activity of neural ensembles lead to differences in Ca\(^{2+}\) dynamics of astrocytes corresponding to these neurons through IP\(_3\) production. Thus, the larger the amplitude and duration of the \(J_{\mathrm{glu}}\) elevation, the longer and higher-amplitude calcium event it will cause.
The proposed model of spiking neuron–astrocyte network takes into account the following mechanisms of the astrocytic enhancement of excitatory synaptic transmission due to the gliotransmitter action. Astrocytic glutamate-induced (1) potentiation of the synapse through the generation of the slow inward currents (SICs) in the postsynapse [24, 54]; and (2) mGluR-dependent heterosynaptic facilitation of presynaptic glutamate release [55,56,57]. The revealed dependence of the level of calcium elevations generated by astrocytes on neural activity allows astrocytes to gradually regulate synaptic transmission [51]. For simplicity, the relationship between the astrocyte Ca\(^{2+}\) concentration and synaptic weight of the affected synapses \(g_{syn}\), is described as follows:
where \(\nu _{Ca}^*\) is the strength of the astrocyte-induced modulation of the synaptic weight, \(\Theta\) is the Heaviside step function, \([Ca^{2+}]_\mathrm{max}\) is the maximum Ca\(^{2+}\) concentration in the astrocytic layer at the specific moment. Feedback from astrocytes to neurons is activated when \([Ca^{2+}]\) is greater than \([Ca^{2+}]_\mathrm{thr}\), and the total amount of glutamate released by the neurons corresponding to the astrocyte is greater than the threshold: \(G_{{{\text{sum}}}}^{{(m,n)}} > G_{{{\text{thr2}}}}\). The duration of synaptic transmission by astrocytes is fixed and equal to \(\tau _{{{\text{astro}}}}\) according to the experimental data of astrocyte-induced SICs dynamics [54].
1.5 Stimulation protocol
The size of each visual stimulus is equal to the neural network size: \(W\times H\). The original image I was quantized in 256 shades (8-bit image: values from 0 to 255) (Fig. 2a). Then, to train the network, for each of the 256 shades, a value was assigned from a range of linearly spaced values from 4 to 8 (Fig. 2b). Each pixel value was used as the amplitude of the input signal \(I^{(i,j)}_{\text {app}}\) from Eq. (A1) for the corresponding neuron (i,j). Thus, the input signal \(I^{(i,j)}_{\text {app}}\) for a neuron (i,j) was a rectangular pulse with an amplitude \(A_{\text {stim}}\) equal to the pixel (i,j) value and duration \(t_{\text {stim}}\). A detailed list of stimulation and testing parameters can be found in Table 3.
To illustrate how the network can store and retrieve grayscale patterns, we used four images: the same photo with pixel intensities normalized to the range [4; 9] and with an additional 80% Gaussian noise (Fig. 3a), the same photo with pixel intensities normalized to the range [4; 9] and with an additional 40% “salt and pepper” noise (Fig. 3c), uniform noise with values from the range [4; 9] (Fig. 3e), another photo with pixel intensities normalized to the range [4; 9] (Fig. 3g). Test images were also presented as an input signal to neurons with the duration \(t_{\text {test}}\).
The “salt and pepper” noise level (in \(\%\)) is the fraction of noisy pixels. The Gaussian noise level (in \(\%\)) represents the ratio of standard deviations of the white Gaussian noise from the unaltered normalized image.
1.6 Metrics for evaluating retrieval quality
To assess the retrieval quality of the developed neuron–astrocyte network, we used the PSNR method:
where \({\text{MAX}}_{I}\) = 255 is the maximum possible pixel value. To use this method, we converted all the results obtained (mean neuronal firing rate during testing) into 8-bit grayscale images K and compared them with the original image I. We calculated the mean firing rate of each neuron during testing as the mean number of spikes in a time window of 500 ms from the beginning of the test image presentation.
The algorithm of the proposed model operation is schematically summarized in Fig. 7.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Tsybina, Y., Kastalskiy, I., Krivonosov, M. et al. Astrocytes mediate analogous memory in a multi-layer neuron–astrocyte network. Neural Comput & Applic 34, 9147–9160 (2022). https://doi.org/10.1007/s00521-022-06936-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-022-06936-9