Pavlovian reflex in colloids

Pavlovian reflex is an essential mechanism of nervous systems of living beings which allows them to learn. Liquid colloid computing devices offer a high degree of fault-tolerance, reconfigurability and plasticity. As a first step towards designing and prototyping colloidal neuromorphic computing systems we decided to evaluate if it is possible to implement Pavlovian reflexes. We equate an increase of a synaptic weight with decreased resistance. In laboratory experiments we demonstrated that it is possible to implement Pavlovian learning using just two volumes of colloid liquid.


Introduction
Liquid computers have been known since 1990s when Arnold Emch demonstrated hydraulic extraction of the n th root of any number [1].Later prototypes of liquid computers included hydraulic integrators [2,3], monetary national income analog computers [4], fluid mappers [5], fluid logics [6], Belousov-Zhabotinsky computers [7,8,9,10,11], reaction-diffusion computers [12,13,14,15], fluid maze solvers [16], flow driven droplet logics [17,18,19,20], liquid marbles logic [21], see detailed overview in [22].In these devices a liquid is used to represent signals, actuate mechanical computing devices and to modify signals via chemical reactions.Most prototypes of liquid computers are specialised processors, capable for solving tasks in very narrow domains.The most natural application of a liquid computer is for controlling liquid autonomous systems [23].To generalise applications domains of liquid computers we should embed them into neuromorphic architectures, including learning abilities [24].In [25] we provided an exhaustive analysis of experimental laboratory prototypes where neuromorphic systems are implemented in liquids, colloids and gels.The analysis shown that whilst there are implementations of artificial synapses in liquid phase no cascaded liquid neuromorphic systems have been prototyped.To fill the gap we conducted a series of laboratory experiments on Pavlov's conditional reflexes [26,27] in colloids and produced a first ever prototype of Pavlovian reflex liquid circuit.
The paper is structured as follows.Section 2 introduces an experimental setup.Functioning of the colloid Pavlovian circuit is analysed in Sect.3. Pathways to integration of Pavlovian colloid circuits into liquid neuromorphic systems are discussed in Sect. 4.

Methods
Surfactant solution with a concentration of 0.22 wt% was prepared by adding sodium dodecyl sulphate (SDS, Merc) in de-ionised water (DIW, prepared in the lab with Millipore de-ionised water generator device, model Essential, rated 15 MOhm cm) and stirred to get a homogeneous solution.1 mg ZnO nanoparticles (US Research Nanomaterials, Inc.) were added to DMSO (US Research Nanomaterials, Inc.) with continuous stirring.Then 2 ml of SDS solution and 1 ml NaOH 10M added to the mixture under stirring.The concentration of resulting dispersion was maintained at 0.11 mg/ml.The resulted suspension was placed in an ultrasonic bath for 30 minutes.Then stirring process was continued for a few more hours to get a uniform dispersion of ZnO [28].
All electrical measurements were made with Fluke 88464A and Keithley 2400.Field emission scanning Electron Microscopy (FEI Quanta 650 FESEM) was also used to characterise the nanoparticle suspensions.The accelerating voltage was set to 10 kV in this study, and the working distance was set to around 5 mm.The images' contrast and brightness were optimised so that particles could be easily distinguished from the background.Ultraviolet-visible (UV-vis.)spectrometer (Perkin Elmer Lambda XLS) was used to measure sample absorbance at room temperature.
To examine the colloid's ability to learn the Pavlovian learning algorithm was chosen, which has four steps: ringing bell, feeding food, reaching salivation, repeat the cycle.Based on Pavlov's famous experiment [26,27], ringing the bell by itself will not cause salivation.On the other hand, salivation  appears for food.The dog is repeatedly fed after ringing the bell during the training procedure, which establishes an association between food and ringing.A ringing bell can cause salivation after such training, as shown in condition.The potential mechanism of such learning is shown in Fig. 1.
The axon from a neuron U reacting to unconditional stimuli branches out to establish synaptic connections both on the motor neuron M and synaptic connection from the conditional neuron C to M (Fig. 1a).When neuron U fires the action potential propagates to M and excites but also propagates through synapse C→M thus increasing its synaptic weight (decreasing the synaptic resistance).In designing the colloid Pavlovian circuits our assumptions were that at the onset of learning resistance of a colloid synapse C→M is so high that current passing to M is not high enough to excite M. Simultaneous stimulation of colloids U and C leads to decrease of resistance in C→M to such a degree, that when C is stimulated alone the current reaching M is enough to excited M.This is expressed in an electronic scheme as shown in Fig. 1b.The electronic scheme was converted to experimental setup as shown in Fig. 1c.

Colloid Structural Characteristics
At room temperature, UV-visible absorption spectra of ZnO colloids at three different concentrations are measured in the wavelength range of 200-700 nm.The plots of UV-Visible absorption spectrum are shown in Fig. 2(a).Three peaks in the spectrum are observed at 370 nm to 371 nm for the colloid concentration of 0.011 mg/ml, 0.055 mg/ml, and 0.11 mg/ml.The peaks at 370 nm to 371 nm are the characteristic absorption peaks of hexagonal ZnO nanoparticles [29].According to previous reports [30,31], the results are in good agreement with bulk ZnO (370 nm).Calculation of the optical band gap was based on the below equation: Where E g is the optical band gap, h is the Planck's constant, c is the speed of light,and λ is the wavelength of maximum absorption.Here, the optical band gap was calculated as 3.35 eV, which is consistent with earlier studies [30,32,33].
The morphology and size of particles were studied by FESEM using a thin layer of ZnO colloid prepared by drop casting a drop of as-synthesized colloid

Colloid Learning
The Pavlov's dog experiment was mimicked using a synthesised colloid.To perform Pavlovian learning, first, the resistance of sample (A) and sample (B) were measured (Fig. 3).
Second, a pre-synaptic spike (10 V stimulation) was applied just to sample "A" and the resistance of both samples was measured (mimicking the ringing of the "bell"), shown in Figs.4(a) and 4(b).
Third, a post-synaptic spike (3 V stimulation) was applied just to sample "B" and the resistance of both samples was measured (It acts as Food), Figs 5(a) and 5(b).Salivation is equated to the dropping of the resistance of sample "B".This cycle was repeated 15 times.Then with just applying 10 V stimulation to sample "A" like a bell, as we expected from Pavlovian conditioning, the resistance of sample "B" dropped down (Fig. 6).
Table 1 shows the resistance of sample "B" before performing Pavlovian conditioning, during each step and after finishing cycles.As it can be seen, the resistance of sample "B" dropped down after 15 cycles of experiments   0.11 0.12 0.12 0.12 0.12 0.12 0.13 0.13 0.14 0.15 0.16 0.16 0.16 0.17 0.18 In order to evaluate the effects of Brownian movements and oxygen interactions, we repeated applying the 10 V DC stimulation to sample "A" (mimicking the ringing of the "bell") under Nitrogen gas and at 12 • C. A total of 15 tests were conducted.Figure 7 shows the results of this experiment and compares them with the previous tests in an ambient atmosphere.There is no significant difference between these two atmospheric conditions.Consequently, Brownian movements and oxygen interactions do not appear to affect the learning process of the synthesized colloid.

Discussion
In experimental laboratory conditions we demonstrated that it is possible to implement Pavlovian conditional reflexes with two geometrically constrained colloid volumes.Further cascading of the Pavlovian circuits and their integration into a large scale colloid neuromorphic circuits will require flexible containers and connections.Amongst a plethora of possible scenarios we can select two most feasible approaches.First, we can constrain the colloids into liquid marbles [35].Liquid marbles (LMs) are droplets of liquid coated with hydrophobic power.The LMs behave as soft bodies and are easy to manipulate, they can be arranged into arbitrary geometries.Moreover, LMs can be made with a conductive coating, e.g.Ni particles, thus allowing to make connections between the colloid cargo by simply placing the LMs in physical touch with each other.Feasibility studies on LMs with Belousov-Zhabotinsy cargo, see e.g.[36], shown that computing circuits made of LMs are robust and can function for weeks.Second approach could be in producing the physical structure of a neuromophic device as a network of channels filled with colloid inside a polydimethylsiloxane volume, see e.g.[24,37].Such enclosure will allow for flexibility, robustness and tolerance to environmental conditions.

Conclusion
As the main unit of the nervous system, synapses transmit information between neurons, as well as process learning, memory, and forgetting.In this paper, we demonstrate the synaptic properties of ZnO-based colloids.The colloid showed a comprehensive synaptic function in the Pavlovian associative learning process in the human brain.Artificial intelligence computation, including learning-driven functions, can be achieved using this approach to neuromorphic computing.

Figure 1 :
Figure 1: A scheme of Pavlovian learning: (a) abstract neurons scheme, S1 is unconditional stimulus, S2 is conditional stimulus, U is unconditional neuron, C is conditional neuron, M is a motor neuron, synapses are shown as triangles, (b) electronic real circuit, where the connection wire between samples is modeled as a unique transmission line by a resistor and an inductor in series, the two colloidal suspensions by two memristors and the input signals by a voltage source; (c) Photo of the experimental setup.

Figure 1 :
Figure 1: A scheme of Pavlovian learning: (a) abstract neurons scheme, S1 is unconditional stimulus, S2 is conditional stimulus, U is unconditional neuron, C is conditional neuron, M is a motor neuron, synapses are shown as triangles, (b) electronic real circuit, where the connection wire between samples is modeled as a unique transmission line by a resistor and an inductor in series, the two colloidal suspensions by two memristors and the input signals by a voltage source; (c) Photo of the experimental setup.

Figure 2 :Figure 3 :
Figure 2: Structural characterisation of colloid used in experiments.(a) UV-Visible spectra of the sample with concentrations 0.011, 0.055, and 0.11 mg/ml.(b) SEM images of drop casted ZnO colloids on Copper substrate in different magnifications

Figure 4 :Figure 5 :
Figure 4: (a) Repeated resistance diagrams of sample A, during applying DC stimulation from 0 V to 10 V to sample A, each step 50 mV.(b) Repeated resistance diagrams of sample B, during applying DC stimulation from 0 V to 10 V to sample A, each step 50 mV

Figure 6 :
Figure 6: Resistance diagram of sample A (left) and B (right), during applying DC stimulation from 0 V to 10 V to sample A, each step 10 mV (just bell)

Figure 7 :
Figure 7: (a) Repeated measurement resistance of colloid with DC stimulation from 0 V to 10 V to sample A in the ambient atmosphere.(b) Repeated measurement resistance of colloid with DC stimulation from 0 V to 10 V to sample A in the Nitrogen atmosphere and at 12 • C

Table 1 :
Resistance of sample "B" before performing Pavlovian conditions is about 1.6 MΩ.The resistance of "B" during each step of the stimulation is shown in the table.The resistance of "B" after 15 cycles of stimulation is 0.038 MΩ.