Abstract
Synaptic devices are necessary to meet the growing demand for the smarter and more efficient system. In this work, the anisotropic rhenium disulfide (ReS_{2}) is used as a channel material to construct a synaptic device and successfully emulate the longterm potentiation/depression behavior. To demonstrate that our device can be used in a largescale neural network system, 165 pictures from Yale Face database are selected for evaluation, of which 120 pictures are used for artificial neural network (ANN) training, and the remaining 45 pictures are used for ANN testing. A threelayer ANN containing more than 10^{5} weights is proposed for the face recognition task. Also 120 continuous modulated conductance states are selected to replace weights in our welltrained ANN. The results show that an excellent recognition rate of 100% is achieved with only 120 conductance states, which proves a high potential of our device in the artificial neural network field.
Background
Since the advent of modern computers, the von Neumann structure, wherein the arithmetic unit is separated from the memory, has been widely used. This kind of structure makes data transmission between the arithmetic unit and memory becomes a bottleneck, significantly limiting the improvement of computer performances [1, 2]. Meanwhile, the arithmetic unit and main memory are both volatile devices with high energy consumption, and information will disappear immediately if the power is cut off [3]. In contrast, the human brain is an efficient information storage and computing system with high fault tolerance and low power consumption (about 20 W), and it is based on a highly interconnected, massively parallel, and structurally variable complex network consisted of about 10^{11} neurons and 10^{15} synapses [4, 5]. These neurons are considered to be the brain’s computational engines, receiving input signals from thousands of synapses in parallel. Synaptic plasticity is a biological process that changes synaptic weight through synaptic activities, and it is considered as a source of learning and memory [6].
The twodimension (2D) materials with a small size and excellent electronic properties, such as graphene, transition metal dichalcogenides (TMDCs), and black phosphorus, have attracted significant attention and have been successfully implemented into synaptic devices [7, 8]. The TMDCs with the symmetric lattice, such as MoS_{2} and WSe_{2}, have been widely studied [9, 10]. On the other hand, rhenium disulfide (ReS_{2}) with a distorted octahedral (1T) crystal structure has been rarely explored in the neuromorphic field. Most TMDs have a direct bandgap in the monolayer and an indirect bandgap in the multilayer, so a monolayer material that is difficult to obtain is needed for good device performance. However, ReS_{2} within ten layers are all considered to have a direct bandgap [11], which means ReS_{2} within ten layers can all perform well. Besides, the asymmetric lattice structure leads to weaker interlayer coupling energy, which benefits the exfoliation work, and thus makes the synaptic device much easier to fabricate [12,13,14,15]. In this study, ReS_{2} film is used as a channel material. The crystal structure of monolayer ReS_{2} is shown in Fig. 1a, where directions a and b denote the second shortest axis and the shortest axis in the basal plane, respectively. Based on the previous scientific researches and plenty of optical images of our exfoliated ReS_{2} film [13], direction b denotes the crystallographic orientation with the highest electron mobility. To illustrate the electrical characteristics of our ReS_{2} synaptic device better, direction b is considered as a direction of channel current, as shown in Fig. 1b.
There have been many devices with different structures that successfully simulated synaptic dynamics, such as shortterm plasticity (STP), longterm potentiation (LTP), and longterm depression (LTD) [16,17,18]. A MoS_{2}/PTCDA hybrid heterojunction synapse has been demonstrated with efficient photoelectric dual modulation [10]. A carbon nanotube synapse [19] and siliconbased MoS_{2} synapse [20] showed dynamic logic. However, the mentioned studies focused only on the synaptic level. In some studies, different conductance states were realized to prove that their devices could be used to build artificial neural networks (ANNs), but they did not put the conductive states into the ANNs for calculation [21, 22]. In this work, 120 continuous conductance states are modulated, and the corresponding conductance values are used in the trained face recognition network for calculation; an excellent recognition rate of 100% is achieved.
Methods
The schematic structure of our synaptic device is shown in Fig. 1d, where it can be seen that a 70nm ITO (indium tin oxide) film was deposited on the SiO_{2}/Si substrate as a back gate electrode. The substrate was a Si wafer with 200nm SiO_{2} on top. It was first cleaned with the acetone, isopropyl alcohol, and deionized water, and then dried with N_{2} gas before the ITO deposition. The ITO layer was first deposited by sputtering and then annealed at 400 °C in the N_{2} atmosphere for 10 min by rapid thermal processing (RTP). Transparent ITO electrodes are used in order to accurately fabricate source and drain electrodes using electron beam lithography. The Al_{2}O_{3}/ZrO_{2}/Al_{2}O_{3} sandwiched structures with a thickness of 12 nm, 4 nm, and 4 nm were grown on the ITO by atomic layer deposition (ALD) as a barrier layer, an electron capture layer, and a tunneling layer respectively. Next, the mechanically exfoliated ReS_{2} flakes with a thickness of about 3.6 nm were deposited as a channel under the patterned Ti/Au electrodes. The Ti/Au electrodes with 10nm and 70nm thickness were patterned using the electron beam lithography followed by the electron beam evaporation as a source and a drain, respectively. Figure 1c shows the atomic force microscope image of our 3.6nm thickness ReS_{2} film (about five layers); the channel length was designed to be 1.5 μm (see the inset in Fig. 1b). In this work, the ITO back gate acted as a presynapse neuron, and the Ti/Au electrodes acted as a postsynapse neuron. A small and constant voltage was applied between the source and drain electrodes, while the ITO back gate electrode was applied with pulses to modulate synaptic device performance.
Results and Discussion
Figure 2a shows the transfer characteristics of our synaptic device at a 2V back gate voltage (V_{bg} = 2 V) under a fixed draintosource voltage (V_{ds}) changing from 100 to 700 mV with the step of 100 mV. An On/Off current ratio over 10^{6} could be observed. The curve displayed the draintosource current (I_{ds}), which first increased rapidly and then became saturated; the excellent saturation characteristics corresponded to the strong channel regulation by the ITO back gate electrode. Unlike the traditional transistors, which use silicon as a bottom gate electrode and SiO_{2} as a dielectric at the operation voltage of usually more than 20 V [23], the operation voltage of our synaptic device with only a 20nm distance between the ReS_{2} channel and ITO back gate electrode was below 5 V, significantly improving the efficiency of synaptic device. The inset in Fig. 2a shows the superlinear relationship under the lowV_{ds} regimes, which demonstrates a good Schottky contact between the ReS_{2} channel and source and drain electrodes. As shown in Fig. 2b, I_{ds}–V_{bg} hysteresis curve could be observed when V_{bg} changed from − 5 to 5 V and then reversed back at a constant bias of 0.1 V (V_{ds} = 0.1 V). In the measurements, a small constant voltage of 0.1 V was applied between the source and drain electrodes to “read” the postsynaptic current. The memory window, which provided the basis for synaptic performance, was about 3.5 V; such a big memory window made our ReS_{2} device very promising for synaptic applications [24]. Since the top of the valence band of ZrO_{2} was higher than that of Al_{2}O_{3}, and the bottom of the conduction band was lower than that of Al_{2}O_{3} (see the inset in Fig. 2c), ZrO_{2} used as an intermediate layer sandwiched between alumina could capture charge effectively. The energy band diagrams under positive and negative back gate voltage are shown in Fig. 2c and d, respectively. When a positive voltage was applied, electrons in the ReS_{2} channel would first tunnel through the Al_{2}O_{3} tunneling layer, then be captured by the ZrO_{2} trapping layer. On the contrary, when ITO was applied with a negative voltage, electrons gathered in the ZrO_{2} layer would be sent to the ReS_{2} channel; the energy bands bent in the direction of the channel.
In Fig. 3a, a typical excitatory postsynaptic current (EPSC) was detected after applying a negative input pulse (with the amplitude of − 1 V and duration of 10 ms) at the ITO back gate. Also, an inhibitory postsynaptic responded to a positive voltage pulse (with the amplitude of 1 V and duration of 10 ms) was observed in Fig. 3b, which is similar to a biological synapse [25]. The pulse signal from the presynapse neuron was transmitted to the postsynapse neuron through the synapse and converted into the postsynaptic current (PSC) [26]. The PSC value was determined by pulse amplitude and duration. When the pulse was negative, the electrons from the defects of ZrO_{2} gained enough energy to tunnel through the upper Al_{2}O_{3} dielectric layer into the ReS_{2} channel. The constant value of the current was slightly higher than the previous value (∆PSC = 0.04 nA) and could maintain for a long time. This phenomenon corresponded to the longterm potentiation (LTP) in the biological synapse. However, when the pulse was positive, electrons in the ReS_{2} channel tunneled through the Al_{2}O_{3} layer under the attraction of the electric field and were captured by the defects of ZrO_{2}. Thus, the constant value of the current was slightly lower than the original value and could maintain the same for a long time (∆PSC = 0.06 nA). This process corresponded to the longterm depression (LTD) in the biological synapse. The LTP and LTD provided a physiological substrate for learning and memory in synaptic devices. When the negative pulses with the amplitude of − 2 V and duration of 10 ms were applied continuously, with a 1s interval between pulses, the rising current in the two steps was observed, as shown in Fig. 3c. The rising current values were 1.6 nA and 1.4 nA, respectively. Therefore, a continuous and uniformly rising current could be obtained under the periodic gate voltage pulses, and the steady current after stimulation could last for a long time, as shown in Fig. 3d. This finding provided a basis for obtaining the multiple stable conductive states.
Figure 4a shows 120 current values after applying 120 negative pulses with an amplitude of − 2 V and a duration of 10 ms and with a 1s interval between pulses. Apparently, the current curve showed excellent linearity, 120 effective highstability conductance states were obtained in each state. Different conductance states corresponded to different ANN weight values [27].
In this work, a threelayer artificial neural network for face recognition task is proposed, and its structure is presented in Fig. 4b, wherein it can be seen that the input layer consists of 1024 neurons that correspond to 1024 pixels of an image, the middle (hidden) layer consists of 256 neurons, and the output layer consists of 15 neurons that correspond to 15 classes of faces.
The development of the proposed ANN is as follows. A total of 165 pictures, including 15 types of pictures from Yale Face database [28] are used for ANN training and testing. Eight images of each type are used for ANN training, and the remaining three images of each kind are used for ANN testing. Given that the modules are smooth functions relative to their inputs and their internal weights, the multilayer architectures can be trained by simple stochastic gradient descent, and the gradients are generally computed by the backpropagation procedure [29]. Therefore, we use the classical backward propagation (BP) algorithm to build our network and show how the BP algorithm works for our ANN.
In this work, X_{m} represents an input neuron, so the input value of a hidden neuron can be expressed as:
where V_{mn} represents the weight value between an input neuron X_{m} and a hidden neuron Y_{in}, and all V_{mn} form the matrix V having a total of 1024 × 256 weight values; the initial value of this matrix is randomly assigned. The activation function of the hidden layer is the sigmoid function, so the output value of a hidden neuron is given by:
Thus, the input value of an output neuron can be expressed as:
where W_{nk} represents the weight value between a hidden neuron Y_{on} and an output neuron Z_{ik}, and all W_{nk} form the matrix W with a total of 256 × 15 weight values; the initial value of W_{nk} is also randomly assigned. Besides, we use the sigmoid function as an activation function of the output layer, so that the output value of an output neuron is given by:
Comparing the abovecalculated output with the correct output, the total output error can be obtained, and it is expressed as:
where O_{k} is the correct output value. So far, the forward propagation process of the network has been completely described. To improve the recognition rate, the backpropagation process is needed to calculate the errors of the weights, and they are used to update the network weights in the next iteration.
In the above mathematical expressions, ∆V_{mn} and ∆W_{nk} respectively represent the errors of V_{mn} and W_{nk}; after adding the errors to the original weight, we get the updated weight V_{mn}^{′} and W_{nk}^{′}; μ is the learning rate, and μ = 0.06. After updating the weights, a new image is fed to the ANN, and the weight update process is repeated until all 120 images have been used for training. Next, we use the trained network to identify the remaining 45 images and calculate the recognition rate. The ANN testing process requires only the forward propagation process. Each image used for testing gets 15 output values after a forward propagation. The output value reflects the probability that the input image is of a certain type. The output with the maximum probability value is selected, and the corresponding type is the type of the input picture identified by the network. The recognition results are compared with the standard output; all correctly identified pictures are counted, and their total number is n. In each trainingrecognition cycle, the recognition rate r is given by:
Generally, the recognition rate of the first recognition is very low, and in our ANN with 256 hidden neurons, the first recognition rate is only 17.78%. The above trainingrecognition process is repeated until the maximum recognition rate is obtained. The whole trainingrecognition cycle is shown in Fig. 4c.
As shown in Fig. 5a, during the ANN development process, the maximum recognition rate and rising speed of recognition rate (training speed) were different at a different number of hidden neurons. A larger number of hidden neurons led to a higher maximum recognition rate and a faster rising speed, but also increased energy consumption, so certain tradeoff should have to be made. In the case of 256 hidden neurons, the recognition rate reached 100% after 600 iterations of training, as shown in Fig. 5b. Since this was definitely the maximum recognition rate that could be achieved, in our ANN, we set the number of hidden neurons to 256. The distribution of weight values after different trainingtesting cycles is presented in Fig. 5c, and it indicates that the weights became more scattered after more cycles, that is to say, to reach a higher recognition rate, the weights in the ANN had to be adjusted. Once we achieved the maximum recognition rate, the matrices V and W having the optimal weight value were obtained. To demonstrate better that our ReS_{2} device is suitable to be applied to ANNs, all weight values in the weight matrices V and W were replaced by device’s conductance values. We used I_{j}(j = 1, 2, 3⋯120) to represent 120 conductance values that were obtained after 120 cycles, and we made a linear transformation of the original conductance values so that conductance range was consistent with the weight range, which was given by:
where C_{j} represented the weight value after the linear transformation. In the case of 600 cycles, the linear transformation coefficients were A = 1.3769 × 10^{10} and B = − 65.784. Next, we subtracted each C_{j} from each weight value and replaced the weight value with C_{j} that had the smallest absolute value after subtraction; namely, we calculated minV_{mn} − C_{j}, minW_{nk} − C_{j} and replaced each weight value with the corresponding C_{j}. In this way, we obtained new V and W weight matrices wherein all the weight values were replaced by C_{n}. Then, we used our new weight matrices in ANN testing, and the ANN recognition rate of 100% was achieved, which proved that our 120 conductance states could be perfectly used as weight values in the ANN. For the purpose of further analysis, we replaced the weight values after 100~500 training cycles (in steps of 100), and the identification results obtained after the replacement are completely consistent with the original one, as shown in Fig. 5d. This proves that these 120 current values could perfectly replace over 10^{5} weight values for calculation. By further increasing the number of gate pulses, more conductance states could be obtained, which proved that our ReS_{2} device could be used in a largescale neural network system.
Conclusions
In this work, we introduce a highk dielectric stack based 2D ReS_{2} synaptic device and demonstrate some fundamental synaptic behaviors such as longterm potentiation and longterm depression. The results show that our ReS_{2} device can simulate synaptic performance well. Also, an ANN is constructed to prove the application of the proposed device in artificial neural networks. Applying 120 periodic gate voltage pulses, 120 effective, clearly distinguished conductance states are obtained, and they are used to replace more than 10^{5} weights in the ANN for face recognition. The recognition rate of 100% is achieved after replacement. This excellent result demonstrates that our ReS_{2} synapse can be used to build an artificial neural network.
Availability of Data and Materials
The authors declare that the materials, data, and associated protocols are available to the readers, and all the data used for the analysis are included in this article.
Abbreviations
 2D:

Twodimension
 ALD:

Atomic layer deposition
 ANN:

Artificial neural network
 LTD:

Longterm depression
 LTP:

Longterm potentiation
References
Popper KR (1968) Birkhoff and von Neumann’s interpretation of quantum mechanics. Nature 219:682
Cheng S, Jheng S (2016) Physical realization of von Neumann lattices in rotating Bose gases with dipole interatomic interactions. Sci RepUk 6:31801
MacPherson RD, Srolovitz DJ (2007) The von Neumann relation generalized to coarsening of threedimensional microstructures. Nature 446:1053
Cole MW, Bassett DS, Power JD, Braver TS, Petersen SE (2014) Intrinsic and taskevoked network architectures of the human brain. Neuron 83:238–251
Churchland MM, Cunningham JP, Kaufman MT, Foster JD, Nuyujukian P, Ryu SI, Shenoy KV (2012) Neural population dynamics during reaching. Nature 487:51
Burnstock G (2007) Physiology and pathophysiology of purinergic neurotransmission. Physiol Rev 87:659–797
Tian H, Guo Q, Xie Y, Zhao H, Li C, Cha JJ, Xia F, Wang H (2016) Anisotropic black phosphorus synaptic device for neuromorphic applications. Adv Mater 28:4991–4997
Liu C, Yan X, Song X, Ding S, Zhang DW, Zhou P (2018) A semifloating gate memory based on van der Waals heterostructures for quasinonvolatile applications. Nat Nanotechnol 13:404–410
Wang Y, Cong C, Yang W, Shang J, Peimyoo N, Chen Y, Kang J, Wang J, Huang W, Yu T (2015) Straininduced directindirect bandgap transition and phonon modulation in monolayer WS 2. Nano Res 8:2562–2572
Wang S, Chen C, Yu Z, He Y, Chen X, Wan Q, Shi Y, Zhang DW, Zhou H, Wang X (2019) Others: A MoS2/PTCDA hybrid heterojunction synapse with efficient photoelectric dual modulation and versatility. Adv Mater 31:1806227
Mak KF, Lee C, Hone J, Shan J, Heinz TF (2010) Atomically thin MoS 2: a new directgap semiconductor. Phys Rev Lett 105:136805
Jadczak J, KutrowskaGirzycka J, Smole N, Ski T, Kossacki P, Huang YS, Bryja L (2019) Exciton binding energy and hydrogenic Rydberg series in layered ReS 2. Sci RepUK 9:1578
Liu E, Fu Y, Wang Y, Feng Y, Liu H, Wan X, Zhou W, Wang B, Shao L, Ho C (2015) Others: Integrated digital inverters based on twodimensional anisotropic ReS 2 fieldeffect transistors. Nat Commun 6:6991
Tongay S, Sahin H, Ko C, Luce A, Fan W, Liu K, Zhou J, Huang Y, Ho C, Yan J (2014) Others: Monolayer behaviour in bulk ReS 2 due to electronic and vibrational decoupling. Nat Commun 5:3252
De Sanctis A, Amit I, Hepplestone SP, Craciun MF, Russo S (2018) Strainengineered inverse chargefunnelling in layered semiconductors. Nat Commun 9
Prezioso M, MerrikhBayat F, Hoskins BD, Adam GC, Likharev KK, Strukov DB (2015) Training and operation of an integrated neuromorphic network based on metaloxide memristors. Nature 521:61
Hu SG, Liu Y, Liu Z, Chen TP, Yu Q, Deng LJ, Yin Y, Hosaka S (2014) Synaptic longterm potentiation realized in Pavlov’s dog model based on a NiOxbased memristor. J Appl Phys 116:214502
Yu S, Wu Y, Jeyasingh R, Kuzum D, Wong HP (2011) An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation. IEEE T Electron Dev 58:2729–2737
Kim K, Chen C, Truong Q, Shen AM, Chen Y (2013) A carbon nanotube synapse with dynamic logic and learning. Adv Mater 25:1693–1698
Jiang J, Guo J, Wan X, Yang Y, Xie H, Niu D, Yang J, He J, Gao Y, Wan Q (2017) 2D MoS2 neuromorphic devices for brainlike computational systems. Small 13:1700933
Wang ZQ, Xu HY, Li XH, Yu H, Liu YC, Zhu XJ (2012) Synaptic learning and memory functions achieved using oxygen ion migration/diffusion in an amorphous InGaZnO memristor. Adv Funct Mater 22:2759–2765
Wang T, He Z, Liu H, Chen L, Zhu H, Sun Q, Ding S, Zhou P, Zhang DW (2018) Flexible electronic synapses for face recognition application with multimodulated conductance states. Acs Appl Mater Inter 10:37345–37352
Tian H, Mi W, Wang X, Zhao H, Xie Q, Li C, Li Y, Yang Y, Ren T (2015) Graphene dynamic synapse with modulatable plasticity. NANO LETT 15:8013–8019
Arnold AJ, Razavieh A, Nasr JR, Schulman DS, Eichfeld CM, Das S (2017) Mimicking neurotransmitter release in chemical synapses via hysteresis engineering in MoS2 transistors. Acs Nano 11:3110–3118
Bean BP (2007) The action potential in mammalian central neurons. Nat Rev Neurosci 8:451
Shi Y, Kirwan P, Smith J, Robinson HP, Livesey FJ (2012) Human cerebral cortex development from pluripotent stem cells to functional excitatory synapses. Nat Neurosci 15:477
De Domenico M, Nicosia V, Arenas A, Latora V (2015) Structural reducibility of multilayer networks. NAT COMMUN 6:6864
Belhumeur PN, Hespanha JAOP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis \& Machine Intelligence:711–720
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436
Acknowledgements
We thank Ms. Jing Xu at the Fudan University for the guidance on device fabrication.
Funding
This work was supported by the NSFC (61704030 and 61522404), Shanghai RisingStar Program (19QA1400600), the Program of Shanghai Subject Chief Scientist (18XD1402800), and the Support Plans for the Youth TopNotch Talents of China.
Author information
Authors and Affiliations
Contributions
ZHF and LC prepared the ReS_{2} synaptic devices and designed the electrical measurement method of synaptic plasticity. MZ and LRG helped exfoliate ReS_{2} flakes. LC, HZ, QQS, and DWZ supervised the whole work. All authors critically read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Fan, ZH., Zhang, M., Gan, LR. et al. ReS_{2} Charge Trapping Synaptic Device for Face Recognition Application. Nanoscale Res Lett 15, 2 (2020). https://doi.org/10.1186/s116710193238x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s116710193238x
Keywords
 Charge trapping memory
 Synaptic device
 Twodimension material
 Artificial neural network