Time-domain digital-to-analog converter for spiking neural network hardware

We propose a new digital-to-analog converter (DAC) for realizing a synapse circuit of mixed-signal spiking neural networks. We named this circuit"time-domain DAC (TDAC)". This produces weights for converting a digital input code into voltage using one current waveform. Therefore, a TDAC is more compact than a conventional DAC that comprises many current sources and resistors. Moreover, a TDAC with leak resistance reproduces biological plausible synaptic responses that are expressed as alpha functions or dual exponential equations. We will show numerical analysis results of a TDAC and circuit simulation results of a circuit designed by the TSMC 40 nm CMOS process.

ing analog circuits [5] due to the advantage that highly efficient multiplyaccumulate (MAC) operations can be achieved.
Neuromorphic computation requires many MAC operations for the input signals and synaptic weights, and therefore implementation of highly efficient MAC operations is important for realizing the highly efficient neuromorphic hardware. To improve the energy efficiency of MAC operations, attempts have been made to (i) binarize the synaptic weights, (ii) realize weighted summation using current (wired-sum), (iii) realize weighted summation using capacitors, and (iv) develop a method that combines (i)-(ii).
Bankman et al. [3] realized a highly energy-efficient binary convolution neural network chip to act as a recognition processor. In this circuit, the synaptic weights express binary information and are stored in digital memory. The weighted summation is then calculated by converting the binary information into analog voltages. The capacitors on this chip for weighted summation work also as a digital-to-analog converter (DAC), and this circuit is categorized as a mixed-signal circuit. A mixed-signal circuit that has multi-bit synaptic weights requires a large footprint area and a large amount of energy for the DAC, but the chip developed by Bankman et al. has low power consumption because the synaptic weights are restricted to be binary and the weighted summation is realized by capacitance coupling.
Weighted summation by current is often used in analog neuromorphic hardware [14,5]. In such circuits, the current is produced by synapse circuits. Current-based summation is implemented very simply by connecting each of the metal lines of the synaptic output. In analog neuromorphic hardware, synaptic weights are often expressed by analog voltages that are maintained by capacitors. A circuit with this architecture is highly efficient, but it is difficult to reuse the capacitors that hold the synaptic weights because the former cannot hold the weights for long due to charge leakage. To solve this problem, there have been many studies of synapse circuits that use analog memory [13,15,1,4,6]. However, there are many problems to be solved for establishing analog memory as reliable technology.
To realize highly efficient neuromorphic hardware with reuseable synaptic weights fabricated using conventional complementary metal-oxide-semiconductor (CMOS) technology, a circuit architecture synaptic weights are kept by digital memories and MAC operations are achieved by an analog circuit, i.e. a mixed-signal architecture is suitable. Realizing a mixed-signal circuit that has multi-bit synaptic weights is important for achieving on-chip learning, but it is difficult to realize high-integration and highly efficient neuromorphic hardware because conventional multi-bit DACs comprise many current-source circuits or resistor arrays, thereby necessitating a large footprint and high power consumption [7,9]. It is especially difficult to implement a highly energy-efficient asynchronous spiking neural network (SNN) chip that has multi-bit synaptic weights because using one DAC per time division is difficult in an asynchronous system.
To realize high-integration and highly energy-efficient SNN hardware with on-chip learning, we propose a new DAC circuit that weights each bit of digital memory by using a current (or voltage) waveform. This makes our DAC more compact than a conventional one, and we refer to this DAC as a "time-domain" DAC (TDAC). Herein, we present the results of numerical analysis of TDAC and circuit simulation. This paper is organized as follows: In Section 2, we explain the principle of TDAC, and in Section 3, numerical analysis. Circuit design of TDAC and circuit simulation results are shown in Section 4. Finally, we conclude this paper in Section 6.
2 Circuit principle of time-domain digital-to-analog converter Figure 1(a) shows the TDAC principle in circuit form, and we explain the operation of a four-bit TDAC as an example. The TDAC consists of an analog block and a digital block that comprise AND gates, an OR gate, switches, a switched current source (SCC), resistors, and capacitors . The digital memory values are B 1 -B 4 , and in this example, B 1 and B 4 are the least significant bit (LSB) and the most significant bit (MSB), respectively. S in (t) is the trigger signal for activating the DAC and corresponds to spike pulses when the TDAC is used as the output stage of a synapse circuit. Signals S B,1 (t)-S B,4 (t) are non-overlapping digital signals with pulse width t w for the DAC, and these values are either zero or unity. The SCC outputs a current I out (t) (∝ V non (t)), and the leak resistance R out is an option for realizing the waveform of synaptic potential.
The DAC process using the TDAC without the leak resistance R out is as follows (see Fig. 1 At the same time, V non (t) increases exponentially with time constant C lk R lk . If B 4 is high, then capacitor C out is charged during t w by a current proportional to V non (t). 3) S B,3 (t) is generated at the trailing edge of S B,4 . If B 3 is high, capacitor C out is charged during t w by a current that is proportional to V non (t). 4) Operation with 3) is repeated until S B,1 (t) is generated.
For example, V out,1110 represents the voltage-converted digital input code (1110) 2 (see Fig. 1(b)) that is obtained by charging, charging, charging, and not charging.
Conventional DACs use many resistors or current sources to weight each bit of digital memory. Implementing these circuit components causes that the footprint area must be squared for every unit increase in the length of the memory. By contrast, a TDAC uses a current waveform to weight each memory bit. In the TDAC, the numbers of AND gates, OR gates, and signal generators of S B,k needed to sample the current waveform increases as the length of the memory increases. Therefore, the number of transistors increases linearly, but the disadvantage is that the time taken by the DAC increases. A TDAC is suitable for hardware that must be compact and operate at low speed, such as SNN hardware. In this example, the output current is positive but the TDAC can output negative current.
3 Numerical analysis of time-domain digital-to-analog converter

Time-domain digital-to-analog conversion without leak resistance
We define the time of the trailing edge of S in (t) to be zero, and we express the voltage V out (t) as where q is the memory length, t is continuous time (t ≥ 0), and f scc (·) characterizes the SCC. In this section, we analysis the TDAC when V non (t) is described by V set exp(− t C lk R lk ) and f scc (V non (t)) = V non (t). Under these conditions, Eq. (1) is expressed as We integrate Eq. (2) to obtain  To ensure that the DAC characteristics remain linear, the weight of the upper bit of the adjacent must be twice that of the lower bit, namely We solve Eq. (4) to obtain t w C lk R lk = ln 2.
(5) Figure 2 shows the input-output characteristics of the TDAC obtained from numerical simulation for varying tw C lk R lk . As shown therein, the characteristics are linear when tw C lk R lk = ln 2 but nonlinear otherwise. In particular, monotonicity is also lost when tw C lk R lk < ln 2.

Time-domain digital-to-analog conversion with leak resistance
When a TDAC is used as the output stage of a synaptic circuit, we can view C out as being the membrane capacitance of an analog neuron circuit, where the leak resistance R out is connected in parallel with C out . In this situation, the temporal variation of V out (t) with R out is expressed by We will show that Eq. (6) fits well with biological data on synaptic potentials when all the memory bits are unity and qt w is sufficient large, in which case q−1 k=0 S B,q−k (t)B q−k is nearly unity regardless of t. We assume that V out (0) is zero, and in this case Eq. (6) is expressed as We solve Eq. (7) by using the method of variation of constants, setting C out R out = τ 1 and C lk R lk = τ 2 . The solution of Eq. (7) is given by where g(·) is a function of time. By using Eqs. (6) and (7), dVout(t) dt is expressed as If τ 1 = τ 2 , then the integral of Eq. (9) with respect to t is tV set because g ′ (t) is a constant. In this case, V out (t) is given by which is an alpha function. If τ 1 = τ 2 , then the integral of Eq. (9) with respect to t is where g c is a constant of integration. By using V out (0) = 0, we obtain g c = τ1τ2 τ1−τ2 V set . Substituting g c for g(t), we obtain Substituting Eq. (12) for Eq. (8), we obtain which is a dual exponential function. It is known that alpha and dual exponential functions fit well to biological data on synaptic potentials [8,11,16]. Figure 3 shows synaptic potential waveforms obtained from numerical simulation. Panels (a) and (b) show the waveforms with the alpha function (τ 1 = τ 2 ), and (c) and (d) show those with the dual exponential function (τ 1 = τ 2 ). To obtain panels (a) and (c) and panels (b) and (d), we changed tw τ2 and the digital input code, respectively. In panels (a) and (c), the waveforms are so similar that they cannot be distinguished visually, and the peak of the potential does not change when tw τ2 is varied. By contrast, as shown in panels (b) and (d), when the input code is (10101010) 2 or (01010101) 2 , the peak changes. In the TDAC, C out is not charged during S B,k when memory bit k is zero, and this is caused by alternating charging and leaking. Moreover, the upper bit is converted into analog current faster than the lower one. However, we can ignore the influence of these when qt w is sufficiently smaller than the time constant of the membrane potential of a neuron circuit.

Proposed circuit
A block diagram of our proposed circuit is shown in Fig. 4, where an eight-bit (sign+7) DAC can output negative and positive current. The circuit consists of a digital block and an analog block, the details of which are shown in Fig. 5.
The digital block consists of logic gates and delay circuits, DLs. Each DL consists of 11 transistors that output S B,k with pulse width t w . The pulse width is the adjusted bias voltage V dl . If the bit-length of the TDAC increases, then an additional DL, two NAND gates, a NOT gate, two NMOS transistors, and two PMOS transistors are required, thereby increasing the total number of transistors by 25. The output signal of the digital block is the input for the analog block.
The analog block consists of voltage-controlled current sources, switches, MOS resistors, and MOS capacitors. Negative and positive current outputs are realized by transistors M n and M p , respectively. The number of transistors in the analog block does not increase when the bit-length of the TDAC increases. We explain the operation of positive and negative current output in Sections 4.2 and 4.3, respectively.

Positive current output
The proposed circuit outputs positive current when B 8 is unity. As an example, Fig. 6(a) shows the timing diagram of the input signal and node voltages when the input digital bit code is (11101000) 2 . The process for outputting positive current is as follows.
1) When S in (t) turns high, S rsp (t) and S stp (t) turn high and low, respectively.
At the same time, V gp (t) is set to V rsp , where V rsp is a small voltage. 2) When S in (t) turns off, S B,7 (t) is generated at the trailing edge of S in (t).
At the same time, V gp (t) increases exponentially.  3) S B,6 (t) is generated at the trailing edge of S B,7 . If B 6 is high, then capacitor C out is charged during t w by M p . 4) Operation 3 is repeated until S B,1 (t) is generated.
We prevent wasteful power consumption by setting S stp (t) to low while S in (t) is high. This is because wasteful current flows from M 5 to M 4 if S stp (t) is high while S in (t) is high.

Negative current output
The proposed circuit outputs negative current when B 8 is zero. Figure 6(b) shows the timing diagrams of the input signal and node voltages when the in- put digital bit code is (01101000) 2 . The process for outputting positive current is as follows.
1) When S in (t) turns high, S rsn (t) and S stn (t) turn low and high, respectively. At the same time, V gn (t) is set to V rsn , where V rsn is a high voltage. 2) When S in (t) turns off, S B,7 (t) is generated at the trailing edge of S in (t).
At the same time, V gn (t) decreases exponentially. 3) S B,6 (t) is generated at the trailing edge of S B,7 . If B 6 is high, then capacitor C out is charged during t w by M n . 4) Operation 3 is repeated until S B,1 (t) is generated.
We prevent wasteful power consumption by setting S stn (t) to high while S in (t) is high. This is because wasteful current flows from M 9 to M 10 if S stn (t) is low while S in (t) is high.

Time-domain digital-to-analog conversion without leak resistance
We designed an eight-bit TDAC as shown in Fig. 4 with the TSMC 40 nm CMOS process (1 poly, 8 metal), and we evaluated the circuit by means of the Spectre simulation. We set the bias voltages and the capacitance as V DD = 700 mV, V dl = 180 mV, V lkp = 140 mV, V lkn = 420 mV, and C out = 0.5 pF, and the output voltage V out (t) was reset to 350 mV on every input.   Fig. 6 Timing diagrams of input signal and node voltages for (a) positive and (b) negative current output operations. Figure 7 shows the input-output characteristics of the TDAC when V rsn and V rsp are varied separately. In the designed eight-bit TDAC, the output current is negative when the digital input code is between zero and 128, and it is positive when the digital input code is between 129 and 255. The slopes of the characteristics in the two regions can be adjusted separately by varying V rsn and V rsp as shown in Fig. 7. The energy per one digital-to-analog conversion is 27 fJ when the digital input code is (11111111) 2 and V rsp = 0 mV.

Time-domain digital-to-analog conversion with leak resistance
The circuit simulation for synaptic-potential generation was conducted by adding a MOS resistance between the output node and the ground. We set the bias voltages and the capacitance as V DD = 700 mV, V dl = 340 mV, V lkp = 300 mV, V lkn = 300 mV, V rstn = 570 mV, V rstp = 70 mV, and C out = 0.5 pF. Figure 8 shows the synaptic-potential waveforms for various digital input codes S in (t) and DL outputs. As shown in Fig. 3, which was obtained by numerical simulation, the waveforms are smooth when the lined bit is unity or zero, but in other cases the waveform has multiple peaks. We obtained waveforms that are similar to those from the numerical simulation.

Conclusion
We proposed a new DAC, or a TDAC, in which the weight of each bit that codes for the DAC is realized by a current waveform sampled using nonoverlapping digital signals. The number of transistors needed to implement a TDAC increases linearly with the bit-length, but the transistors can be made small because they work as a digital circuit. A TDAC is therefore more compact than a conventional DAC in which the number of transistors increases  exponentially with the bit-length. Moreover, a TDAC with leak resistance realizes biologically plausible synaptic responses without the need for other circuit components. TDACs are therefore suitable for implementing mixed-signal SNN hardware that requires high integration.
We showed the condition under which a TDAC remains linear, namely that the ratio of the pulse width for sampling the current waveform to the time constant should be ln 2. To realize a TDAC that has good linearity and is robust against fabrication mismatches, the pulse width (resp. time constant) should be set according to the time constant (resp. pulse width), and we intend to develop such a circuit in our future work.