1 Introduction

Reservoir computing [1, 2] is one of the most popular paradigms in recent machine learning and is especially well-suited for learning sequential dynamic systems. Even when systems display chaotic [3] or complex spatiotemporal phenomena [4], which are considered as exponentially complicated problems, an optimized reservoir computer can handle them efficiently.

On the other hand, von Neumann-type architecture is known as one of the most familiar computer architectures, which consists of a processing unit, a control unit, memory, external data storage, and input/output mechanisms. While such architecture is now widely utilized, in the conventional von Neumann-type architecture, bottlenecks between the processing unit and memory are inevitable when implementing the parallel operation of sequential processing [5]. Hence, the development of a physical reservoir using various physical phenomena that can act as a non-von Neumann-type architecture is required. Unlike other types of neural network models, reservoir models are expected to be suitable for physical implementation as their inner dynamics are fixed as a general definition and do not need to be modified during processing. Thus far, various methods for the physical implementation of reservoirs, such as electrochemical cells [5], analog VLSI [6], and memristive nanodevices [7], have been actively discussed.

We focused on the energy propagation between dispersed quantum dots (QDs) as a phenomenon for implementing a physical reservoir. QDs are nanometer-sized structures that confine the motion of charge carriers in all three spatial directions, leading to discrete energy levels based on quantum size effects. In general, the emission properties of QD can be adjusted by changing their sizes and structures. In addition, since QDs are typically fabricated using semiconductor materials, they exhibit good stability and durability. Recently, QDs have been incorporated into semiconductor devices, such as light-emitting diodes [8, 9], lasers [8, 10], and field-effect transistors [11]. Our QD reservoir (QDR) consists of randomly connected transfer paths of optical energy between the QDs and reveals the spatiotemporal variation in the fluorescence output. Recently, we experimentally demonstrated short-term memory capacity as a physical reservoir [12].

In this Chapter, we have discussed the fundamentals of QDR and the experimental protocol for preparing QDR samples. Moreover, the results of actual reservoir computing based on the spatiotemporal fluorescence outputs of QDR samples have been shown. Generally, a reservoir requires large amounts of computational resources during the learning process. Using the spatiotemporal data obtained from the nonlinear transformation from an optical input signal to the fluorescence output via QDR, learning without calculating the individual states of all the nodes in the reservoir layer is possible. As a result, by physically implementing the reservoir layer and learning based on spatiotemporal data, we expect to solve the target problem with a lower reservoir power consumption than other existing implementations.

2 Quantum-Dot-Based Physical Reservoir

2.1 Basics

As shown in Fig. 1, a QDR consists of randomly dispersed QDs. The optical input to the QDR was determined by the incidence of the laser pulse, and some QDs were excited by this incident light. While the excited electron energy behaves as a localized optical energy, it can be directly relaxed to the ground energy level with/without fluorescence radiation. Another typical phenomenon of such a localized optical energy transfer, based on the Förster resonance energy transfer (FRET) mechanism, is that a QD, initially in its electronic excited state, may transfer energy to a neighboring QD through nonradiative dipole–dipole coupling. Generally, the fluorescence and absorption spectra of QD partially overlap; thus, FRET is probabilistically allowed. Furthermore, if the FRET network consists of different types of QDs, each type of QDs is defined as a donor or acceptor. FRET from an acceptor-QD to a donor-QD is prohibited, and an acceptor-QD often acts as a destination in each network. During the energy transfer, optical energy percolates in the FRET network, which can be regarded as the optical input being partially memorized with subsequent forgetting.

Fig. 1
figure 1

Schematic of the inner dynamics of a QDR by sequential incidence of light pulses occurring due to localized transfer and temporal stagnation of optical energy during periodic irradiation of light pulses

In particular, in our scheme for realizing the optical input/output of QDR, the light pulses are spatially incident on the QD network in parallel and excite multiple QDs. For the excitation of the QDs, the optical energy of the input light must be greater than the bandgap energy of each QDs. The excited QDs probabilistically emit fluorescent photons with optical energy similar to the bandgap energy. In contrast, part of the optical energy is transferred from one excited QD to another QD based on the FRET mechanism. After single or multiple steps of energy transfer, some of the optical energy in the network is emitted from the destination QD. Generally, light emissions via multiple energy transfers necessarily occur later than that involving no energy transfer. Consequently, the fluorescence output from the QDR can be sequentially obtained by sparsely counting fluorescence photons.

Additionally, as shown in the lower half of Fig. 1, when the next light pulse is input to the QDR during the temporal stagnation of the inner optical energy, the transfer process and corresponding emission of fluorescence photons reveal a different tendency from the previous condition of the network because of the different internal state of the QDR from the previous condition; namely, unlike the upper half of Fig. 1, some QDs are already excited. Here, a saturated situation in which all the QDs are excited is not anticipated. Consequently, in such cases, the optical outputs in response to sequential optical inputs cannot be predicted using a simple linear sum of a single input/output. Therefore, the non-linearity of an input/output can be quantitatively evaluated by comparing the linear sum of a single input/output. In other words, such a setup works as a fusion-type setup of the processor and storage device, which maintains the inner states during multiple optical inputs and outputs and can directly read out its output as fluorescence via complicated signal processing in the network.

2.2 Experimental Demonstration: Randomly-Dispersed QDR

For the experimental preparation of QDR sample, we used two types of QDs as components of the QD network: CdS dots (NN-labs, CS460; peak wavelength of the emission: 465–490 nm, 3.0 nmol/mg, represented in catalog) and CdS/ZnS dots (NN-labs, CZ600; peak wavelength of the emission: 600–620 nm, 1.0 nmol/mg, represented in catalog) with toluene solutions. In this case, the CdS and CdS/ZnS dots acted as donors and acceptors, respectively. Additionally, polydimethylsiloxane resins (PDMS; Dow Corning, Sylgard184) were used as base materials to fix and protect the inner QDR. The basic procedure for preparing a QD sample using these materials is shown in Fig. 2a and is as follows.

Fig. 2
figure 2

a Schematic of experimental process. b Appearance of three QD samples under UV light illumination

First, the two QD solutions were mixed with 1,000 μL of PDMS base solution. To control the configuration of the QDR, the CdS and CdS/ZnS dots were mixed in ratios of 3:1, 1:1, and 0:1 for Samples A, B, and C, respectively. After mixing, each mixture was heated for evaporating toluene. Then, 100 μL of polymerization initiator PDMS was added to the mixture, and the resulting solution was dropped on the cover glass. The mixture was spread on a cover glass using a spin coater (MIKASA, MS-B100, rotational speed: 3,000 rpm) for 100 s to randomly disperse the QDs in the resin, and the respective QDR was expected to be formed. The assumed thickness of the samples was less than 1 μm. After the mixtures were thinned, to fix the alignments of the QDs in each mixture, the thinned samples were heated on a hot plate (EYELA, RCH-1000) at 150 °C for 600 s. The prepared samples appeared transparent under ambient light; however, they emitted fluorescence under UV illumination, as shown in Fig. 2b.

2.3 Experimental Demonstration: Electrophoresis

Electrophoretic deposition (EPD) was applied as another method to prepare QDR sample [13,14,15,16]. The EPD of the QD layer was accomplished by applying a voltage between two conductive electrodes suspended in a colloidal QD solution. The electric field established between the electrodes drives QD deposition onto the electrodes. EPD, as a manufacturing process, efficiently uses the starting colloidal QD solutions.

To start the EPD process, two GZO glasses coated with a Ga-doped ZnO layer were secured 1.0 cm apart with their conductive sides parallel and facing one another, as illustrated in Fig. 3a. An electric field of 15 V/cm was then applied, and the electrodes were placed in the QD solution. After a few minutes, the electrodes were QD-rich toluene droplets that left uneven QD deposits after being dried from the surface using compressed air. In this demonstration, we used CdSe/ZnS dots (Ocean Nanotech, QSR580; peak wavelength of emission: 580 nm, 10 mg/mL, represented in the catalog) with a toluene solution. Owing to their emission and absorption spectra, the CdSe/ZnS dots can function as both donors and acceptors. Figure 3b, c show the appearance of the negative and positive electrodes under UV light illumination, respectively. As the fluorescence of the QD layer can be observed by the eye only on the negative electrodes under UV light, the QD layer was confirmed to be deposited by the EPD process and not by any other phenomenon.

Fig. 3
figure 3

a Schematic of the EPD process. A voltage applied between two parallel, conducting electrodes spaced 1.0 cm apart drives the deposition of the QDs. Appearance of b negative and c positive electrodes after 60 min of EPD process under UV light illumination

Here, we prepared three samples using the EPD method under 10, 20, and 60 min of deposition time, which we define as 10, 20, and 60 M samples, respectively. Figure 4a–c show fluorescence microscopic images under UV irradiation. As shown, the coverage rate of each sample by deposited-QDs was varied from 49.8 to 76.1% by increasing deposition time from 10 to 60 min. Atomic force microscopy (AFM) topography images of the QD layers on the GZO anodes are shown in Fig. 5a–c. The resulting QD layer apparently consisted of aggregated QD components, and the size of each unit component was 100–500 μm. The number of QD components increased with an increase in the deposition time.

Fig. 4
figure 4

Fluorescence microscopic images of the surface of an electrophoretically deposited QD layer on a 10 M sample, b 20 M sample, and c 60 M sample, respectively

Fig. 5
figure 5

AFM topography images of the surface of an electrophoretically deposited QD layer on a 10 M sample, b 20 M sample, and c 60 M sample, respectively

3 Non-linearity of the QD Reservoir

3.1 Experimental Setup

We focused on the fluorescence relaxation from acceptor QDs for verifying the non-linearity of our QDR. Here, photons emitted from each QD were necessarily output at various times, regardless of whether the photons are emitted via energy transfers. In other words, the results of photon counting, which are triggered by the timing of the optical input, indicate the characteristics of QDR.

For experimental verification, we used a Ti:Al2O3 laser (Spectra Physics, MaiTai), which emitted optical pulses with a pulse length of 100 fs, an optical parametric amplifier (Spectra Physics, TOPAS-prime), and a wavelength converter (Spectra Physics, NirUVis) as the light sources for irradiating the QD samples. The oscillation frequency and wavelength were set as 1 kHz and 457 nm, respectively. The laser power and polarization were appropriately controlled for exciting the QD samples and effectively counting the fluorescent photons, as shown in Fig. 6.

Fig. 6
figure 6

Schematic of experimental photon-counting setup to identify characteristic of the QDR sample

The delay line generated a time lag \(\Delta t\) between the first and second pulses incident on the QDR sample. The range of \(\Delta t\) was set at 0.64–7.4 ns, and the corresponding optical length was controlled by a stage controller driven by a stepping motor with a position resolution of 20 μm/step. Fluorescence photons induced by optical excitation of the QDR samples passed through the focusing lens again and were reflected in the detection setup using a Glan–Thompson polarizer. After passing through a bandpass filter (UQG Optics, G530, transmission wavelength: 620 ± 10 nm), the fluorescent photons were propagated to a photon detector (Nippon Roper, NR-K-SPD-050-CTE). Since the irradiated light passed through the polarizers, only the fluorescence photons from the acceptor-QDs were selectively obtained by the single-photon detector. Then, by synchronizing with the trigger signal using a Si-PIN photodiode (ET-2030) on a time-to-amplitude converter (TAC; Becker and Hickl GMbH, SPC-130EMN), the time-resolved intensities were obtained, and the results were collated using PC software (Becker and Hickl GMbH, SPC-130 MN) as the lifetime of each fluorescence. Here, excitation power was set as 5.0 μW, which was enough to suppress and did not induce saturated situation of QD excitations.

Before verifying the non-linearity, the fluorescence relaxation due to multiple incident laser pulses was experimentally verified in the setup as the basic specification of our three samples: Samples A, B, and C. The left-hand side of Fig. 7 shows an example of the obtained photon-counting result from the acceptor QDs in response to double incident laser pulses, which were obtained at a certain area in Sample A. As shown, two rising phases were recognized, which were due to the first and second incident laser pulses. The time lag \(\Delta t\) between the two pulses was 7.4 ns. Under these conditions, the second pulse was irradiated before the induced optical energy was dissipated, and the corresponding photons were counted in response to the first incident pulse, which corresponded to the stagnation time of the QDR induced by the first incident pulse. Therefore, we focused on photon counting after the second incident pulse, which was extracted from the right side of Fig. 7.

Fig. 7
figure 7

Experimentally obtained photon counting result of a QDR sample under irradiation by double pulses with time difference \(\Delta t\) of 7.4 ns

To discuss the spatial variation of the QD networks in each sample, three individual areas were irradiated in the three samples, and the fluorescence photons were counted. To quantitatively compare the fluorescence relaxation of each sample, the results were fitted using an exponential equation:

$$ C\left( t \right) = A + Be^{{{{ - t} \mathord{\left/ {\vphantom {{ - t} \tau }} \right. \kern-0pt} \tau }}} , $$
(1)

where A and B are individual positive constants, and τ denotes the time constant of the fluorescence lifetime. The values of these parameters were selected to appropriately match the experimental results. As a result, \(\tau\) was calculated to be 168–315, 129–257, and 99–212 ps for Samples A, B, and C, respectively. Clear differences were observed in the results for each sample. For Sample C, which contained no donor QDs and a smaller total number of QDs, energy transfer between QDs rarely occurred, and the QDs excited by the laser pulse directly emitted photons without any energy transfer. Therefore, Sample C revealed the shortest \(\tau\). In contrast, since Sample A contained the largest number of donor QDs, frequent energy transfers were expected to occur. As a result, the excited optical energy was allowed to stagnate over a longer time in the QDR and obtained at various times owing to various energy transfers. Therefore, Sample A revealed the longest \(\tau\) among the three. The results indicated a clear relationship between the configuration of the QD network and the extent of the echo state owing to FRET between the QDs.

3.2 Qualitative Non-linearity

To quantitatively evaluate the non-linearity of each photon-counting result, we employed their correlation analysis in response to double incident pulses with a linear sum of single inputs/outputs, which are the photon counts due to a single pulse incident on each sample. For correlation analysis, Pearson correlation coefficients, R, were calculated as follows:

$$ R = \frac{{\sum\nolimits_{i = 1}^{n} {\left( {x_{i} - \overline{x}} \right)\left( {y_{i} - \overline{y}} \right)} }}{{\sqrt {\sum\nolimits_{i = 1}^{n} {\left( {x_{i} - \overline{x}} \right)^{2} } } \sqrt {\sum\nolimits_{i = 1}^{n} {\left( {y_{i} - \overline{y}} \right)^{2} } } }}, $$
(2)

where x and y represent the data obtained with the first and second incident pulses, respectively, n is the data size, xi and yi are individual data points indexed with i, and \(\overline{x }\) and \(\overline{y }\) represent the data means of x and y, respectively. While nonlinear inputs/outputs were expected to be difficult to approximate with a linear sum of the separately obtained single inputs/outputs, lower and higher correlation coefficients corresponded to the larger and smaller nonlinearities of each input/output, respectively. Furthermore, during irradiation with optical pulses, the length of the delay path in the optical setup, as shown in Fig. 6, was controlled to set the time difference \(\Delta t\) between the two pulses. Here, \(\Delta t\) was set to 0.64, 0.84, 1.6, and 7.4 ns, and the photon counts in response to the second incident pulse were determined for the three samples. Correlation coefficients were calculated from the photon counting results, which were obtained with several \(\Delta t\) values at three individual areas on the three samples.

Overall, as shown in Fig. 8, with increasing Δt, higher and more converged correlation coefficients were observed, implying a gradual dissipation of the echo states induced by the first incident optical pulse. Conversely, for \(\Delta t\) shorter than 1.0 ns, the second pulse was incident before dissipation of the echo state excited by the first incident pulse. Consequently, lower correlations and corresponding higher non-linearity were successfully revealed.

Fig. 8
figure 8

Comparison of correlation coefficients corresponding to non-linearity of fluorescence of three samples, a Sample A, b Sample B, and c Sample C

Furthermore, the three lines in the results for Sample C revealed similar curves, and the correlation coefficients were greater than 0.95, which corresponded to a smaller non-linearity. These findings were attributed to the sparse alignment of the QDs, as schematically shown in the insets of Fig. 8. Specifically, the input/output varied from area to area, and FRET between the QDs was rarely allowed. As a result, the nonlinear input/output was not sufficiently revealed in Sample C. However, in the case of Sample A, since the number of QDs was sufficient in all areas, many paths for FRET were expected. Similar input/output tendencies were obtained in each area, and a higher non-linearity than that of Sample C was revealed. In the case of Sample B, tendencies showing the most variation in each area were observed, and higher non-linearity was often revealed in some areas. As a result, we verified the echo state properties of our QDR samples and quantitatively measured hold times of less than 1.0 ns with our experimental conditions. As shown in Fig. 8, the hold time and spatial variation of the echo state properties clearly depended on the composition of the QDR sample.

4 Spatio-Temporal Photonic Processing

4.1 Basics

Based on the FRET mechanism, as described in the previous section, after the laser irradiation of the QDR, the excited energy in some QDs was probabilistically transferred to the surroundings. After multistep transferences through several FRET paths, the energy was probabilistically irradiated as the fluorescence of the QDs at various times. Moreover, the fluorescence intensity varied spatially because of the random distribution of the QDs. Consequently, the fluorescence output of the QDR can be defined as two-dimensional spatiotemporal information, which is reflected as the nonlinear input/output and short-term memory of the QDR, as conceptually shown in Fig. 9. Reservoir computing can be driven effectively using an appropriate readout of several calculation parameters for reservoir computing from fluorescence outputs.

Fig. 9
figure 9

Schematic of the spatiotemporal fluorescence output based on various FRETs in the QDR

4.2 Streak Measurement

As an experimental demonstration of QDR-based reservoir computing, we prepared QDR samples using the EPD method, which we defined as 10, 20, and 60 M samples in the previous section. The experimental setup for the fluorescence measurement is shown in Fig. 10. The fluorescence output of the QDR sample was detected using the time-correlated single-photon counting (TCSPC) method [17]. The setup and experimental conditions of the light source were the same as those for the previous setup shown in Fig. 6.

Fig. 10
figure 10

Schematic of experimental setup for streak measurement based on TCSPC method

Several parameters of the reservoir model were identified. The QDR sample was irradiated with first, second, and double pulses. As shown in Fig. 10, the delay line generated 1 ns of the lag between the first and second incidents on the QDR sample. Corresponding streak images were obtained using a streak camera (Hamamatsu C10910) upon the insertion of an appropriate bandpass filter (Edmund Optics, #65–708, transmission wavelength: 600 ± 5 nm) to extract the fluorescent output. The streak camera mainly consisted of a slit, streak tube, and image sensor. The photons of fluorescence to be measured as outputs of the QDR sample were focused onto the photocathode of the streak tube through the slit, where the photons were converted into a number of electrons proportional to the intensity of the incident light. These electrons were accelerated and conducted toward the phosphor screen, and a high-speed voltage synchronized with the incident light was applied. The electrons were swept at a high speed from top to bottom, after which they collided against the phosphor screen of the streak tube and were converted into a spatiotemporal image. Streak images of a QDR sample irradiated by the first, second, and double pulses are shown in Fig. 11a, b, c, respectively.

Fig. 11
figure 11

Examples of obtained streak images as outputs of a QDR sample with a the first pulse, b the second pulse, and c double pulses

4.3 Spatiotemporal Reservoir Model

Referring to the echo state network [1], we define the updating formula for the state vector s of a reservoir as follows:

$$ {\varvec{S}}_{t + 1} = \tanh \left( {\alpha \left( {W_{res} {\varvec{S}}_{t} + {\varvec{W}}_{in} u_{t} } \right)} \right), $$
(3)

where st is the vector of the reservoir node at t, and Wres denotes the connection weight between nodes. The non-linearity of each node is revealed as a hyperbolic tangent function. Reservoir dynamics can be described using iterative applications of the function. Win is the vector of input to the reservoir, and ut is the coefficient representing the sequential evolution of Win.

Input weight matrix. Win is defined as a linear mapping vector that projects the input vector u to the state vector space s, namely, reservoir nodes. In the QDR experiment, Win was determined by the intensity distribution of the fluorescence, which corresponds to the distribution of dispersed QDs. Therefore, in streak images, the one-dimensional fluorescence intensity of all nodes immediately after irradiation must be read to determine the Win, as shown in Fig. 12.

Fig. 12
figure 12

One-dimensional fluorescence intensity for experimental determination of Win

Leakage rate. As the fundamentals of reservoir computing, the leakage rate of the respective nodes directly controls the retention of data from previous time steps. Therefore, the reservoir can act as a short-term memory. In the experiment of QDR, relaxation time \(\tau \) of fluorescence output is directly related to the leakage rate of each node. In the streak images, we focused on the respective nodes and extracted the fluorescence relaxation, as shown in Fig. 13. The relaxations were fitted using \(N={N}_{0}exp\left(-t/\tau \right)\) for determining the relaxation time \({\tau }_{i}\) for each node.

Fig. 13
figure 13

Extraction of fluorescence relaxation for experimental determination of relaxation time \(\tau \)

Connection weight matrix. The connection weights between various nodes in the reservoir corresponded to the FRET efficiencies between the dispersed QDs in our experiment. Based on these interactions, the inner state of a reservoir can sequentially evolve to handle complex tasks. To search Wres, we assumed that multiple Wress and fluorescence relaxations of neighboring nodes were compared for each Wres as shown in Fig. 14. For quantitative comparison, the respective correlation coefficients were calculated. We then determined Wres, which revealed a similar variance \({\sigma }_{R}^{2}\) with the experimental results.

Fig. 14
figure 14

Comparison of fluorescence outputs between all nodes for determination of Wres

Activate function. The change in the inner state s due to the variation in the input vector u is defined by an active function. Here, we approximately set \(\alpha \mathrm{tanh}\) as the active function of QDR and optimized its coefficient \(\alpha \). We varied \(\alpha \) and respectively compared with experimental results based on the calculation of Pearson’s correlation coefficients. Specifically, fluorescence relaxation by the first incident pulse and the second pulse was added and compared with relaxation by double incident pulses to identify non-linearity, as shown in Fig. 15. Then, a was determined, which revealed similar non-linearity with the experimental results.

Fig. 15
figure 15

Comparison of fluorescence relaxation by the first pulse incident, the second pulse incident, and the cumulative result of the two with relaxation by double pulses incident

4.4 Experimental Demonstration

Based on the experimental conditions and obtained results, we demonstrated the sequential prediction of XOR logic based on machine learning. As XOR is one of the simplest tasks that is linearly nonseparable, it is often selected for experimentally demonstrating machine learning using an artificial neural network. In our experiment, the original streak image was arranged with 40 nodes in the spatial direction and 20 steps in the temporal direction. Each temporal step corresponded to 0.434 ns and each size of node was 1 \(\upmu \)m. Figure 16 shows some examples of data sequences predicted by our reservoir models, which were constructed by utilizing streak images of 10, 20, and 60 M samples, respectively.

Fig. 16
figure 16

Prediction results obtained by our reservoir models based on streak images of a 10 M, b 20 M, and c 60 M samples

Furthermore, we successfully predicted sequential XOR logic using 1.0% of the mean bit error rate (BER) with 100 trials, as shown in Fig. 17. The results clearly show that each model exhibits its own performance owing to the variations in the QDR, as shown in Figs. 4 and 5. However, the theoretical relationship between each performance metric and the corresponding QDR is now under discussion.

Fig. 17
figure 17

Comparison of MSEs of reservoir models using 10, 20, and 60 M samples with 100 respective trials

5 Conclusion and Future Prospect

In this section, the results of an investigation into the applicability of QDR as a physical reservoir with an optical input/output is presented. Based on the idea that QDR is expected to reveal nonlinear input/output owing to the short-time memory of optical energy in the network, photon counting of the fluorescence outputs obtained in response to sequential short-light pulses was performed for verifying the optical input/output of our original QDR. Consequently, the non-linearity of the input/output, which is a fundamental requirement for the realization of effective machine learning, was qualitatively verified. Moreover, we demonstrated that reservoir computing based on the spatiotemporal fluorescence output of QDRs can learn the XOR problem and make correct predictions with a low BER. In future studies, we will extend the applicability of our idea to execute practical tasks for a larger amount of time-series data based on the outputs of our spatiotemporal fluorescence processing. In addition, larger spatial variation is also expected to be one of the fundamental requirements in QDR for physical implementations of reservoir computing because varied nonlinear input/outputs in a single reservoir are useful for effective machine learning based on QDR, that is, nanophotonic reservoir computing. The optimization of the composition required for targeted processes in nanophotonic reservoir computing remains an open topic for further investigation.