1 Introduction

Pattern recognition through multiclass classification constitutes a very important field in recent years that has led to many technological and societal advancements thanks to its progression. Numerous technological applications have been developed focusing on human healthcare and well-being [1] to optical character detection [2], to voice recognition to even face and fingerprint recognition [3, 4]. Current state-of-the-art artificial intelligence (AI) systems such as DeepMind’s AlphaGo consume up to ~ 10 MW compared to only 20W of human brain operation [5]. Despite the incredible progression of AI over the years across various applications, it is required to adopt new strategies in designing materials, devices, architectures, and algorithms to mimic low energy consumption per operation of biological systems while reducing the manufacturing cost [6,7,8,9,10,11]. To overcome these hurdles, on the architecture level, implementing classification tasks by splitting the problem into simpler subtasks is a way to relax hardware constraints despite the less accuracy of the approach. On the computation unit level, memristive devices are an ideal platform to implement on hardware-level deep neural networks compatible with low power consumption. Neural networks have demonstrated significant potential for implementing low-energy pattern recognition in neuromorphic hardware, such as memristor crossbars. However, conventional algorithmic approaches require neural networks consisting of thousands of synapses and neurons, making them very hardware and energy intensive, prohibiting any experimental verification of the models. Recent advancements have been based on different techniques for pattern recognition applications such as implementing various algorithms for problems classification on memristors and analogue electronics that have tunable weights [12]. The challenge still lies in the demanding hardware requirements necessary to support these hardware accelerators for pattern recognition and other AI applications. Reducing the number of memristors is thus a critical need for practical applications with minimal energy consumption. Algorithmic strategies, like the one-vs-one (OvO) and one-vs-rest (OvR) binary to multiclass classification can assist in partitioning the classification into multiple smaller problems. New algorithmic approaches and hardware configurations however should be developed to address this energy deluge and complexity.

Neuromorphic computing is an ever-evolving field that aims to mimic the neural structure and function of the brain, with high energy efficiency by adopting an in-memory-processing scheme [13, 14] that could enable systems to perform local learning and decision-making paving the way for applications in Internet of Things (IoT) sensors and autonomous robots. [5, 6, 15,16,17,18] This strategy will reduce the energy consumption and the dependency on the time-consuming cloud communication. Specifically, the gradual resistance changes in memristors induced by electrical voltage pulses resemble the continuous synaptic weight changes in biological systems, hence a variety of synaptic functionalities can be emulated [19]. Memristors can be used as synapses (requiring non-volatile operation mode) or even emulating neuron activity (volatile operation mode) in any neuromorphic computing system. Notably, optoelectronic memory devices for neuromorphic computing provide concrete advantages such as low energy consumption and smaller delays, high-speed non-destructive read method not dependent on electrical wires that solves crosstalk issues [19], visual sensing, and signal processing[7, 20, 21], and logic operations[22]. Optoelectronic memristors can act as the artificial retina and combine optical sensing and high-level image processing toward artificial visual systems [23], as well as can be used to develop optical neurons, aside to synaptic function, extending further their application domains. By incorporating both light and electric modularity, optoelectronic memristors could resemble the multi-modal sensory nature of biological systems[24], as they can directly sense optical signals[25] and use them to control their resistance state [23, 26]. The light modulation of the synaptic weight of optoelectronic memristors could permit the implementation of computation tasks without requiring changing further the applied electrical bias on various memristive nodes, offering therefore the potential for lowering the number of required memristors in the chip. Various materials have been implemented for optoelectronic memristors including traditional oxides [27], 2D materials [28], organic materials [29], and perovskites [30].

Hereby, we present a novel algorithmic approach for multiclass classification tasks through splitting the problem into binary subtasks while using optoelectronics memristors as synapses. Our approach leverages the core fundamentals from the OvO and the OvR classification strategies towards developing a novel Outcome-Driven One-vs-One (ODOvO) approach. The light modulation of synaptic weights, fed in our algorithm from experimental data [13, 26, 31], is a key enabling parameter that permits classification without modifying further applied electrical biases. In general, classification (not specifically binary) implemented through light modulation of resistance states would require extra multiple iterations at different light illumination amplitudes while keeping electrical biases constant allowing from one hand reduced hardware requirements however extending the time needed per classification step. To this end, our novel ODOvO algorithm provides a solution when instead of iterating through all the pairs we search through the N−1 most likely pairs to classify the sample decreasing both the algorithmic complexity and hardware needs. It is shown that ODOvO retains similar accuracies to the conventional OvO classification while requiring even fewer iterations compared to the OvR. As an example, our MNIST dataset classification using the conventional OvO approach yielded accuracy levels of around 63%, whereas the innovative ODOvO yielded accuracy levels of 62.65% with \(\frac{N}{2}\) fewer iterations than OvO, and N-1 less iterations in comparison to the OvR that yielded accuracy of 56.15%. Overall, our approach based on light modulation of optoelectronic memristors resistance state requires at least a 10X less synapses (only 196 synapses are used) while reducing the total classification time. Consequently, our approach constitutes a feasible solution for neural networks where key priorities are the minimum energy consumption i.e., small iterations number, fast execution, and the low hardware requirements that allow experimental verification. For simple datasets like the MNIST, our binary-classifiers-based algorithms can be used in a fast and efficient way offering the potential for experimental implementation of multi-class classification without the need of hidden layers or convolutional layers.

2 Optoelectronic perovskite memristors in neuromorphic computing

The mixed ionic-electronic conductivity of perovskites renders them suitable for resistive switching memory applications [32]. Perovskite-based memristors have been demonstrated exhibiting a large ON/OFF ratio [33, 34], fast switching speed [35], and good retention [36]. Perovskite memristors have been successfully used for developing crossbar arrays [37], performing logic operations [38], artificial synapses [8], and neural networks [39]. Furthermore, their light-tunable resistance enables the design of photonic perovskite devices that perform logic operations [40] and are used as artificial optoelectronic synapses [41]. The solution-based processing of perovskites enables the use of printing techniques for device fabrication on flexible substrates [42], reducing the complexity and manufacturing cost compared to other technologies such as oxide-based memristors, thus opening the path for flexible processing engines to be integrated in mobile accelerators for wearables IoT systems.

Perovskite memristors operate by either forming conductive filaments between the top and bottom electrodes, or by charge-trapping processes at the interfaces that induce corresponding Schottky barriers [43, 44] allowing the system to transit from a low resistance state (LRS) to a high resistance state (HRS). In the filament-based operation mode, the conductive filament is formed either by active metal cations or by ion migration of mobile species (mainly iodine vacancies (VI), since they possess the lowest activation energy in the system [45]). Light illumination in perovskite memory devices has reported to cause a Schottky-barrier modification [46] or a filament annihilation [47], leading thus to resistance modulation. Light illumination can alter the ON/OFF ratio of the device during the endurance test, as light causes a controllable modulation of either LRS or HRS, or both states, resulting in a slight reduction of the ON/OFF ratio in many studies [46]. Additionally, the state retention time has been reported not to be affected by light. It is possible that under illumination, some modulation might occur at the LRS and HRS during the first seconds of the measurement, which this process does not affect the retention time of the device compared to measurements taken under dark [48]. The intrinsic conduction mechanism being either filamentary or interface is also affecting how resistance is modulated under light illumination. In devices operating with filaments, light-assisted recombination of ions and vacancies occurs, which causes resistance enhancement under illumination [48]. Overall, the resistance modulation by light illumination in optoelectronic memristors is not expected to cause further variability issues, than these reported in conventional memristors. Ye et al. introduced a lead-free Cs2AgBiBr6 perovskite memory in an ITO/Cs2AgBiBr6/PMMA/Ag structure. The device showed a resistance increase under UV light illumination, while being in the LRS, which is attributed to the annihilation of Br vacancies. An optically controlled logic gate and artificial neuromorphic visual system was demonstrated [49].

The structure of the perovskite memristive device used for the experimental data of the current study is depicted in Fig. 1a. It consists of the material stack ITO/PTAA/Perovskite/PCBM/BCP/Ag. All layers are solution-processed and deposited by spin coating except the top electrode, Ag (100 nm), which is thermally evaporated. The perovskite layer is based on a four-cation mixture of Rb, Cs, FA, and MA, abbreviated as RbCsFAMA. More details about the experimental procedures can be found in previous works of the authors [26, 31, 50,51,52]. The commercially available platform ARKEO (Cicci Research s.r.l.) is used for current–voltage (I–V) measurements. The voltage is swept from + 1 to − 1.2 V, and a compliance current of 10 mA is used to protect the device. All measurements are performed in ambient air under 25 °C. Figure 1b shows 20 representative I-V curves of a typical memristive device. As it is evident, the device shows stable bipolar switching behaviour and operates at low voltage bias (< 1 V). The average SET voltage is 0.13 ± 0.04 V, and the average RESET voltage is − 0.75 ± 0.04 V. The ON/OFF ratio is ∼ 103. The device operates based on the formation and rupture of a conductive filament between the top and the bottom electrode. According to our previous work, the device operates by the formation and rupture of a conductive filament based on metallic electrode (Ag) and by halide vacancies (I and Br) [36]. The device exhibits an inverse photoconductivity behaviour, thus the resistance increases upon illumination because of a light-enhanced halides recombination process [45], which ruptures the conductive filament [53]. The same effect has also been confirmed in other halide perovskite memory devices [54, 55]. Hence, the device gradually transits to HRS, and the transition can be tuned by controlling the illumination intensity. Figure 1d depicts an almost linear correlation of light intensity with memristors resistance, for specific illumination intensity range (~ 40 mW/cm2–80 mW/cm2) as extracted by experimental perovskite optoelectronic memristors. The formation and rupture of the conductive filament during SET and RESET, as well as the filament rupture under illumination, are illustrated in Fig. 1c. Table 1 shows a comparison between conventional oxide-based and perovskite memristors. It is noted that perovskite devices with high ON/OFF ratios operating at low electric fields can be realised, with good retention properties approaching the performance of oxide-based devices. However, the cycling endurance of these devices is inferior compared to regular oxide systems. Variability is an issue that hinders the performance of perovskite memristive devices as well.

Fig. 1
figure 1

a Cross-sectional view of the halide perovskite memory device, showing layers from ITO to Ag electrode. b 20 I–V curves illustrating the low-power, stable bipolar switching behaviour of the device. c Schematic diagram showing the formation and rupture of the conductive filament in the device during operation and under illumination. d Almost Linear Correlation of light intensity vs Resistance in perovskite based optoelectronic memristors as extracted by experiment [13, 26],

Full size image
Table 1 Comparative key performance indicators of Perovskite-Based and Regular Oxide-Based Memristors
Full size table

Herein, we used the experimental dependence of memristors resistance on light illumination reported in Fig. 1d [13, 26, 31]. Owing to this light-modulation feature, we can apply various binary classification algorithms such as OvO, OvR and our proposed ODOvO and proceed to classification by tuning purely the memristive state by light intensity. This extra degree of freedom allows one single crossbar array to classify the problem using the electrical bias as inputs, while being able to be adjusted further to other synaptic weights by using variable light intensities.

3 Background and methodology for OvO, OvR, and the novel Output-Driven OvO

This concept of binary classifiers has been already addressed in the field of pattern recognition and specifically in neural networks using the OvO [65] and OvR [66] classification strategies, which split the problems into binary classifications that combined assist in finding the solution of the problem. These approaches can be applied effectively in the classifications of multiclass complex problems where the easiness of the approach and hardware simplicity are priorities despite their lower accuracy compared to complex classification algorithms. We take advantage of the light modulation in perovskite optoelectronic memristors and build corresponding neural networks implementing various algorithmic approaches for simplified multiclass classifications based on binary classifiers. The three binary classification algorithms (OvO, OvR, ODOvO) when applied to conventional memristors that do not allow optoelectronic control will require higher number of synapses by a factor of 10X to achieve the same accuracy with the same number of iterations. Optoelectronic memristors light tunability is therefore the key enabling parameter to implement all binary classifiers (OvO, OvR, ODOvO) without applying new electrical biases after each iteration, a process that consumes power and also adds time delays [67]. Consequently, using optoelectronic memristors gives the advantage of splitting up big computation tasks or big classifications for pattern recognition into many simpler binary classifications which wouldn't make sense for traditional memristors. Optoelectronic memristors therefore can be used to reduce the hardware complexity required for neural network applications [68, 69].

It is noted that a specific local photo-addressing hardware is required for the experimental realisation of our concept. There are available techniques to apply locally varying light intensity so be able to set training weights through light modulation. One option is to use laser pulses to scan across the various memristive nodes and control light intensity locally [70,71,72]. Alternatively, an array of LEDs can be developed on top of the memristors array controlling the light intensity on each memristor. A third option could be the integration of electrochromic pixels on top of each memristive node [73],where in this case one constant light source is needed and light modulation could be achieved locally by tuning the light transparency of the electrochromic pixels through electrical pulses application. The experimental realisation of the local light control could add some extra hardware complexity, however, resolving this extrinsic to the memristive nodes hardware complexity is out of the scope of the current study.

Both OvO and OvR could have a key advantage when implemented using optoelectronic memristors in real-life problems. An example of OvO and OvR in action for classifying a 10 multiclass problem is depicted in Fig. 2. In the OvO, each complex multiclass problem is split into all the \(\frac{[N\left(N - 1\right)]}{2}\) [65] pairs of classifiers that are available for this classification. This allows the classifiers to scan across all the available pairs and distinguish the complex relationships between the classes, thus allowing a more accurate inference for the classification. This process is however computationally expensive requiring \(\frac{[N\left(N - 1\right)]}{2}\) iterations of all the trained weights to classify each sample of the problem. For multiclass problems using OvO, addressing the light modulations for all the trained classifiers would require significant time. Since a single hardware architecture should handle the classification problem, more effective ways towards reducing the time of execution per each sample should be identified. To this end, a more effective approach to classifying the problem in binary subtasks is the OvR. This approach has the benefit of requiring less training steps since only \(N\) classifiers should be trained for all the cases mentioned, while a larger range of training per each classifier can be used, \(whereas N\) is the number of classes in the dataset. In this approach, each class is trained against all the other classes; therefore, each classifier can distinguish the class we train it on and keep all the rest classes as one class. Following this method, \(N\) iterations should be applied after the training phase based on all the available classes, and then, the class that resembles the highest probability is the one selected to classify the problem. This process however has issues in cases of having similar classes because this approach cannot capture in detail the relationships and differences of the classes. This is especially the case when the data are resized to be utilised in smaller harwares, where some classes might be overlapping. Overall, this approach can capture enough details for most applications when implemented with optoelectronic memristors.

Fig. 2
figure 2

Example of One vs One and One Vs All in action for classifying in a 10 multiclass problem. We can see clearly that the One vs All classifier trains each class independently against all the rest available as one ending with ten classifiers for the MNIST dataset. On the right, we can see the classifier is creating individual pairs among all the possible combinations of numbers available and taking statistics on how many times each individual number has been classified

Full size image

To overcome the above mentioned issues, we hereby propose another classification algorithm based on a modified version of the OvO approach, while being able to achieve classification at sufficient accuracy with as low as possible number of iterations per classification. The proposed solution, termed as Outcome Driven One vs One (ODOvO), utilises the core basics of OvO in which each pair of classes is separately split leading to \(\frac{[N\left(N - 1\right)]}{2}\) trained classifiers. An example of the ODOvO in action for classifying the fifth class in a 10 multiclass problem is depicted in Fig. 3. Following ODOvO on the other hand, a dynamic approach is implemented to classify the sample thus \(N\)-1 iterations are needed, instead of the \(\frac{[N\left(N - 1\right)]}{2}\) iterations needed for standard OvO classifications. For architectures based on optoelectronic memristor this novel approach can have a huge advantage since it allows to reduce the required iterations of light modulations that can have a significant impact on the total time needed to classify the problem. To achieve an ODOvO classification, we initially train the classifiers similarly to the OvO approach, but adding to this, the outcome of the first pair of classes classification is used as input for the next classifier. Specifically, based on the outcome of the first class that is either 0 or 1, we accordingly continue with the next trained weights of the classifiers. This means that if the result of the first classifier remains “zero,” meaning the first class of the problem seems to be the most likely result, we then move to the next trained classifier of the first class (move vertically from top to bottom in Fig. 3). If the outcome is “zero,” meaning the second class is more likely to resemble the sample, we move on the trained weights of the classifier that most closely resembles the class from the previous iteration (move horizontally from left to right in Fig. 3). This class will be compared with the next available class. Therefore, based on the outcome of each pair classification, we move on to the next pair that would be the most likely for our classification problem.

Fig. 3
figure 3

Example of the Outcome-Driven One vs One (ODOvO) in action for classifying the fifth class in a 10 multiclass problem. We note that the pairs of Binary classifiers of the sample are being driven by the result of each iteration to achieve the final classification

Full size image

Overall, this approach makes iterations across all the classes but in the form of the most likely pairs achieving accuracies close to OvR, while reducing the number of iterations needed for the classification from \(\frac{[N\left(N - 1\right)]}{2}\) of the OvO down to \(N-1\) That is even less than the OvR approach.

The pseudocode for this classification’s algorithm is listed below.

Algorithm 1
figure a

Outcome Driven One vs One (ODOvO)

Full size image
Algorithm 2
figure b

Final Classification

Full size image

Now let's try an example using the above pseudocode. Let's assume that the sample we are trying to classify is the number “5” class (see Fig. 3). The first classifier for this will be the pair of “0” and “1” classes, where this comparison will result in zero that means it has closer relation to the class “1” (iteration 1 in Fig. 3). Now it will adjust the weights to classify the class “1” and “2” resulting in the outcome of one, which means this sample has closer relation to the class “1” (iteration 2). It now adjusts the weights for the class “1” and “3” which results in the outcome of zero (iteration 3) leading to swapping the second class to become the first and the new second class to be the next digit of the new first class, therefore, classifying for “3” and “4” (move vertically from iteration 3 to 4 in Fig. 3). This process will continue based on the outcome that is still zero (iteration 4), adjusting the weights for classification to identify whether the class is “4” or “5”, which still results in zero (iteration 5), which will result, in changing to the classes pair of “5” and “6”. After that, this classifier results into one (iteration 6) changing the weights to classify the pair of “5” and “7”, which also results into one (iteration 7) therefore also changing the weights to “5” and “8”, which then also results to one (iteration 8) leading the final weights loaded to the classifier being the classes of “5” and “9” which will classify the final result of the sample based on the result of those two classes. This outputs also “1” (iteration 9) which will categorise this sample to the class number “5” correctly (Result). Above we can see the visual interpretation of the abovementioned example for classification.

This approach, similarly, to the previous ones, possesses big advantages for the implementation of neural networks using optoelectronic memristors since they can harness the light modulation of memristors conductance to adjust the weights and classify multiclass complex problems with simple binary classification architectures. It’s important to note that this algorithm might not perform equally well in cases where the first classes in the iteration are not very separable, but for examples where the time of execution is the key parameter would be an ideal solution. In applications where optoelectronic memristors are integrated, the iterations are handled as hardware changes in response to light intensity thus offering the ability to raise the accuracy while requiring fewer iterations. For some specific applications or datasets, the use of the traditional OvO or OvR might perform better than ODOvO thus it is important to test all the algorithms for various use-case scenarios. The ODOvO approach is an intermediate step between the two already established algorithms for binary classifications with the goal of reducing the iterations needed to classify the problem. Its application would result to fewer light modulations needed on optoelectronic memristors while retaining accuracy levels comparable to the OvO approach and reducing the iterations needed for classification by a factor of \(\frac{N}{2}\).

4 Proposed architectures for performance evaluation and benchmarking

The proposed algorithmic architectures employ a cascading series of binary classifiers based on Feedforward Neural Networks (FNNs). Each of the classifiers is a single-layer NN with sigmoid activation function [74] to ensure that the synaptic weights are positive in agreement with the experiment where the imported conductance values of optoelectronic memristors are always positive. The weights and biases of the FNNs are adjusted and learned through a basic gradient descent algorithm based on the error of the network output and the actual labels. Binary classifiers are used to split those multiclass problems by implementing the OvO, the OvR, and the ODOvO approaches. This approach can be implemented with optoelectronic memristors as cascading classifiers by directly modulating the light intensity of the interconnected memristors, which alters the conductivity of each memristor almost instantaneously. In this work, we utilised the MNIST Dataset (Modified National Institute of Standards and Technology database) [75] due to its inherent nature of classifying numbers as well as its popularity and wide use for benchmarking. We trained the dataset through individual pairs training focusing on only two specific numbers using a rather simple architecture for the FNN resulting in 45 pairs. This results in 45 different classifiers for the case of OvO and the ODOvO, and 10 classifiers for the OvR approach.

Aiming at developing classification architectures that can be realised experimentally we tried to reduce the hardware requirements as much as possible. To this end, we pre-processed the MNIST [75] dataset from the original 28 × 28 size to 14 × 14 size. This scaling down to only 196 synapses (thus 196 memristors), although could limit the accuracy of the classification algorithm, can be experimentally realised and allow the hardware benchmarking of the proposed algorithms. The images of Fig. 4 are encoded applying a threshold to the pixel intensity that serves as input for the NN. We can see in Fig. 5 the process it takes on the crossbar for the actual classification process. The electrical inputs are the inputs of the classifying sample and with the iterations of the different trained light intensities we can come to the final solution. It's crucial to mention that the same hardware synapses are used for all the 9, 10 or 45 iterations, depending on the classification approach, in order to achieve the final classification of the sample. In all cases, the only adjustments in synaptic weights of the FNN are implemented through modulation of light intensities.

Fig. 4
figure 4

Structure of the Neural Network Architecture. The sample is received as an input, then resized to a smaller dimension to be encoded and classified on the Binary classifiers of the OvO, OvR and ODOvO

Full size image
Fig. 5
figure 5

Proposed architecture with the different light intensities applied in the devices until the final classification

Full size image

All three approaches showed promising results when used to classify the same 2000 images using the same architectures and epochs for the training of the MNIST Dataset (Table 2). The OvO required 45 iterations of weights per classification and was the most accurate with accuracy levels of 63.2%. The novel approach of ODOvO required only 9 iterations of weights per classification and had very similar accuracy levels results with the OvO (62.65%) while requiring 5 times fewer iterations of weights and therefore 5 times fewer light modulations implemented on the memristors. Finally, the OvR required 10 iterations of weights per classification (like the ODOvO), while only 9 iterations were needed achieving accuracy levels of 56.15%. More details and extended results can be found in the supplemental information. The use of all these approaches offers major benefits as we can classify a 10-class classification problem that would normally require more complex architectures by harnessing the properties of the light-triggered memristors. Additionally, the proposed ODOvO algorithm reduces the number of iterations needed by a factor of \(\frac{N}{2}\), and thus the number of light modulations needed per successful iterations, without losing on accuracy in comparison to standard algorithms such as the OvO.

Table 2 Different algorithms utilised for binary classifications of the Optoelectronic Memristor Neural Networks
Full size table

The innovative ODOvO algorithm and the use of binary classifications in general, such as OvO and OvR for multiclass classifications, combined with the use of optoelectronic memristors constitute an advancement compared to conventional computation architectures. These approaches could also be achieved in Spiking Neural Networks (SNN), although this implementation raises the complexity [76]. We additionally implemented the OvO approach using Snntorch [77] with Leaky-Integrate and fire [78] as a proof of concept in a similar architecture to the originally proposed architecture (see supplementary information for further information).

5 Conclusion

We explored the theoretical usage of binary classification algorithms such as OvO and OvR to address the hardware limitations of experimental neuromorphic crossbars targeting models’ verifications in real-life scenarios. Furthermore, we proposed an innovative variation of OvO, termed as ODOvO that retains satisfactory accuracy levels while reducing the required iterations by a factor \(\frac{N}{2}\). These binary classification architectures take advantage of the light modulation of the resistance states achieved in optoelectronic memristors to perform multiple binary classifications within the same crossbar device, simplifying the problem of multiclass classification and making it a viable solution towards less hardware-intensive configurations requiring at least a 10X less synapses compared to conventional memristors. We used the MNIST [75] dataset as a testbed to validate this approach, demonstrating that the use of optoelectronic memristors in NN applications could reduce energy consumption and hardware complexity. This approach provides a foundation for further research for tailored algorithms targeting neuromorphic computing hardware, showing potential for efficiently addressing complex computational tasks such as multiclass classification.