1 Introduction

In the last few decades, the advancement in quantum information processing has significantly impacted the fields of computation and communication technologies. The superiority of a quantum protocol lies in the fact that it can accomplish tasks either impossible by a classical protocol (Bennett et al. 1993; Bennett and Wiesner 1992), or in some cases, it performs exponentially or polynomially faster compared to its classical analogue, the most prominent examples being Shor’s factorization algorithm (Shor 1997) and Grover’s search algorithm (Grover 1997). These traits of a quantum system, particularly the computational speed-up, has been crucial in developing powerful algorithms for quantum simulation (Lloyd 1996; Peng et al. 2009; Peng and Suter 2010; Álvarez and Suter 2010; Peng et al. 2014; Chen et al. 2016; Li et al. 2017), Boson sampling (Aaronson and Arkhipov 2011; Broome et al. 2013; Spring et al. 2013; Tillmann et al. 2013; Crespi et al. 2013; Wang et al. 2017), solving systems of linear equations (Harrow et al. 2009; Cai et al. 2013; Pan et al. 2014; Barz et al. 2017; Zheng et al. 2017) and many other computational problems. Recently, the possibility to use quantum properties to facilitate artificial intelligence and machine learning has been extensively investigated (Manzano et al. 2009; Lloyd et al. 2013; Rebentrost et al. 2014; Cai et al. 2015; Li et al. 2015; Ristè et al. 2017; Dunjko et al. 2016; Buffoni and Caruso 2020). We are living in the era of quantum supremacy (Arute et al. 2019; Zhong et al 2020; 2021; Wu et al 2021). It has been possible to build quantum processors (processing units) with several tens of qubits, some of them being accessible to users from anywhere in the world through cloud-based sharing (https://www.ibm.com/quantum-computing/). However, these machines are still extremely noisy and, for this reason, they are called Noisy Intermediate-Scale Quantum (NISQ) devices (Preskill 2018).

This progress is starting to affect the field of image processing. Vision is the most fundamental mechanism of obtaining information about a system. Thus, image processing is a necessary task in a broad range of directions like medical science, space science and automobile technologies. With the advancement in quantum technologies, a growing interest has been directed towards implementing quantum systems for improved image processing. Some quantum image processing protocols have been proposed and tested, which show polynomial and exponential speed up; an example is quantum edge detection (Zhang et al. 2015; Yao et al. 2017; Cavalieri and Maio 2020; Xu et al. 2020). Another widely used image processing task is pattern recognition. The application of the latter ranges from identifying a string of integers or texts present in an 1D array, to recognizing 2D patterns such as faces of people, handwritten digits, geometric shapes and other tasks. In 2002, Trugenberger introduced the idea of quantum associative memory, and put forward a quantum pattern recognition protocol based on that (Trugenberger 2002). Using quantum Fourier transform, Schutzhold in 2003 devised another protocol for pattern recognition which demonstrated exponential speed-up over its classical analogue (Schützhold 2003). A number of follow-up works attempted to improve the above protocols or presented similar algorithms inspired by them (Pham and Park 2014; Prousalis and Konofaos 2019; Banchi et al. 2020). Except these, quantum pattern recognition protocols based on the framework of classical Hopfield neural network (Neigovzen et al. 2009), the hidden shift problem (Montanaro 2015), pixel gradient calculation (Zhang et al. 2015) and Grover’s algorithm (Jiang et al. 2016; Soni and Rasool 2020; Tezuka et al. 2022) have been proposed. Some of them adopted a quantum machine learning approach (Lloyd et al. 2013; Kapoor et al. 2016; Benedetti et al. 2017; Denchev et al. 2012; Schuld et al. 2016; 2015). However, when tested on real quantum systems, the protocols are inevitably subject to noise, which degrades their efficiency. Thus, one must learn how to control noise and counteract its effects.

In this work, we consider a quantum pattern recognition algorithm based on swap test. The swap test is used to calculate the closeness between two quantum states. In quantum image processing, the classical images are encoded in quantum states; hence, the swap test emerges as a very plausible approach to find similarity between two quantum images. This has been previously mentioned and briefly discussed in Yao et al. (2017) and Cavalieri and Maio (2020). The question remains as what will be the efficiency of this protocol in real quantum systems. This is motivation of our work, where the real systems are the IBMQ devices (https://www.ibm.com/quantum-computing/) and NV centers in our experimental setting. We aim to identify a pattern in the target image by comparing it with the patterns present in a large set of reference images, and finding an exact match. We find that, for systems as small as three qubits, and corresponding states which can be prepared using a few Hadamard and controlled NOT gates, the performance of swap test is low due to the strong presence of noise. To curb the effect of noise, we try an alternate but equivalent circuit, often called as ‘destructive swap test’ in the past literature (Garcia-Escartin and Chamorro-Posada 2013), in which the number of necessary gates is reduced. By using this novel approach, we observe an improved performance of the pattern recognition protocol for three-qubit states in real IBMQ devices (see Section 4 for more details). We then go on to try the destructive swap test in these systems for higher dimensional binary and grayscale images, including small-dimensional biomedical images obtained in the lab with MRI. We also present an experimental setup and the resulting data where destructive swap test is performed using diamond nitrogen vacancy (NVs) centers. Our results suggest that destructive swap test can be a potential alternative to achieve the goal of swap test, at a reduced level of noise. Lastly, we propose a quantum protocol inspired from the quantum associative memory, by which one can recognize a pattern from a large set of reference patterns.

The paper is organized as follows. In Section 2, we describe the encoding of a classical image in a quantum state. In Section 3, we discuss swap test and its performance in real quantum computers. This leads the way to destructive swap test and its improvements over swap test. In Section 4, we demonstrate the performance of destructive swap test in real quantum systems, when identifying similar geometric patterns encoded in binary images, as well as binary images obtained from MRI of human blood vessel. We also present a short analysis of how the success probability of the protocol changes with varying noise in the real quantum processors. In Section 5, we present similar results for grayscale images. In Section 6, we show the experimental result of destructive swap test in diamond NVs. In Section 7, we propose a quantum algorithm analogous to supervised learning, for using destructive swap test in pattern recognition. Lastly, Section 8 contains the concluding remarks.

2 Encoding an image in a quantum state

The first task in quantum image processing is to encode the pixel positions and corresponding pixel values of a classical image in a quantum state. A number of encoding processes have been proposed to date, e.g. Flexible Representation of Quantum Images (FRQI) (Le et al. 2011) and Novel Enhanced Quantum Representation (NEQR) (Zhang et al. 2013). In this work, we will use a different encoding method first used by the authors in Yao et al. (2017), which is often referred to as Quantum Probability Image Encoding (QPIE).

We work with a 2D grayscale image of N = P × Q pixels, which is classically encoded in a P × Q dimensional matrix M, the matrix element M(i,j) denoting the pixel value at position (i,j). To encode this image in a quantum state, we can use any m-qubit system, such that 2m ≥ N. The Hilbert space \({\mathscr{H}}_{2}\) of each qubit is spanned by the computational basis {|0〉,|1〉}. From the m-qubit register, we choose n qubits such that 2n = N. We initialize them in the state \(\vert f\rangle ={\sum }_{i}c_{i}\vert i\rangle \) where the basis vectors |i〉 span the 2n dimensional Hilbert space \({\mathscr{H}}_{2}^{\otimes n}\), encoding the positions of the pixels. The coefficients ci encode the corresponding pixel values. The first P elements of |f〉 encode the first column of M, the next P elements encode the second column and so on. Thus, each pixel position corresponds to a particular basis state |i〉 of \({\mathscr{H}}_{2}^{\otimes n}\), and its pixel value corresponds to the coefficient ci in |f〉. Lastly, the state |f〉 must be normalized.

Most generally, the images can have a color format, e.g. RGB or sRGB, where the pixel values can vary over a range. The images can also be grayscale, where the varying pixel values denote different shades of gray. However, depending on which type of information we want to acquire from the images, converting them to binary images can be operationally advantageous for further image processing tasks, e.g. edge detection. In a binary image, all the pixel values are 0 or 1. The conversion from a grayscale to a binary image can be achieved by choosing a suitable threshold pixel value p in between the minimum and maximum pixel values of the greyscale image. The pixel values equal or higher than the threshold are converted to 1, those below the threshold are converted to 0. Depending on the choice of p, the resulting images differ.

Now that we are able to encode the images, we proceed to apply a quantum pattern recognition protocol in the next section. In classical image processing, a brute-force method is to measure the pixel values of all the N pixels. However, once encoded as an n-qubit quantum state, the number of measurements necessary to obtain certain information about an image is drastically reduced compared to the classical case. To define our pattern recognition protocol, we assume that we have a target image with an unknown pattern to be detected, and a large set of reference images with different patterns which are known to us. The target pattern can be detected if it is compared with each reference image and an exact match is found. In case an exact match does not exist in the reference set, one can still look for the closest match. Translated into the language of quantum physics, one needs to calculate the closeness or distance between the quantum states representing the images. A schematic diagram of the protocol is presented in Fig. 1. The distance is determined by swap test, as we discuss in the next section.

Fig. 1
figure 1

A schematic diagram of our pattern recognition protocol. In the target image, one aims to detect the handwritten number ‘9’. There is a set of reference images with handwritten digits from 0 to 9. The target image and an arbitrary chosen reference image, in this case ‘2’ for demonstration purpose, is encoded as quantum states, and passed as the inputs of a quantum protocol to calculate the distance between them. The measurement result undergoes a classical post-processing. The protocol is repeated for all the reference images. At the end, the reference image with minimum distance to the target image is assigned as the pattern to be detected

Full size image

3 Swap test

Swap test is a widely used protocol for calculating the overlap between two quantum states (Buhrman et al. 2001; Gottesman and Chuang 2001; Kang et al. 2019). Specifically, if |ψ〉 and |ϕ〉 are two pure states, swap test can be used to calculate |〈ψ|ϕ〉|2, which is also a valid distance measure between the two states, known as quantum Fidelity (Jozsa 1994; Schumacher 1995). The circuit of swap test for single qubit states is shown in Fig. 2a. It can be generalized to n-qubit states by repeating the controlled-swap gate for all qubit pairs, as shown in Fig. 2b. Each of the states |ψ〉 and |ϕ〉 is encoded using three qubit quantum registers Q1 and Q2 respectively. There is an auxiliary qubit Qa which is initialized in the computational basis state |0〉. After initializing all the qubits, a Hadamard gate H is applied on Qa, which transforms |0〉 to \(\vert +\rangle =\frac {1}{\sqrt {2}}(\vert 0\rangle + \vert 1\rangle )\). This qubit is then used as the control qubit to apply the controlled swap operation between all consecutive qubit pairs of Q1 and Q2. In the controlled swap operation, if the control qubit is in state |1〉, the states of the two target qubits are interchanged. If the control qubit is in state |0〉, nothing changes. This is followed by another Hadamard operation on Qa. Lastly, there is a classical register ‘cr’ which registers the measurement outcome on the auxiliary qubit. The outcome can be ‘0’ or ‘1’, associated with the measurement basis states |0〉 and |1〉 respectively. The measurement is taken on a large number of identically executed circuits. If |ψ〉 = |ϕ〉, the measurements will return ‘0’ with probability P(0) = 1. If they are orthogonal, \(P(0)=P(1)=\frac {1}{2}\). For all other cases, P(0) varies between \(\frac {1}{2}\) and 1. The fidelity between two states is then

$$ \mathcal{F}=\vert\langle \phi\vert\psi\rangle\vert^{2}=2P(0)-1, $$
(1)
Fig. 2
figure 2

The circuit diagram of swap test for (a) single qubit states and (b) three-qubit states. (c) The swap test circuit after transpilation in ibmq_manila, for two single qubit states, both being |0〉

Full size image

and the overlap \(\mathcal {I}\) is

$$ \mathcal{I}=\vert\sqrt{\mathcal{F}}\vert. $$
(2)

A higher value of \(\mathcal {I}\) implies higher degree of similarity between two states. Both \(\mathcal {F}\) and \(\mathcal {I}\) can be equivalently used to measure the degree of similarity between two quantum states. In this work, for most parts (except the diamond NVs), we stick to \(\mathcal {I}\) since it is more sensitive to small changes in P(0).

We assume that for our pattern recognition protocol, both the target and reference images have same dimension. The target image is encoded in a quantum state, and passed as one of the inputs in the swap test circuit. On the other hand, each of the reference images is encoded in a quantum state, and then passed as the other input. If two patterns match exactly, one gets ‘0’ with probability 1 in the measurement of the auxiliary qubit. Otherwise, one can detect the closest pattern of the target image as the one with which it has highest overlap.

In real quantum processors, the performance is unavoidably affected by noise. The latter can appear due to decoherence, interaction among qubits, gate fidelity, state preparation and measurements. The adverse effect of noise on any protocol increases with increasing number of qubits and gates. The IBMQ real quantum processors provide a number of virtual qubits to the user for constructing the circuit, whereas the real quantum processor realizes this circuit using same number of physical qubits which follow a certain connectivity or coupling map between them. Each processor also has a set of ‘basis gates’ consisting of some particular single qubit and two-qubit gates. When a circuit is run in these processors, it typically needs to be ‘transpiled’. Transpilation automatically analyses the gates used in the circuit, and according to the required connectivity between virtual qubits, maps them to a set of physical qubits. Next, the action of all the complex gates is obtained by using the basis gates. The depth of the resulting transpiled circuit can be different for devices with different coupling maps. Both of the above steps are executed in a way to minimize the noise in the output, and the efficiency depends on the optimization level chosen by the user while executing the run on real devices. Thus, in the actual circuit realized using physical qubits, the number of gates is higher compared to what we add to the virtual circuit. This has been shown in Fig. 2c for single qubit states. For higher dimensional states, performance of swap test invariably degrades. We demonstrate this in Fig. 3 where we perform swap test for two states encoded using one qubit, two qubits and three qubits

Fig. 3
figure 3

The mean overlap \(\mathcal {I}_{\text {mean}}\) between two identical states encoded using n qubits and the corresponding standard deviation σ for swap test. The data are obtained by performing 100 runs of the circuit in the real IBMQ processors ibm_lagos

Full size image

respectively. For all three cases, we take the two states to be same and calculate the overlap between them. The states corresponding to these three cases are \(\vert \psi _{1}\rangle =\frac {1}{\sqrt {2}}\big (\vert 0\rangle + \vert 1\rangle \big )\), \(\vert \psi _{2}\rangle =\frac {1}{\sqrt {2}}\big (\vert 00\rangle + \vert 11\rangle \big )\) and \(\vert \psi _{3}\rangle =\frac {1}{\sqrt {2}}\big (\vert 000\rangle + \vert 111\rangle \big )\). These states are typical exemplary states, and we prepare them by directly adding Hadamard and controlled NOT (CNOT) gates to the circuit. The CNOT gate has a control qubit and a target qubit, the state of the latter undergoes a bit-flip operation (gate corresponding to pauli-x matrix σx) if the control qubit is in state |1〉, otherwise nothing changes. The overlap is calculated for 100 realizations of the identical circuit, and then the mean overlap \(\mathcal {I}_{\text {mean}}\) and the standard deviation σ is calculated. For this, we have used the 7-qubit real quantum processor ibm_lagos, which is compatible for encoding two 3-qubit states, and had minimum gate noise and readout error compared to all the 7-qubit real devices at the time of generating the data. To create each one of the subsequent plots, we used the real quantum device which has the lowest noise indicator at the time of execution. From Fig. 3, we see that for the single qubit case, in spite of the presence of noise, the performance of swap test is quite good. For two qubits, the noise becomes significant, and for the three-qubit case, the value of the overlap worsens. Due to the cloud inaccessibility to higher dimensional systems, we could not check the swap test for four qubit states. However, the results are sufficient to suggest that the swap test quickly loses its relevance for states having dimension higher than 3, and this poses a primary challenge to utilize it for pattern recognition in even fairly small images. Even in the NV setup, the experimental results deviate because of imperfect realization of the pulse sequences, as will be discussed in Section 6. However, the quantum simulators are ideal and do not suffer this problem. One can check that for two same images, the mean inner product and standard deviation will always return values 1 and 0 respectively. This is because the IBMQ simulators are classical computers that simulates the results of an ideal quantum evolution. Hence, they can still be used to check the performance of swap test irrespective of the dimension, while being within their size limit, which is 32 qubits for the qasm_simulator.

3.1 Destructive swap test

In Garcia-Escartin and Chamorro-Posada (2013), the authors showed that an equivalent circuit for swap test can be built without using the auxiliary qubit, at the cost of measuring all the qubits used to encode the states. The controlled swap gate used in the previous section is a three-qubit gate which can be realized by two CNOTs and one CCNOT (controlled controlled NOT) gate. In the new circuit, all the controlled swap gates are replaced by same number of two-qubit CNOT gates. Also, only one Hadamard gate is required. Thus, compared to the original swap test, in this case, the number of gates in the transpiled circuit decreases significantly, hence reducing the noise in the output. This protocol is referred to as ‘destructive swap test’, because any superposition in the output is destroyed on measuring all the qubits. Later in Cincio et al. (2018), the authors used machine learning approach to find the optimized circuit for calculating the overlap between two states, and they rediscovered the destructive swap test as the optimum algorithm. Their results, obtained using 5-qubit IBMQ and 19 qubit Rigetti’s quantum computer, show a huge improvement of destructive swap test over the auxiliary-qubit-assisted swap test in terms of reducing the noise in the output. The circuit for destructive swap test for single qubit states is shown in Fig. 4a. For n-qubit states, the gate sequence of (CNOT+Hadamard) is applied to the n pairs of qubits belonging to the two states, as shown in Fig. 4b. The circuit of Fig. 4a after transpilation is shown in Fig. 4c, which clearly uses a lot less number of gates compared to Fig. 2c. For each shot of the circuit run, the classical bit string corresponding to the measurement outcomes from the qubits of |ψ〉 is denoted by Oψ, and that for |ϕ〉 is denoted by Oϕ. Each of Oψ and Oϕ can have 2n different possibilities. The joint string OψOϕ constitutes the measurement outcomes of the total system, which has 22n possibilities. Each element \(O^{\psi /\phi }_{i}\) (i = 1,2,...,n) is a bit, taking values either ‘0’ or ‘1’. Suppose \(O^{\psi }_{i}\) and \(O^{\phi }_{i}\) are the i th bits of the two strings respectively. As shown in Garcia-Escartin and Chamorro-Posada (2013), an equivalent situation of obtaining ‘1’ in the auxiliary qubit measurement in the standard swap test occurs when Oψ and Oϕ are such that \({\sum }_{i}\text {AND}(O^{\psi }_{i},O^{\phi }_{i})\) has odd parity. For all such sequences Oϕ and Oψ, we take summation over the probabilities P(OψOϕ) of the joint outcomes OψOϕ. A nonzero value of this sum implies that |ψ〉 and |ϕ〉 are not same. For this reason, we call this sum as the ‘probability of failure’. By subtracting it from unity, we get the ‘success probability’ which we denote by P(0). A higher value of P(0) implies higher overlap between two states. When |ψ〉 and |ϕ〉 are same, the success probability is 1. The fidelity and the overlap can again be calculated using Eqs. 1 and 2. For instance, for two-qubit states the probability of failure is obtained by summing over the probabilities of the outcomes 0101, 0111, 1010, 1011, 1101 and 1110. For each of them, the first two-qubit correspond to the measurement outcomes of qubits from one state, and the last two bits from the other state. In ideal systems, the swap test and destructive swap test will give same success/failure probability to distinguish between two images. However, it is not the case for noisy systems.

Fig. 4
figure 4

The circuit diagram of destructive swap test for (a) single qubit states and (b) three-qubit states. (c) The destructive swap test circuit after transpilation in ibmq_manila, for two single qubit states, both being |0〉. Here ‘cr2’ stands for two bit classical registers

Full size image

4 Destructive swap test for pattern recognition

In this section, we present a comparative study between the swap test and destructive swap test in real quantum processors as well as in the ideal qasm_simulator provided by IBMQ. We use binary images where the black pixels compose a ‘pattern’ against the background of white pixels. Our goal is to identify this pattern in a certain image. We start with small images with 2 × 2 and 2 × 4 pixels, as shown in Fig. 5a and 5b. The black and white pixels correspond to pixel values ‘1’ and ‘0’ respectively. The images are encoded using 2 and 3 qubits respectively. As shown in the previous section, the swap test in these systems is highly noisy. We want to check whether destructive swap test can reduce the effects of noise. For 100 runs of the same circuit in the real quantum processor ibm_lagos, we calculate the mean overlap \(\mathcal {I}_{\text {mean}}\) and the standard deviation σ. For comparison, we also show in Fig. 5a and 5b the theoretical value of the overlap, and that obtained from qasm_simulator. For the two-qubit images, the result shows 7% error to distinguish between same images, which is a significant improvement over Fig. 3. For the three-qubit images, this error is around 31%, which is still an improvement over the swap test. Also the relative behaviour of \(\mathcal {I}_{\text {mean}}\) for different reference images is consistent with the actual overlap, and hence the destructive swap test can be faithfully used for pattern recognition. This also demonstrates that, even though the number of measurements increases, the noise in the output is much less compared to swap test. However, the destructive swap test needs a more complicated classical post-processing of the measurement outcomes. Thus, there is a trade-off between swap test and destructive swap test in terms of classical post-processing, and number of quantum gates used in the circuit. The reader may note that the overlap between two same images obtained from a real quantum device can vary little bit depending on the particular quantum image states being considered. This is because the number of gates used to prepare different initial states can be different; hence, the effect of noise can vary.

Fig. 5
figure 5

Destructive swap test for pattern recognition in (a) two-qubit and (b) three-qubit images. In each vertical panel, the image on the left is the target image, and the one in the right is the reference image. The y-axis in the left denotes the overlap and the one in the right denotes the standard deviation corresponding to real quantum processors. The legendbox in (b) applies to both the figures. The mean and standard deviation are calculated over 100 runs of the circuit in the real quantum processor ibm_lagos

Full size image

To perform destructive swap test for larger images, we need access to higher dimensional real quantum processors. The largest systems we have access to are the 7 qubit IBMQ processors. A way out is to divide the original image into smaller segments each of which can be encoded using the available few-qubit systems. Then one can apply destructive swap test on the pair of segments belonging to the same coordinates from the target and the reference image. In case of binary images, there can exist segments on which all the pixels have pixel values 0. Using our encoding method, we cannot encode these white blocks as valid quantum states because ci = 0, ∀i. Hence, we run the circuit only when both the blocks corresponding to target and reference image have at least one pixel with non-zero pixel value. For each such pair of blocks, we calculate the overlap. One drawback of this segment-wise approach is that it does not calculate anymore the distance between the full quantum states encoding the images. Also by taking this approach, we exclude those cases when one among the pair of blocks is white and another has black pixels. This exclusion induces error in the results.

In other words, a particular pattern can have same number of pixels in common with two or more different patterns, e.g. in Fig. 6, the triangle has the same pixels overlapping with the square, the circle and with itself. Hence, all those reference patterns will correspond to exactly the same number of blocks contributing to the overlap with the triangle. Thus, to detect the closest pattern, one should also take into account the white blocks which are not common between the target and reference pattern. For this, we take summation of all segment-wise overlaps, and divide it by \(\max \limits \{N_{1}, N_{2}\}\), where N1 and N2 are the number of blocks having non-zero pixel values corresponding to the patterns in the target and reference image respectively. We call this ‘average overlap’ \(\mathcal {I}_{\text {avg}}\), and use this to capture the closeness between two patterns. We show the results in Fig. 6, where we divide the original 32 × 32 pixels images containing simple geometric patterns into 2 × 2 pixels segments, and apply destructive swap test on the 16 × 16 pairs of segments with same coordinates from the target and reference image. The data corresponding to real quantum processors have been produced using ibmq_manila. We also have presented the actual value of the overlap between the full quantum image states and those obtained from the qasm_simulator. Clearly, the average overlap is highest when the patterns from both the images match exactly. In this case, there is around 20% error in the overlap corresponding to real quantum backends. In the worst case, there is around 40% error in case of square-cross overlap. However, despite the noise, the comparative behaviour of \(\mathcal {I}_{\text {avg}}\) is consistent with that of the actual overlap between the patterns. This segmenting process indeed needs significantly higher number of measurements and computational time. However, as the computational power of quantum computers improves more and more over the years, we will be able to select bigger segments from original images, eventually being able to encode the whole images using the available qubits. To check the validity of this segmenting process for bigger than 2 × 2 blocks, we increase the block size step by step, and compare \(\mathcal {I}_{\text {avg}}\) between the same pair of patterns as in Fig. 6. Due to the size limitation of accessible real quantum processors, we use qasm_simulator to simulate the circuits, and the results are presented in Fig. 7. For every block size, the comparative behaviour between different pairs of patterns remains consistent with the ideal situation. There exist small anomalies, e.g. for the \(\mathcal {I}_{\text {avg}}\) of square-circle pair when the blocks are 8 × 8 pixels. In this figure, the errors arise not because of gate noises, but because Iavg is not entirely equivalent to the overlap \(\mathcal {I}\) between full quantum images. For example, comparing \(\mathcal {I}_{\text {avg}}\) from Fig. 6 to \(\mathcal {I}_{\text {avg}}\) in Fig. 7 for square-circle overlap, the error can vary between 3% minimum to 20% maximum depending on block size. For two same images, there is no error due to averaging process. However, the comparative behaviour again indicates that the destructive swap test is faithful for pattern recognition, and \(\mathcal {I}_{\text {avg}}\) can be used for large dimensional images with simple binary patterns.

Fig. 6
figure 6

The destructive swap test applied to compare the overlap between different geometric patterns in binary images. Each image is 32 × 32 pixels. The two images placed next to each other in a vertical column are compared. In each column, the green circle denotes the ideal overlap, the black plus denotes the overlap obtained from the IBMQ qasm_simulator and the red star is the average overlap obtained from the real quantum processor ibmq_manila by dividing the original image into 2 × 2 segments. The vertical axis is used to plot both the overlap for full images (I), and the average overlap over the segments (Iavg)

Full size image
Fig. 7
figure 7

The variation of the average overlap \(\mathcal {I}_{\text {avg}}\) with varying dimension of the blocks used to divide the original 32 × 32 images. The results are obtained by using qasm_simulator. Different markers and colors correspond to comparison between different pairs of geometric shapes as indicated in the legend

Full size image

4.1 Noise robustness

Due to the impossibility of completely removing the presence of noise, the real IBMQ processors are subject to gate errors, state preparation and measurement errors, and readout errors. The former two are due to imperfect implementation of gates, and the latter two are due to imperfect measurements. As all the IBMQ processors are calibrated on certain time intervals, the values of these errors fluctuate. To understand the robustness of our results against these fluctuating errors, it is important to check how the success probability of destructive swap test changes with changing noise (Martina et al. 2022). Since we cannot manually change the errors in IBMQ processors, we resort to the simulation using the “NoiseModel” class’ provided by IBMQ simulators. The IBMQ Aer_simulator copies the error values as well as other system parameters like topology, coupling and basis gates from a user-specified real quantum processor, so that the simulation results mimic what is obtained from that real device. It is also possible to customize the noise for all the basis gates, and then use the qasm_simulator to simulate the results. For example, one can choose among depolarizing noise, Pauli noise etc., vary the noise strength and select gates to which the noise is to be applied. With this approach, we study three cases, i.e. (i) the single qubit gates are significantly noisy, (ii) the two-qubit gates are significantly noisy and (iii) all the gates are equally noisy. In all three cases, we consider a depolarizing noise. A quantum state ρ subject to depolarizing noise evolves to a mixture of itself and the maximally mixed state, i.e. \(\rho \rightarrow (1-p)\rho + p\frac {I}{4}\). Here p is the noise strength which satisfies the bound \(0 \leq p \leq 1+\frac {1}{d^{2} - 1}\), d being the dimension of ρ, and I is the identity operator. Depolarizing error is the most general error in a qubit, which models the decoherence process. It takes into account the effects of all the three Pauli noise channels, i.e. bit-flip, phase-flip and bit-phase flip. Since the Pauli operators span SU(2), an error-correcting code for depolarizing error can correct any other SU(2) error. This is one of the reasons we choose depolarizing noise for studying noise robustness. We vary p from 0.05 to 1.05 and calculate the overlap between two identical 2 × 2 image by destructive swap test. Our results are shown in Fig. 8. For this study, we keep the readout error at the constant value of 0.01, which is of the same order of magnitude as the typical readout errors in real quantum processors. As we can see from the figure, the depolarizing noise in single qubit gates is more detrimental than the depolarizing noise in two-qubit gates for destructive swap test. When all the gates are noisy, then the circuit performance worsen, as expected.

Fig. 8
figure 8

The behaviour of the overlap \(\mathcal {I}\) between two identical two-qubit image states, with varying strength of depolarizing, amplitude damping and phase damping noise in the circuit gates. The blue curve corresponds to the case when the depolarizing noise in the two CNOT gates is varied, keeping all the single qubit gates at very low noise. The green one is for the case when the CNOT gate depolarizing noise is low, but all the single-qubit gate depolarizing noise varies. The red curve is when the depolarizing noise of all the gates are varied together. For the amplitude damping and phase damping noise, the curves corresponds to only the single qubit gate noises, while the CNOT gate noise is kept at a very low depolarizing noise

Full size image

Two other important noise models for a qubit are amplitude damping and phase damping channel. For completeness, we also present the behaviour of the overlap between two identical states with varying amplitude damping and phase damping noise strength in single-qubit gates. The overlap seems to be most robust against phase damping noise. The standard noise models in IBMQ does not provide amplitude damping and phase damping channels for two qubit gates, and we do not consider these cases to avoid constructing the complicated Kraus operators. The fluctuation in the overlap value reduces for higher values of noise strength, but in that regime, the value of \(\mathcal {I}\) is also small. However, for the IBMQ systems that we used, the single-qubit gate errors are always well below 0.01, and the two-qubits gate errors are below 0.1, which implies that \(\mathcal {I}>0.55\) for two-qubit states in this noise range. Since we do not know the exact nature of the complex noise present in the IBMQ real devices, the actual results obtained from them are not exactly the same as shown in Fig. 8. However, the noise simulation gives an idea about the overall behaviour of the protocol with changing noise at different gates.

We also study the success probability by varying the readout error r. The readout error is the fraction indicating the number of times a ‘0’ or ‘1’ in the output is measured erroneously as ‘1’ or ‘0’. The result is presented in Fig. 9. The gate errors corresponding to all the gates have been kept at 10− 3 which is of the same order of magnitude as the lowest gate errors in real processors. Typically the readout error in these processors is of the order of 10− 2.

Fig. 9
figure 9

The behaviour of the overlap \(\mathcal {I}\) between two identical two-qubit states with varying readout error strength, obtained using destructive swap test

Full size image

4.2 Feature recognition in MRI images

To consider a more practical application, e.g. in biomedicine, we repeat the destructive swap test to detect a particular feature in an image of human blood vessel obtained from MRI in our lab. The original images are colored images. We choose a threshold value to binarize the image so as to reveal all the details of the vessel against the background of Formaldehyde. In this particular case, we want, for instance, to identify a cavity present in the tissue of the smaller vessel in the left of the image (see Fig. 10a). To do so, we compare different sections of that blood vessel with a sample image of a cavity. The later is extracted from another MRI image of human blood vessel. In Fig. 10a, we have highlighted a few of the sections of the blood vessel which are compared with the reference cavity in Fig. 10b. We select 2 × 2 blocks from each image, belonging to the same coordinates, and calculate \(\mathcal {I}_{\text {avg}}\) using ibm_lagos. In Fig. 10c, we present the comparison of \(\mathcal {I}_{\text {avg}}\) with the theoretical overlap and that obtained from ideal qasm_simulator. In this case as well, \(\mathcal {I}_{\text {avg}}\) has highest value when the target and reference cavities match very closely, proving again the utility of destructive swap test, as well as the average overlap measure defined by us. This also opens up the possibility of pattern recognition in medical images using destructive swap test.

Fig. 10
figure 10

Destructive swap test performed in real quantum processors to detect a cavity in the MRI image of human blood vessel. (a) (In the left) The full image after conversion from colored to binary. (In the right) A zoomed-in segment of the smaller vessel containing the cavity to be identified. The five rectangular areas with red outline are compared with the reference cavity. (b) The reference cavity, obtained from another similar MRI image. (c) The theoretical overlap \(\mathcal {I}\), the overlap from qasm_simulator and the average overlap \(\mathcal {I}_{\text {avg}}\) obtained from ibm_lagos when comparing (b) with the six segments in (a)

Full size image

5 Pattern recognition in grayscale images

While binary images are the simplest image prototypes, in practice, the images acquired for quantum image processing tasks are mostly grayscale or with an RGB color profile. Hence, it is important to investigate whether the destructive swap test can efficiently identify patterns in grayscale/RGB images. Irrespective of the pixel values, we can encode these images in a quantum state using QPIE method, and then apply the destructive swap test to measure the overlap between them. First, we use MNIST images as the prototype for grayscale images. MNIST images are a large set of images, frequently used in machine learning and pattern recognition algorithms (Deng 2012). Each image is 28 × 28 pixels, containing a handwritten digit between 0 and 9. The dimension of these images is not an integer power of 2; hence, they cannot be encoded using QPIE method. To tackle this, we add rows and columns all having black pixels to the original images, to increase the dimension to 32 × 32 pixels, which can be encoded using 10 qubits. As our target image, we pick a particular image containing ‘0’ as the pattern, and compare it with a set of other images with different numbers. We simulate the circuit in ideal qasm_simulator to obtain the overlap \(\mathcal {I}\) between full quantum images. Then we divide them into 2 × 2 segments to calculate \(\mathcal {I}_{\text {avg}}\) using the real quantum processor ibmq_manila. The results are presented in Fig. 11a, which shows that the behaviour of \(\mathcal {I}_{\text {avg}}\) for different reference images is consistent with the theoretical overlap and the simulated one. It suggests that the destructive swap test can identify images with similar grayscale patterns by picking the images with highest \(\mathcal {I}\) or \(\mathcal {I}_{\text {avg}}\). For example, the overlap of the target image with the reference image containing ‘6’ is close to the overlap with reference images containing ‘0’. This is expected because the images are handwritten and in some cases ‘6’ can resemble the circular nature of ‘0’. Of course, if the reference images are enough noisy, the destructive swap test results can indicate a wrong digit for the target image.

Fig. 11
figure 11

Comparison between the actual overlap \(\mathcal {I}\) between two quantum images, the overlap obtained from a qasm_simulator and the average overlap \(\mathcal {I}_{\text {avg}}\) obtained from real IBMQ processors. The plots are for (a) MNIST numbers and (b) fashion MNIST images. Two images along a vertical column are compared. As real quantum processor, we use ibmq_manila

Full size image

We also run the same circuit for fashion MNIST images using the same real quantum processor. The result is presented in Fig. 11b. In this case, \(\mathcal {I}_{\text {avg}}\) is not as efficient as for MNIST numbers. Though the reference images with comparatively higher \(\mathcal {I}_{\text {avg}}\) matches well with those having highest overlap, choosing one particular closest pattern based on \(\mathcal {I}_{\text {avg}}\) may not exactly match with the actual closest pattern. This is also due to the higher complexity of the images, as it is clear from the pictures that the pixels in these images encode more shades of gray compared to MNIST numbers. However, the \(\mathcal {I}_{\text {avg}}\) in this case can still be used to distinguish between widely different categories of the fashion images, e.g. shirts and dresses.

6 Experimental demonstration of destructive swap test in diamond NVs

The IBMQ processors are based on superconducting qubit technology. To test the efficiency of destructive swap test in a different quantum system, we use two qubits in diamond NVs (Wrachtrup and Jelezko 2006; Suter and Jelezko 2017) to demonstrate the performance of destructive swap test. The experiment scheme is shown in Fig. 12. The demonstration starts with the pure state |00〉, which is prepared by the schemes proposed in the previous works (Zhang et al. 2019; 2020; Hegde et al. 2020; Zhang et al. 2021). In Fig. 12a, we show the structure of the NV center with a coupled 13C spin. Here we choose the electron spin in states with mS = 0 and mS = − 1 as qubit 1, and 13C spin as qubit 2, where mS denotes quantum number for the electron spin. Figure 12b shows the quantum circuit to demonstrate the destructive swap test.

Fig. 12
figure 12

Experiment scheme to demonstrate the destructive swap test in NVs. (a) Structure of the NV center with a coupled 13C spin. (b) Quantum circuit with the steps of the state preparation and destructive swap test, indicated by the green dotted boxes. The operation 𝜃β denotes an operation as \(\theta _{\beta }=e^{\mp i\theta I_{y}}\), corresponding β = π/2 and β = 3π/2, respectively. The observable \(\mathcal {F} = \vert \langle \phi \vert \psi \rangle \vert ^{2}\) is the population of the electron state |0〉. (c) The microwave (MW) pulse sequence for implementing the circuit in (b). The dashed lines indicate the correspondence between the operations in the circuit (b) and pulse sequence (c), where the first MW pulse is for 𝜃β, and the other three MW pulses are for the two Hadamard gates sandwiched by the CNOT gate, and the laser pulse detects \(\mathcal {F}\). The Rabi frequency of the MW pulses is 2 MHz. The pulse durations are t1 = 0.206, t2 = 0.122, t3 = 0.182 μ s, the pulse phases are ϕ1 = 317∘, ϕ2 = 273∘, ϕ3 = 90∘, and delays τ0 = 4.336, τ1 = 3.478, τ2 = 2.368, τ3 = 4.988 μ s

Full size image

We choose \(\vert \phi \rangle =\frac {1}{\sqrt {2}}(\vert 0\rangle +\vert 1\rangle )\), generated by applying a Hadamard gate H to state |0〉. We use the destructive swap test to measure the fidelity between state |ϕ〉 and various states |ψ〉, which is generated by applying an operation \(\theta _{\beta }=e^{\mp i\theta I_{y}}\) to state |0〉, where β = π/2 or β = 3π/2, respectively, and Iy denotes y component of the spin 1/2 operator. The fidelity \(\mathcal {F}\) is the population of the electron spin in state |0〉. The theoretical \(\mathcal {F}\) is calculated as

$$ \mathcal{F}_{th} = \frac{1}{2}(1\pm \sin\theta) $$
(3)

with maximums at 𝜃opt = π/2 or 3π/2, for β = π/2 or β = 3π/2, respectively. In Fig. 12c, we show the microwave (MW) pulse sequence for implementing the circuit in (b). The first pulse is for the operation 𝜃β, and the other three MW pulses are for the two Hadamard gates sandwiched by the CNOT gate. The laser pulse detects the observable \(\mathcal {F}\).

The experiment results are shown in Fig. 13, where subfigures (a) and (b) correspond to β = π/2 and 3π/2, respectively. The experiment data \(\mathcal {F}\) can be fitted by the theory \(\mathcal {F}_{th}\) as

$$ \mathcal{F} = a + b \mathcal{F}_{th} $$
(4)

where parameters a and b are obtained in fit as a = 0.04, b = 0.89 in Fig. 13a, and a = 0.10, b = 0.83 in Fig. 13b. The deviation between experiment and theory can be mainly attributed to the theoretical imperfection of the pulse sequence with theory fidelity 0.9 in the optimization of the pulse parameters, dephasing effects of the electron spin since the total pulse duration is up to 16.3 μ s and is comparable to \(T_{2}^{*} \approx 35\) μ s, and the statistics of the photon detection.

Fig. 13
figure 13

Experiment results for \(\mathcal {F}\) vs 𝜃β to demonstrate the swap test in NVs, where the flip angles and the phases in operation 𝜃β are indicated as the horizontal axis and in the panels. The vertical dashed lines indicate the angles corresponding the maximums fidelity in fit

Full size image

7 Supervised learning with destructive swap test

In classical computers, information is stored in a particular memory address of the RAM. These computers recognize a given input by knowing its exact location in the memory. The term associative memory is used to describe instances when an object is recognized not by accessing its location in the memory, but by associating the partial knowledge about it with the already stored information in the memory. This is similar to the functioning of human brain, e.g. while solving a crossword puzzle. A computational application of associative memory is artificial neural networks (Müller et al. 2012). Associative memory is advantageous over RAM while solving complex problems, but on the other hand requires huge capacity to store information for an efficient performance. In Trugenberger (2002), the author introduced the idea of quantum associative memory to be used for quantum pattern recognition. Besides exploiting the quantum speed-up, the protocols based on quantum associative memory can overcome the limitation of short storage capacity in classical systems. Inspired by Trugenberger (2002), a number of previous works have proposed quantum pattern recognition protocols which work in a similar way to classical supervised learning. In Lloyd et al. (2013), the authors proposed a machine learning protocol for assigning a particular class to a vector by optimizing the mean distance between the target and reference vectors. In Pham and Park (2014), the authors propose a similar protocol for pattern recognition by taking the k-nearest neighbours approach to optimize the distance between target and reference states. In this section, we devise a quantum machine learning protocol for pattern recognition which uses destructive swap test. The corresponding circuit for a three-qubit state is shown in Fig. 14. The target state |ψ〉 has been encoded using three quantum registers, which are represented by the black horizontal lines in the figure. Let us assume that there are d reference states ϕi (i = 1,2,...,d), each having the same dimension as |ψ〉, from which we have to detect the closest pattern to |ψ〉. The three quantum registers which can be used to encode these states are denoted by the blue horizontal lines in the figure. We take a d dimensional quantum system (qudit), the basis vectors of which can be associated with the d reference states. In Fig. 14, the qudit is represented as the red horizontal line. Now we prepare the following superposition,

$$ \vert {\Phi}\rangle=\frac{1}{N^{\prime}}\sum\limits_{i}\vert \phi_{i}\rangle\vert q_{i}\rangle, $$
(5)

where qi (i = 0,1,...,d − 1) is the i th basis vector in the Hilbert space \({\mathscr{H}}^{d}\) of the qudit, and \(N^{\prime }\) is the normalization constant. The blue and the red quantum registers are initialized in the joint state |Φ〉. There is an auxiliary qubit which is initialized in the state |0〉, represented by the green horizontal line in Fig. 14. Thus, the initial state of the total system is,

$$ \vert {\Psi}_{0}\rangle=\frac{1}{N^{\prime}}\vert \psi\rangle \otimes \sum\limits_{i}\vert \phi_{i}\rangle\vert q_{i}\rangle \otimes \vert 0\rangle. $$
(6)

Now we apply the gates in accordance to the destructive swap test, i.e. CNOT gates on each pair of qubits from target and reference states, followed by Hadamard gates on the qubits of the target state. After this the total state is,

$$ \begin{array}{@{}rcl@{}} \vert {\Psi}_{1}\rangle&=&\frac{1}{N^{\prime}}\sum\limits_{i}({C^{i}_{1}}\vert 000000\rangle+{C^{i}_{2}}\vert 000001\rangle+... \\ &&+C^{i}_{64}\vert 111111\rangle)\otimes \vert q_{i}\rangle \otimes \vert 0\rangle, \end{array} $$
(7)
Fig. 14
figure 14

The quantum circuit of supervised learning algorithm based on destructive swap test, to detect the closest pattern to a given pattern

Full size image

where \({C_{j}^{i}}\) (j = 1,1,...,64) are the coefficients of the vector in the joint Hilbert space \({\mathscr{H}}^{6}\) of target and reference states associated with |qi〉. We remind that the basis vectors in this space for which the pairwise AND of the bits have odd parity, contributes to the failure probability. To differentiate between these basis vectors and the rest, we apply CCNOT gates such that each pair of the target and reference state qubit acts as the control qubits, and the auxiliary qubit flips its state when both of the former are in state |1〉. Thus, the additional qubit keeps a count of the parity. If the basis vector has odd parity, the auxiliary qubit has final state |1〉. In case of even parity, the qubit state remains |0〉. The total state after applying the CCNOT gate can be written as,

$$ \begin{array}{@{}rcl@{}} \vert {\Psi}_{2}\rangle\!&=&\!\frac{1}{N^{\prime}}\sum\limits_{i}\big[\big({C^{i}_{1}}\vert 000000\rangle + {C^{i}_{1}}\vert 000001\rangle + .. \big)\!\otimes \vert q_{i}\rangle \!\otimes \!\vert 0\rangle \\ &&+\big(C^{i}_{16}\vert 001111\rangle + C^{i}_{19}\vert 010010\rangle \!+... \big)\otimes \vert q_{i}\rangle \!\otimes\! \vert 1\rangle\big]\\ \end{array} $$
(8)

Thus, the qudit keeps the memory of the reference images, whereas the auxiliary qubits keeps record of the basis vectors contributing to success and failure probabilities. Now, repeated joint measurement on the qudit and the auxiliary qubit can reveal the success and failure probabilities corresponding to each reference image. The one having the largest success probability is closest to the target image. In Fig. 15, we show the measurement outcomes as a histogram when a two-qubit target image is identified by comparing it with four reference images. For this task, we use the same images as in the first four columns of Fig. 5. We denote the target image as |S1〉, and the set of reference patters as {|S1〉,|S2〉,|S3〉,|S4〉}. To label the reference states, we use the basis states {|00〉,|01〉,|10〉,|11〉} of a two-qubit system and prepare the following superposition,

$$ \vert {\Phi}\rangle=\frac{1}{N^{\prime}}\big(\vert S_{1}\rangle\vert 11\rangle + \vert S_{2}\rangle\vert 01\rangle + \vert S_{3}\rangle\vert 10\rangle + \vert S_{4}\rangle\vert 00\rangle\big) $$
(9)

Since the target image is |S1〉, we expect that at the end of the protocol described above, the joint measurement on the two-qubit system and the auxiliary qubit will give highest probability corresponding to the state |110〉. Here the first two bits correspond to the two-qubit system, whereas the last bit corresponds to the auxiliary qubit. The height of the bar plots in Fig. 15 is consistent with the expectation. The circuit in this case has been run using a 7 qubit IBMQ qasm_simulator.

Fig. 15
figure 15

The supervised learning protocol based on destructive swap test performed to detect the closest pattern of a two-qubit target image from a set of four reference images shown also in Fig. 5. The horizontal axis shows the classical bits corresponding to the measurement outcomes from the joint system composed of the qudit (in this case four dimensional) and the auxiliary qubit. The vertical axis denotes the corresponding probabilities. The success probability corresponding to the basis |110〉 is the highest, which corresponds to the case of equal target and reference images

Full size image

8 Conclusion

Image processing is indispensable in a plethora of everyday applications. Some quantum image processing algorithms have been studied in theory, which show polynomial and exponential speed-up over their classical analogues. There exists a number of quantum image encoding methods, one of which (QPIE) encodes the pixel positions and pixel values as the basis vectors and corresponding amplitudes of a quantum state. This naturally implies that swap test, which calculates the closeness between two quantum states, is a plausible tool to identify similar patterns in quantum images. In this paper, we investigated the efficiency of swap test for this task in real quantum processors. Our results showed that for the typical values of the gate errors present in IBMQ quantum machines, the output of swap test becomes completely noisy corresponding to three or more qubits. For lower dimensional states, it can still faithfully identify the pattern closest to a given one. To mitigate the effect of noise, we propose to exploit the destructive swap test, which showed considerable improvement for the three-qubit states. With access to larger quantum processors, it will be interesting to see whether the destructive swap test still remains successful against the noise for higher dimensional states. Considering the fact that a simple circuit such as swap test becomes completely noisy for three qubits, it is questionable how much practically useful the larger IBMQ processors will be for moderately complex algorithms. A scaling analysis with respect to dimension of the image states can be performed to investigate this question. To apply destructive swap test to larger images, we divided them into smaller segments which can be encoded using the 5 or 7 qubit quantum devices. The segment-wise overlaps were aggregated by defining an average overlap, which though not being equivalent to the actual distance between full quantum image states, remained consistent while distinguishing between two very similar or very different ones. The above findings remain true for both binary and grayscale images. We tested the destructive swap test for identifying a cavity in the binary image of human blood-vessel obtained by MRI, which again showed consistency, thus implying possible applications of this protocol in biomedical fields. As observed from the results involving simple and complex binary patterns, and grayscale MNIST images, the efficiency of average overlap may diminish with increasing range of pixel values and the complexity of the pattern. However, if it is possible to encode the full quantum states using the real IBMQ processors with suitably controlled noise, the destructive swap test is without any doubt a potentially useful tool for pattern recognition, as clear from our results in this work. Besides IBMQ devices, we also used the nitrogen vacancy center in diamond to exhibit the destructive swap test for single qubit states, which shows good agreement with theoretical results. A future aim will be to find a suitable function of the segment-wise overlaps, which is equivalent to the distance between the full images. Also an important objective will be to find whether the machine learning tools can be used to counteract the noise in the outputs. We believe that the destructive swap test based protocol can be used for pattern recognition in all the relevant fields and for quantum machine learning algorithms.