Semi-supervised time series classification method for quantum computing

In this paper we develop methods to solve two problems related to time series (TS) analysis using quantum computing: reconstruction and classification. We formulate the task of reconstructing a given TS from a training set of data as an unconstrained binary optimization (QUBO) problem, which can be solved by both quantum annealers and gate-model quantum processors. We accomplish this by discretizing the TS and converting the reconstruction to a set cover problem, allowing us to perform a one-versus-all method of reconstruction. Using the solution to the reconstruction problem, we show how to extend this method to perform semi-supervised classification of TS data. We present results indicating our method is competitive with current semi- and unsupervised classification techniques, but using less data than classical techniques.


Introduction
The field of quantum computing has experienced rapid growth in recent years, both in the number of quantum computing hardware providers and their respective processors' computing power. Companies such as D-Wave Systems, Rigetti, and IBM offer access to their quantum processors, and their use in proof-of-concept demonstrations has been widely discussed in literature. Quantum processing units (QPUs) have been used to solve a wide variety of problems such as traffic flow [39], logistics and scheduling [54,50], quantum simulation [51,37,20], and more [53,41]. Notably, a recent study by Google showed how their QPU can perform the task of sampling from random quantum circuits faster than state-of-the-art classical software [7], ushering a new era in the field of quantum computing. These applications use so-called noisy intermediate scale quantum (NISQ [43]) processors to solve various forms of optimization and sampling problems. Most commonly, the problem is formulated as a quadratic unconstrained binary optimization (QUBO) problem, or its equivalent form of an Ising Hamiltonian. The former uses a basis of binary {0, 1} Therefore, the task of programming a quantum annealer or QAOA circuit involves finding a suitable Ising Hamiltonian representation for the optimization task. In this paper, we motivate classification of time series data based on extracting features which exist within the data, and use combinatorial optimization techniques to match and reconstruct data with other time series. We start by reducing the dimensionality of the TS data and encode it as a string. We introduce a pulling procedure, comparing the encoded strings to form a collection of sets, where common features between the strings are extracted. All extracted common features are pooled together, which we can then use to construct new TS and compare between existing ones. We perform these tasks by using the pulled features as elements of the universe in the set cover problem, which has a known QUBO/Ising formulation and can be solved using quantum computers. By reformulating the critical task in our clustering algorithm as a set cover problem, we introduce two novel ideas to quantum clustering algorithms: (1) we avoid representing single vectors with polynomial numbers of qubits, instead representing the features within the data as the qubits, and (2) we perform the clustering task by transferring the core concepts of clustering (and reconstruction) to the quantum algorithm for set cover, as opposed to a direct translation of a distance-based minimization procedure.
The rest of this paper is organized as follows. Section 2 provides a short overview of existing methods for both classical and quantum clustering. Section 3 motivates the task of TS reconstruction, explains the methods used to discretize the data, and how to convert the discretized data to the set cover problem and its representative QUBO. Section 4 shows how to extend the reconstruction method to classify the TS data. Section 5 outlines the experiments performed to test the developed method using various open-source data sets, and conclusions are presented in Section 6.

Previous works
Quantum computing-based approaches which exist in literature involve fundamentally different approaches than that introduced in this work; we provide a brief overview of some key methods and algorithms related to quantum clustering. Assuming the existence of error-corrected quantum processors (and the existence of quantum RAM), it has been shown that quantum computers could perform k-means clustering exponentially faster than their classical counterparts [35]. Other works have also shown how to reformulate parts of classical clustering algorithms as quantum subroutines that can be executed on error-corrected gate-model QPUs [6,4,57,25]. In quantum annealing, a similar approach has been shown in which the objective function of the clustering task (minimizing distance metrics between high-dimensional vectors) has been directly translated to a QUBO, with each vector's possible assignment represented via one-hot encoding to physical qubits [31,40].
Classical time series (TS) analysis is considered to be a challenging task due to high number of dimensions involved resulting in the Curse of Dimensionality phenomenon. A series of works address the question of efficient dimensionality reduction [30,33,34,48,47,42,21,16], explaining the trade-off between information loss and search space size. Main results presented in this manuscript are obtained with Symbolic Fourier Approximation (SFA) method [47] due to its pruning power, noise-robustness and scalability. SFA represents each real-valued TS in a frequency domain by a symbolic string using the discrete Fourier transform. These transformed TS can then be used by classical string-based similarity algorithms such as phonetic distance based, Levenshtein, Hamming, Jaro, Jaro-Winkler measures, and more [18].
Classical TS clustering techniques can be split into the following categories: model-based, feature-based, shape-based and their combinations [2]. In the model-based approach the TS is encoded and fit by parametric models and clustering is applied to these extracted parameters [32]. In feature-based methods, the features of TS, like Fourier components, periodicity, trend, number of peaks, variance, etc., are extracted and later clustered by conventional algorithms [23,11]. Shape-based approaches refer to comparing shapes of TS directly and matching them according to specifically chosen metrics. A typical example for this approach is Dynamic Time Warping (DTW) [46], which has been shown to outperform Euclidean metrics [12,56]. DTW-based classical methods are used to evaluate the accuracy of our approach in Section 5. For more details on classical approaches to TS clustering, we refer the reader to [19,13,2].

Time Series Reconstruction: Problem Formulation
Clustering techniques generally require specific data representation, similarity measure definitions, and clustering algorithm selection. Similarly, in our quantum computing based approach, we represent the TS data as encoded strings from which we formulate semi-supervised clustering and optimal reconstruction as a set cover problem, and provide metrics based on solutions to the set cover problem. While different than classical approaches [26,1,17,49], we preserve the computational complexity of the problem, while introducing a method that is based on latent features within the data.
In order to reconstruct given time series data, we start by discretizing the data, and comparing the encoded strings to generate the elements of our universe to form the set cover. This pulling technique is crucial to allow feature-wise comparison of the data, as well as arbitrary reconstruction of TS using existing (or training) data. We use existing techniques for discretization, and explain the pulling procedure in detail. We then show how to use this data to construct the set cover problem for quantum optimization.

Discretization and Pulling Technique
There are many ways to discretize time series data, as reviewed previously. For our purposes, we use the symbolic Fourier approximation (SFA) method [47], as it provides differentiation between separate TS classes and features in high-dimensional data sets, allowing us to use these representative symbols for our quantum algorithm. Nevertheless, the exact discretization is datadependent, with various hyperparameters (such as number of letters in the alphabet, length of each encoded string, etc.) present in the method; for a full explanation we refer the reader to [47]. Given the encoded strings, we compare the time series using the following pulling procedure, illustrated in Figure 1. This pair-wise comparison is considered a preprocessing step necessary to formulate our set cover problem. Starting with one fixed string (red in the figure), we consider each encoded character as an independent element in the universe set 1 (U = {0, 1, 2, 3, 4} in the figure). A second string (green in the figure) is compared element-wise by successively moving the second string along the first, as illustrated. At every iteration, all character matches between the two strings are recorded as a new set. In the example from Figure 1, the set of sets is The procedure is repeated for the rest of the encoded training TS to form the set of sets V . This set, which is a union of all subsets obtained via the pulling technique, represents the features in common between the target time series and all other time series in the data set. Given this aggregate set, the goal is now to select the minimal subset that most closely reconstructs the universe, which is the NP-hard set cover problem. In the case illustrated in Figure 1, the optimal selection of subsets is underlined in red. In principle, solutions of this set cover problem do not preserve order of elements, and allow the use of the same element multiple times. This feature is useful for TS comparison, as elements of the time series data can be permuted and duplicated without affecting our reconstruction method.

Formulating the Set Cover Problem
Given the encoded strings and the common set of features V , we can now formulate the set cover problem as a QUBO, following the method demonstrated in [36]. Consider the universe set U = {1, ..., n}, and a set of Finding the smallest number of subsets V i whose union is U is a well-known NP-hard optimization problem in the worst case [29]. In order to map the set cover problem to a QUBO problem, we use the following binary variables: and Here, α ∈ U denotes an element of universe set, and m signifies if element α appears in m subsets.
We consider the full QUBO as a sum of two components: and The complete QUBO is given by H = H A + H B [36]. The first summation in H A imposes that exactly one of x α,m must be selected in the minimum via a one-hot encoding. The second term in H A represents the number of times α is selected, and that this is equal to the number of selected subsets α appears in (m, as only one x α,m can be 1 in the minimum). The final term H B (5) serves to minimize the number of V i needed to cover the universe U . The total number of variables required is N + n(1 + M ), where M is the maximal number of sets that contain given element of U (see [36] for details). The limiting case where each element of V i included covers only one element of U constrains the coefficient of H A and H B to 0 < B < A. The closer the coefficient B and A, the more weight is given to (5), minimizing the number of elements selected from V .
In our application of time series reconstruction, the final size of the QUBO is heavily dependent on our choices during discretization. For example, the number of binary variables is equal to N train TS (2L − 1) (L + 1), where N train TS is the number of TS in the training set used for reconstruction, and L is the length of string that encodes the TS. Increasing the string length to encode each TS changes the size of the universe U . Allowing longer encoded strings to represent the data creates more subsets V i . Therefore, there exists a trade-off between the granularity of the encoded strings and the ability to solve the set cover representation of the problem. Including more characters in our alphabet for discretization changes the non-empty sets V i , which the number of quadratic elements in the QUBO depends on. The general trend is, however, that the number of the quadratic element decreases with the increase of the characters used in our alphabet. This could be explained due to the properties of the pulling procedure described above, since a smaller alphabet produces more non-empty elements V i which could be used for reconstruction of the universe U . In Figure 2 we show how varying these hyperparameters of the discretization affects the size of the QUBO problem, based on 20 test samples from the BeetleFly data set [24]. We encode both the training and test data sets into strings using the method described in Section 3.1. We then perform the reconstruction procedure for every TS in our test set using the entire training set. Each TS from the test set is assumed to individually form a universe U , and is to be reconstructed using the sets V i , obtained via the pulling procedure. Explicitly, using Figure 1, the red string is the TS from the test data set, and all strings in the training set are pulled through (greed strings) to obtain the V i 's. This allows us to compare every test TS to the full training set in one-versus-all manner. Then, using the universe U and V i 's from the pulling procedure, we formulate the set cover problem outlined in Section 3.2. Thus, a single solution to that set cover problem (even sub-optimal in the worst case) allows us to reconstruct each TS from the test set using a set of discretized features obtained from all elements which appear in the training set. Furthermore, since annealing-based sampling methods produce finite-temperature Boltzmann distributions [45], various optima of the set cover problem could yield different ways to reconstruct the test TS using the training set. Due to this, it is therefore the users' task to use these reconstructions to associate each test TS with a label from the training set. We outline the steps of our classification procedure using pseudo-code in Algorithm 1.

Algorithm 1 Semi-supervised QUBO-based clustering algorithm
Dimensionality reduction Apply encoding of all TS into a string of characters using SFA [47] or PAA [21]-like methods. for TS in test data do Produce set of sets for cover problem Assume encoded TS forms the universe U . Apply puling procedure (Figure 1) to the full training data set to compute V and map the problem to the set cover problem.

Formulate the QUBO problem
Generate set cover QUBO an according to Section 3.

QUBO optimization
Find minimum of the QUBO; record subsets V i selected to cover U .

Analyze results of QUBO optimization
Trace back selected V i to respective time series from training set, and assign label to TS based on selection metrics. end for To classify the reconstructed test TS data we evaluated three different similarity metrics using set cover solutions: largest common subset V i , highest number of common subsets V i , and largest sum of common elements in selected V i . We briefly explain how each metric is calculated, and discuss the performance of each.
• Largest common subset. Given a candidate solution to the set cover problem, the label corresponding to the V i which contains the most elements is selected. The label is then assigned to the test TS. This metric captures the longest continuous set of features from the training TS data, and assumes that is sufficient to determine the label.
• Number of common subsets. Frequently, multiple V i 's from the same training TS are used to reconstruct a test TS. In this metric, we count the number of V i subsets used to cover the universe. The test label is assigned the same label as the training TS which appears most frequently in the set cover solution.
• Largest sum of subsets. This metric is a combination of the previous two. For every training TS that is used to reconstruct a test set, the total number of elements used by each is counted (summed over all V i 's). The label which corresponds to the training TS with the largest sum is assigned to the test TS.
These metrics allow us to quantify the accuracy of our semi-supervised clustering algorithm. The first two metrics, being based on large sets of common features between the TS, performed the best (results shown in the next section). There was no significant difference between the two metrics, and the superiority of one metric over the other varied between data sets. The third metric, which was a combination of the first two, performed worse than either of the first metrics in the majority of the cases tested. While unexpected to begin with, this observation could be explained by the fact that because the third metric admits matches with many small subsets V i that are selected in the set cover, this metric could miss significant signatures present in the TS data. The largest common subset metric was selected for the experiments presented in the next section. It should also be noted that the use of labeled training data is not designed to not reach the accuracy of supervised learning methods. Moreover, there are modifications that could be made to the methods presented to improve the accuracy, for example increasing the word length and/or using a larger train set. Both are constrained in our use-case to prohibit excessively large QUBOs from being constructed. The goal of this method, as described, is to allow for relatively high accuracy using small sets of training data.
We provide an illustrative example of our QUBO-based reconstruction and classification in Figure 3 using the BeetleFly data set. The task is to reconstruct the data in Figure 3  where U ='ACEEB', the test TS data to reconstruct. For classifying the reconstructed sample, we refer to the classes of the training data used for the reconstruction, and note that the training samples in Figure 3 (b) and (c) belong to two different classes. Using the similarity metrics defined above, it is easy to determine that v 1 and v 2 both originate from the time series (b), whereas only v 3 (which contains only a single element) is obtained from (c). Therefore, (a) is assigned the same label as (b). This example is representative of the majority of cases encountered during classification, with components of the reconstructed TS varying across multiple training samples, and often also across multiple classes.

Experiments and Results
The experiments performed in this work used open-source labeled TS data available publicly [8,9]. We restricted our analysis to univariate TS data with two classes and small training set size to make our work amenable to NISQ devices in the near future. However, this method of semisupervised classification can be used with any number of classes, at the cost of QUBO size. Since both the number of TS in the training data and the word length used to encode the TS contribute to the number of variables in the QUBO, we select data sets that have small numbers of TS in the training set. The exact sizes of the QUBOs for each data set are shown in Figure 4. The smallest QUBOs, from the Chinatown data set, were between 200-300 variables, and the largest were over 700 variables from the GunPoint data set.
Due to their size, the QUBOs were optimized using simulated thermal annealing with 20,000 samples and 1000 sweeps [14], which was sufficient to ensure that low-energy local minima were sampled. The specific parameters used for the TS encoding are shown in Table 1. In general, the longer the TS are and the fewer TS are in the training set, the finer the discretization method required to accurately classify the test data. In all data sets we were able to reconstruct each test TS with elements from the training set, as explained in Section 3.2. To measure the performance of our classification method, we compared the accuracy of our labeling to semi-supervised and unsupervised classical classification methods, explained in more detail below. The results of these experiments for the various data sets are summarized in Table 2.
As expected, the semi-supervised QUBO-based method outperforms classical unsupervised methods. To benchmark the results of our method, we use K-means clustering with pairwise DTW Figure 4: Distribution of number of QUBO variables for all data sets in Table 1. metrics calculated on the original TS (before encoding), with the labels being assigned based on belonging to one of two clusters. We also employ a classical semi-supervised method that is similar in spirit to the QUBO-based one, where the test TS labels are assigned by the DTW metric directly, calculated pairwise between each training and test TS (without encoding). We note however, that the QUBO-based method operates on a reduced dimensionality in contrast to the classical methods which use the original TS, where full information is preserved. Even under this consideration the accuracy of QUBO-based method is comparable with the semi-supervised DTW methods, and could be improved still by enriching the set V , i.e. by augmenting the training set or increasing the discretization granularity.
The worst performance of the QUBO-based algorithm is observed on the TwoLeadECG data set. This could be explained by the nature of our method, as well as the sensitivity of the ECG data. By using the set cover problem, we allow for permutations of subsets of TS data in the reconstruction of the test TS. It is likely that this permutation of TS segments, and similar representation in Fourier space of the signals from the two leads in the ECG measurements, makes our method not suitable for this kind of data. To confirm this is the case, and improve the classification accuracy, we applied SAX [48] encoding, based on sliding window time series magnitude, rather than the Fourier transform. Using this method, and encoding the TS as a word of length 5 constructed from a 5 letter alphabet, the accuracy is improved to 0.62 and 0.85 for the two respective classes (0.74 weighted average). In contrast to the Fourier encoding, the better results with ECG200 data are due to the significant differences between the classes of normal and ischemia ECG readings.
The highest accuracy is obtained using the BeetleFly and Chinatown data sets. In the first case, many permutations of the training set to construct the test set are permissible, which our method takes advantage of. The accuracy of our method is additionally improved by the relative size of the training set, further augmenting the combinatorial space of permutations. This robustness can also  be explained by the dimensionality reduction technique for this data set: the 2D BeetleFly images (with different orientations) were mapped to 1D series of distances to the image centre, which again is beneficial for permutation-based methods. The Chinatown data set, for comparison, contained significantly shorter TS than BeetleFly. Encoding the Chinatown TS data with the same word length as BeetleFly resulted in higher granularity representations, and ultimately higher accuracy. This provides additional evidence that the accuracy of our method can be improved by increasing the granularity of the encoding.

Conclusions
We present a QUBO-based method for TS reconstruction and semi-supervised classification that reaches accuracy scores comparable with classical DTW pairwise approaches, and in most cases outperforms unsupervised clustering. Among the advantages of our method is the utilization of significantly less data with respect to conventional classical methods, as well as a one-versus-all comparison that allows the selection of segments of data from multiple sources to reconstruct a single TS. This provides an additional robustness in our method in permutations of TS segments during the reconstruction. We showed how to reformulate the task of TS reconstruction as the set cover problem with a minimal number of subsets. In order to formulate this problem as a QUBO we apply TS dimensionality reduction by encoding each time series as a separate string. This encoding procedure and selection of comparison metrics (as discussed in Section 4) define the hyperparameter space of the problem. The QUBO-based classification method performed the best on image and traffic data, which is consistent with our method's inherit ability to utilize permutations of features/data within the TS to perform reconstruction.
Time series reconstruction and classification has a wide variety of useful applications, such as: management of energy systems, factory process control, sensor systems, and many more. The methods introduced in this paper show how to reformulate the tasks of reconstruction and classification of such data using quantum computing. The fact that our work uses small training sets of labeled data means that the QUBOs produced could be solved by next-generation NISQ devices. Using quantum technologies, this method could analyze significantly more complex TS data, even in a live setting. The results of the optimization process (the selected subsets used for the reconstruction) would be informative as feedback for live process optimization as well. Future work in this are will be focused on generalising the method to multivariate TS cases, finding application-ready data sets, and execution of the presented methods on quantum processors. Specifically, with the advancement of hybrid quantum-classical algorithms, we will focus on converting the methods presented in this paper to be suitable for commercial applications.