MISCELA: discovering simultaneous and time-delayed correlated attribute patterns

This article addresses a new pattern mining problem in time series sensor data, which we call correlated attribute pattern mining. The correlated attribute patterns (CAPs for short) are the sets of attributes (e.g., temperature and traffic volume) on sensors that are spatially close to each other and temporally correlated in their measurements. Although the CAPs are useful to accurately analyze and understand spatio-temporal correlation between attributes, the existing mining methods are inefficient to discover CAPs because they extract unnecessary patterns. Therefore, we propose a mining method Miscela to efficiently discover CAPs. Miscela can discover not only simultaneous correlated patterns but also time delayed correlated patterns. Furthermore, we extend Miscela to automatically search for correlated patterns with any time delays. Through our experiments using three real sensor datasets, we show that the response time of Miscela is up to 20.84 times faster compared with the state-of-the-art method. We show that Miscela discovers meaningful patterns for urban managements and environmental studies.


Introduction
Many cities have installed a wide variety of sensors to continuously and cooperatively monitor urban conditions, such as the distribution of air pollution, the transition of traffic volume, and the change of temperature. Municipalities analyze the 1 3 urban conditions and make a decision for the urban planning by using such sensor data. For example, Santander, Spain monitors the traffic volume within the city and informs people of the real-time traffic information [12,13]. The accumulated traffic data are used to several urban managements such as the traffic prediction, the road extension, and the traffic signal control. In these services, it is useful to discover sets of roads which are spatially close and whose traffic volume increases or decreases during the same periods (i.e., co-evolve). The problem is called the spatial co-evolving pattern mining (for short, SCP mining), which discovers sensors that are spatially close to each other and temporally co-evolving in their measurements. Since the SCP mining is useful for many applications such as the air pollution analysis in an urban area, several SCP mining methods have been proposed [2,17]. The SCP mining discovers meaningful patterns for analyzing urban environments.

Motivation
Many cities typically monitor multiple attributes, for example, Santander monitors temperature and traffic volume, and China monitors several types of particles such as NO 2 and meteorological data such as rainfall. Multiple attributes are useful to analyze urban environments from diversified viewpoints. To accurately analyze and understand the urban environments, it is expected to discover correlated attributes which are spatially close to each other and temporally co-evolving during both the same and different periods. Additionally, some attributes may co-evolve with time delay. That is, time-delayed co-evolving happens in many sensor data. We show examples of correlated attributes with time delay in the following.
Example Figure 1 shows an example of correlated attributes in a dataset of environmental sensors in China. There are two types of sensors; red ( s 1 ) and blue ( s 2 and s 3 ) points denote the sensors that measure rainfall and NO 2 , respectively. The left figure shows the locations of sensors and these sensors are spatially close to each other. The right figure shows their measurement values for 30 days and each time step indicates each day. We here note that we choose 15 days out of 730 days for visualization so that we easily understand the changes of measured values. s 1 and s 2 are placed at the same location and co-evolve simultaneously (i.e., the measured values often increase/decrease at the same time steps). For example, the values of s 1 and s 2 increase/decrease simultaneously at timestamps 1 and 2. On the other hand, s 1 and s 3 that are placed at different locations and co-evolve with 1 day Fig. 1 Examples of simultaneous and time-delayed correlations in China time delay (i.e., measured values of s 1 increase/decrease 1 day after increasing/ decreasing measure values of s 3 ). For example, the value of s 1 increases 1 day after increasing the value of s 3 at timestamp 8. The co-evolution of 1 day delay can be computed by the same way of simultaneous co-evolution when we shift red lines left by one timestamp. It is interesting that s 1 and s 2 do not often co-evolve with 1 day time delay. These results indicate that rainfall affects the volume of NO 2 in the same location at the same time or in close locations with time delays.
The SCP mining cannot discover the above correlations because it assumes sensors with a single attribute and aims to discover patterns whose measurements simultaneously change. The correlated attribute provides more diverse knowledge than the patterns obtained by the SCP mining because the correlated attribute takes multiple attributes and time delays into account.

Contribution
Motivated by the above examples, we introduce a problem of discovering correlations among different attributes, which is called correlated attribute pattern (CAP) mining. A CAP is a set of multiple attributes measured by a set of sensors which are close to each other and whose measurements co-evolve. Although we can naively extend the existing SCP mining methods to discover CAPs, they are inefficient because they extract unnecessary sets of sensors that all the sensors measure a single attribute data. Therefore, we propose a novel CAP mining method, called Miscela, which can efficiently discover CAPs in a set of sensors whose measurements contain multiple attributes. Miscela can discover not only simultaneous correlated patterns but also time-delayed correlated patterns. Time-delayed correlated patterns are effective to discover diverse knowledge. However, it is difficult to find all the correlated patterns with time delays because the search space is significantly large. To discover all time-delayed correlated patterns with any time delays, we need repeatedly execute a significantly large number of Miscela with changing the time delays. For efficiently discovering all time-delayed correlated patterns with any time delay, we extend Miscela to automatically search for time-delayed CAPs to reduce the number of executing Miscela. This method leverages an apriori-based approach that if finds CAPs consisting of a small number of attributes and then combine multiple CAPs to make new candidates consisting of a large number of attributes. This method is efficient because it can prune unnecessary time delay parameters that do not generate CAPs.
We summarize the main contributions of this article in the following: -We introduce a new problem, CAP mining, which aims to discover correlations among different attributes. -We propose Miscela that efficiently discovers CAPs. Miscela accelerates the CAP mining with a novel data structure called the CAP search tree, which conceptually organizes all CAPs based on the spatial constraint and combinations of attributes.

3
-We propose Miscela with automatic time-delay search to efficiently discover all CAPs with any time-delay patterns. -We conduct experiments with three real sensor datasets measured in Santander and China. The experimental results demonstrate that Miscela is up to 20.84 times faster to that of the state-of-the-art SCP mining method [17]. Miscela with automatic time-delay search significantly reduces the computation cost compared with a naive method that repeats executing Miscela. Furthermore, we discovered meaningful CAPs for urban managements and environmental studies.
This article is an extended version of our previous work [5]. New contents are as follows. The previous work discovers only simultaneous correlation patterns. We redefine the CAP mining and extend Miscela to discover time-delayed correlation patterns. Additionally, we develop Miscela with automatic time-delay search. In experimental studies, we show meaningful CAPs both with and without time delays.

Organization
The rest of paper is organized as follows. We formulate the CAP mining problem in Sect. 2 and present a novel CAP mining method Miscela in Sect. 3. In Sect. 4, we present Miscela with automatic time-delay search. We conduct CAP mining experiments to evaluate the performance of Miscela and show meaningful CAPs in Sect. 5. After that, we summarize the past typical works related to our work in Sect. 6, followed by the conclusion in Sect. 7.

Problem description
We first explain an overview of CAP mining and then details of the problem definition that we solve in this paper.

Overview
We define the CAP mining as a problem of discovering spatially and temporally correlated environmental properties (correlated attributes) such that multiple sensors measure on those attributes that satisfy the following conditions: (1) the sensors are deployed at spatially close locations, (2) the measurements of the attributes co-evolve frequently, and (3) the measurements of a certain number of attributes are temporally similar with permitting time-delay. We introduce Definitions 1 and 2 for spatial connectivity, Definitions 3-5 for temporal co-evolution on a single attribute, and Definitions 6 and 7 for co-evolution among multiple attributes, respectively. Finally, we define CAP in Definition 8. We summarize the description of variables in Table 1.

Details of the definitions
Let S = {s 1 , s 2 , … , s n } be a sensor set in a geographical region. Each sensor s i ∈ S (1 ≤ i ≤ n) is deployed at location l i and has attribute a i ∈ A, A = {a 1 , a 2 , … , a m }, where m indicates the number of attributes of the deployed sensors in a city. Each attribute represents the type of data such as temperature, traffic volume, and PM2.5. The sensor s i has synchronized measurements s i [t j ] (1 ≤ j ≤ T) over the time domain T = ⟨t 1 , t 2 , … , t T ⟩, where each t j is a timestamp. Our goal is to discover sets of attributes which are spatially and temporally correlated measured by sets of sensors in S. The correlation among attributes is treated as the correlation among sensors which measure different attributes. The spatial correlation among sensors is evaluated based on the spatial closeness of the sensors. To introduce the concept of the spatial correlation among sensors, we define spatially connected set and spatially connected congeneric set as follows.
Definition 1 (Spatially Connected Set) Given distance threshold and subset of a sensor set G ⊆ S, G is a spatially connected set if for any subset G ′ of G, there are s ∈ G � and s � ∈ G�G � that dist(s, s � ) ≤ , where dist(s, s � ) is the geographical distance between s and s ′ .
Definition 2 (Spatially Connected Congeneric Set) Given spatially connected set G and attribute a, G is a spatially connected congeneric set if all sensors in G have the same attribute a. We denote it by G a .

3
The temporal correlation among sensors is evaluated based on the number of timestamps of sensors whose measurements change similarly. The time domain T typically includes many uninterested intervals in which the measurements have random and small fluctuations. To obtain the meaningful correlations, we only compare timestamps at which measurements of sensors change significantly in T. Thus, we define change rate and evolving timestamp.
Definition 3 (Change Rate) Given a sensor s i and a timestamp t j , change rate r i [t j ] of s i at timestamp t j is defined as follows: Definition 4 (Evolving Timestamp) Given evolving rate and attribute a, Evolving timestamps for a is defined as the timestamps with the top-% (absolute) change rate in the whole sensor data with a. Let a be the top-% (absolute) change rate of a, timestamp t j is called positive and negative evolving timestamp if r i [t j ] ≥ a and r i [t j ] ≤ − a , respectively.
Next, we define co-evolution on a spatially connected congeneric set.

Definition 5 (Co-evolution on Spatially Connected Congeneric Set)
Let G a be a spatially connected congeneric set for attribute a. Given evolving threshold a = ( a + , a − ), timestamp t j positively co-evolves in regard to a if ∀s i ∈ G a , r i [t j ] ≥ + , denoted as t j + � � � � � � � → a . As well, timestamp t j negatively co-evolves as for a if ∀s i ∈ G a , r i [t j ] ≤ − , which is denoted as t j − � � � � � � � → a . The set of timestamps positively or negatively co-evolving is called co-evolution on G a , denote as E(G a ) = {t j ∈ T|t j The purpose of our research is to discover correlations among different attributes. The correlation among different attributes is defined as a co-evolution among a variety of spatially connected congeneric sets. We are not only interested in the simultaneous correlations but also the correlations with time delay (i.e. the time-delayed correlation). Therefor, we firstly define simultaneous co-evolution among attributes, followed by time-delayed co-evolution among attributes.
Definition 6 (Simultaneous Co-evolution among Attributes) Let * i be a symbol which represents either + or − and * i be an inverse of * i . Let G a 1 , … , G a be spatially connected congeneric sets ( a 1 ⋯ a are different each other). Given evolving thresholds a 1 , … , a , if G a 1 ∪ ⋯ ∪ G a is a spatially connected set, we call the set of } simultaneous co-evolution among the attributes on G a 1 , … , G a .
Definition 7 (Time-delayed Co-evolution among Attributes) Let G a 1 , … , G a be spatially connected congeneric sets. Given evolving thresholds a 1 , … , a and set of time-delay offsets = { a 1 , … , a }, if G a 1 ∪ ⋯ ∪ G a is a spatially connected set, we call the set of timestamps C * 1 ,…, * We specify each time-delay offset { a 1 , … , a } to each attribute. In Fig. 1a, for instance, { Rain , NO 2 } = {0, 0} which means that the CAP is a simultaneous co-evolution between the rainfall and the NO 2 . In Fig. 1b, on the other hand, { Rain , NO 2 } = {+1, 0} which means that the CAP is a time-delayed co-evolution between the rainfall and the NO 2 . Note that the definition of the time-delayed coevolution is equivalent to the definition of the simultaneous co-evolution when all time-delay offsets are equal to zero.
Time-delayed co-evolution among attributes includes any patterns without considering their frequency (i.e., patterns with less number of co-evolving timestamps). If a time-delayed co-evolution among attributes appears frequently, we call it correlated attribute pattern.
Based on the above definitions, we define our problem CAP mining as follows.
Problem Definition (CAP Mining) Given sensor set S over time domain T, minimum support , evolving rate , distance threshold , the maximum number of CAP attributes , and set of time delay offsets , the CAP mining discovers all the correlated attribute patterns which contain two to attributes.
In our previous work [5], we defined the CAP mining as discovering the simultaneous co-evolution among attributes. In this article, we redefine the CAP mining as discovering the time-delayed co-evolution among attributes, which is a more generalized problem than the problem in the previous work.
We here summarize parameters and their impacts to the number of CAPs to be discovered.
-Evolving rate : is for deciding evolving timestamps of each sensor. If is large, many time stamps are evaluated as evolving timestamps and then the number of CAPs likely becomes large. Otherwise, the number of CAPs likely becomes small. -Distance threshold : is for deciding connectivity of sensors. If is large, many sensors are spatially connected and then the number of CAPs likely becomes large. Otherwise, the number of CAPs likely becomes small.

3
-The maximum number of CAP attributes : restricts the number of attributes in CAPs. Thus, if is small, the number of CAPs likely becomes small. Otherwise, the number of CAPs likely becomes large. -The minimum support : is the minimum support for restricting CAPs which are not frequently co-evolving. If is small, many co-evolution among attributes becomes CAPs since the number of co-evolving timestamps is larger than . Thus, the number of CAPs likely becomes large. Otherwise, the number of CAPs likely becomes small.

Example
We show an example of CAP mining by using Fig. 1. We set five parameters , , , , and . We set 70% as to reduce effects of small changes of measured values. Small changes such as timestamps 6 and 9 of s 1 and timestamps 10 and 11 of s 2 and s 3 are not included in evolving timestamp (Definition 4). We set 200 km as because each sensor is distant from each other. s 1 , s 2 , and s 3 are within 200 km. Thus, s 1 , s 2 , and s 3 constitute spatially connected set (Definition 1). s 2 and s 3 constitute spatially connected congeneric set G NO 2 because s 2 and s 3 have the same attribute NO 2 based on Definition 2. Co-evolution on spatially connected congeneric set E(G NO 2 ) on s 2 and s 3 includes the set of timestamps, for example timestamps from 1 to 7 (Definition 5). We set two as and { Rain , NO 2 } = {+1, 0} as to find CAPs such that rainfall increases/decreases 1 day after NO 2 increases/decrease (Definition 7). Since s 1 and s 3 are spatially connected, have different attributes, and often increase and decrease together, the set of attributes {rainfall, NO 2 } on the set of sensors {s 1 , s 3 } is found as the CAP (Definition 8). Of course, if we set small as the minimum support, the set of attributes {rainfall, NO 2 } on the set of sensors {s 1 , s 2 } is also found as the CAP. We can control the number of found CAPs by configuring the parameters , , , and .

MISCELA
In this section, we present our CAP mining method Miscela. Firstly, we describe an outline of Miscela. Then we explain the detail of each component of Miscela.
After that, we show the algorithm of Miscela, followed by a discussion of the time complexity.

Outline of Miscela
According to Definition 8, the attributes on a sensor sets are a CAP if (1) the sensors are spatially connected, (2) the sensors contain two to attributes, and (3) the cardinality of the co-evolution is larger than . We can naively discover all CAPs by searching all the spatially connected sets within a given sensor set, evaluating the number of attributes, and evaluating the cardinality of co-evolution. However, separately conducting the procedures is inefficient because the spatially connected sets often have uninteresting attribute patterns which do not satisfy the CAP conditions; do not contain two to attributes (i.e., contains only single or more than attributes) and/or the attributes do not coevolve. Hence, we structurally search only the spatially connected sets whose attribute patterns satisfy the CAP conditions. For this purpose, we take an expansion-based search, which gradually expands a spatially connected set so that the expanded sensor set would also be spatially connected and contain two to attributes. In addition, we propose a tree structure, called CAP search tree, for effectively expanding the spatially connected set and stopping the expansion. We can avoid evaluating unnecessary attribute patterns which definitely do not satisfy CAP definition by using the CAP search tree.
Miscela comprises the following four steps.
1. Linear segmentation we filter uninteresting data fluctuation by applying a linear segmentation algorithm to time series data. 2. Extracting evolving timestamps we extract evolving timestamps in the measurements of all sensors by using given evolving rate . 3. Discovering spatially connected sets of sensors since CAPs are discovered only from spatially connected sets, we divide a given sensor set into spatially connected sets to restrict the search space. 4. CAP search for each spatially connected set, we search for CAPs. We recursively conduct the CAP search with gradually expanding a spatially connected set according to the CAP search tree.

Linear segmentation
As for the first step, we approximate the time series data to filter uninteresting fluctuations because the time series data often includes noises. To approximate the time series data, we employ a simple and effective linear segmentation algorithm, the bottom-up algorithm in [7]. The bottom-up algorithm first merges successive two measurements to approximate the T-length time series data by Then, we iteratively merge successive two measurements that have the smallest difference between them among any successive measurements. If the smallest difference is larger than a given threshold, we stop the procedure. The linear segmentation reduces unexpected effects on CAPs caused by small fluctuations. Note that any algorithms can be used in this step instead of the bottom-up algorithm.

Extracting evolving timestamps
To discover CAPs, we need to extract evolving timestamps. We permit users to specify the evolving rate instead of directly specifying the evolving threshold in Definition 4. The detail of the extraction is as follows. First, we calculate the change rate for each time series data of each attribute. Next, we compute the change rate with the top-% absolute value as an evolving threshold by which evolving timestamps are extracted. Then, we extract both positive and negative evolving timestamps whose absolute change rate is larger than the evolving thresholds.

Discovering spatially connected sensors
As a CAP is discovered on a spatially connected set, we divide a given sensor set into spatially connected sets so that we restrict the search space only inside of each maximal spatially connected set. Here, maximal means that the spatially connected set is not contained in any larger connected set. In Miscela, we model the spatially connected sensor sets as graphs. Then, we firstly introduce concepts of sensor graph, connected sub-graph, and connected component.
Definition 9 (Sensor Graph) Given sensor set S and distance threshold , sensor graph G S is a graph where each vertex in G S corresponds to a sensor in S and there is an edge between two vertices if their corresponding sensors are located within .
To discover spatially connected sensors, we construct a sensor graph and find connected components of the sensor graph. For efficiently computing them, we use Dbscan [4]. Dbscan is one of the clustering algorithms, which groups points surrounded with many nearby neighbors. It has two input parameters; distance threshold and MinPts. When we set as distance threshold and 2 as MinPts, Dbscan can identify edges between s and s ′ if the distance between two sensors is less than or equal to . The clustering results of Dbscan can be considered as connected components. Dbscan simultaneously constructs sensor graphs and find connected components.
Example Figure 3 shows an example of sensor graph. Let the square and the circle symbols be sensors and the different symbols mean to measure different attributes. Given sensor set S = {s 1 , s 2 , … , s 9 } and distance threshold , we transform it as the sensor graph G S . There is no edge between s 5 and s 6 because the distance between them is larger than . We identify two connected components G 1 and G 2 in G S .

CAP search
We can naively discover all CAPs by searching all connected sub-graphs of each connected component. However, it is quite inefficient because the number of connected sub-graphs exponentially increases as the number of sensors increases. Therefore, we gradually expand a connected sub-graph based on the connectivity. Additionally, it is unnecessary to compute the intersection of evolving timestamps on connected sub-graph which contain only a single attribute or more than attributes for the CAP mining because the connected sub-graphs do not have any CAPs. Thus, we expand a connected sub-graph so as to make the explored connected subgraphs can have CAPs.
For the efficient expansion, we develop a tree structure called the CAP search tree. We construct a CAP search tree for every connected component. For each connected component, the CAP search tree effectively organizes all the connected sub-graphs which contain at most attributes into a tree structure based on the spatial connectivity. Each tree node in the CAP search tree uniquely corresponds to a connected sub-graph of a connected component. In the CAP search tree, we compute the intersection of evolving timestamps only for connected sub-graphs corresponded by tree nodes. If the number of intersections in tree nodes is larger than , the tree nodes contains CAPs. We call the computation of the intersection CAP computation. The CAP search tree effectively reduces the computation cost because it reduces the number of CAP computations.
To construct the CAP search tree based on the spatial connectivity, we introduce parent relation between two connected sub-graphs.
Definition 12 (Parent) Given size-connected sub-graph Y in sensor graph G S , vertex ordering of G S , and the maximum number of CAP attributes . Let s be the first possible vertex in satisfying the condition such that X = Y�{s} is a connected subgraph containing less than or equal to attributes. At this time, X is called a parent of Y.
We use the parent relation to construct a CAP search tree of each connected component. Each tree node in the CAP search tree has one unique parent, and the nodes containing a single sensor are connected the root node . In short, all the connected sub-graphs in the connected component form a tree structure with the empty set as the root. To discover all CAPs, we explore all the tree node from the root to the leaves. The construction of the entire tree structure takes large costs. Thus, we do not construct the entire tree structure beforehand for the efficient discovery of CAPs. Instead, we perform depth-first construction from the root node and only visit the tree nodes that have CAPs.
Here, vertex ordering is used for deciding parents which affect the order of searching CAPs. Our algorithm can handle any order of vertices and the vertex order does not affect the time complexity. Fig. 4 The CAP search tree of G 1 in Fig. 3 Example Consider the connected component G 1 in Fig. 3. Suppose a vertex ordering = {s 1 → s 2 → s 3 → s 4 → s 5 } and the number of maximum attributes = 2. Figure 4 shows the CAP search tree for the connected component G 1 , Each tree node in the CAP search tree corresponds to a connected sub-graph in G 1 . Any connected sub-graphs in the tree contain two to attributes (except for a single sensor). The parent of the connected sub-graph {s 2 , s 3 , s 4 } is the set {s 2 , s 3 } because sensor s 4 is the first possible one in that ensures the remaining sensors are still connected and it contains 2 attributes. We can discover all CAPs in G 1 by exploring all the tree nodes in the CAP search tree.
For any tree node in the CAP search tree, if a node does not have any CAPs, no descendants of the node have CAPs. Hence, we can safely prune the sub-tree rooted at the node by the following theorem.

Theorem 1 Given spatially connected set
where G a 1 , … , G a are spatially connected congeneric sets. Let G ′ be a connected sub-graph of G, G has no CAPs if G ′ has no CAPs.
Proof Let C * 1 ,…, * a 1 ,…,a be a co-evolution among G a 1 , … , G a . All the timestamps in C * 1 ,…, * a 1 ,…,a is also contained in C Thus, G has no CAPs if G ′ has no CAPs. ◻

Algorithm of Miscela
We can efficiently discover CAPs by using four steps in Miscela. The pseudo-code of our proposal, Miscela, is given in Algorithms 1, 2, and 3. Algorithm 1 contains first, second, and third steps. Miscela applies the bottom-up segmentation algorithm to the given sensor data in order to filter uninteresting fluctuations (lines 1-3). Next, it computes the evolving thresholds for all the attributes (line 5) and extracts evolving timestamps (lines 6-15). Then, it identifies connected components by using Dbscan (line 17). Finally, it conducts the CAP search starting from every size-1 connected sub-graph (lines 18-20). Algorithm 2 sketches the fourth step, that is the CAP search. Given connected component X, the algorithm starts the depth-first search from X. First, we select all connected sub-graphs whose parent is X in the CAP search tree (line 1). Second, for each connected sub-graph Y, the algorithm conducts the CAP computations (i.e. the intersection of the evolving timestamps) on Y (line 3). If Y contains CAPs, the algorithm outputs the CAPs on Y (line 5). Then, we recursively conduct depth-first search on Y (line 6). If Y does not contain any CAPs, we prune all the subtree rooted at Y. Algorithm 3 describes the computation process of CAPs. First, the algorithm calculates the time delay by adding the time delay offset to all the positive and negative evolving timestamps of all the sensors in Y, where the added time delay offset is selected for each attribute (lines 1-3). Then, a list of CAPs is initialized as an empty set (line 4). After that, the algorithm computes CAP with for each change pattern among different attributes (lines 5-9). Note that since there are some equivalent combinations of * 1 , … , * ̃ (e.g. { * 1 = +, * 2 = −} is equivalent to { * 1 = −, * 2 = +} ), we skip the computations of either combination. Finally, the CAP computation returns all CAPs on Y as the output (line 10).

Time complexity of Miscela
We analyze the time complexity of Miscela.

Theorem 2 Given the number of sensors n, the length of time domain T, the number of connected components | S |, the maximum number of CAP attributes
, the maximum size of sensor sets in tree nodes , the height of CAP search tree h, and the average degree of sensor graph d, Miscela incurs time complexity of O(nT + n log n + | S |hd2 + | S |hd2 ).
Proof Since Miscela contains four steps, we describe the time complexity of each step. Then, We describe the total time complexity of Miscela.

3
Finally, the total time complexity of Miscela has been calculated by adding all the steps, as O(nT + n log n + | S |hd2 + | S |hd2 ). ◻ We here note that a set of time-delay offsets affects the height of CAP search tree h because it affects the number of CAPs.

Automatic time-delay search
Time-delayed CAP are helpful to analyze and understand phenomena that affect environments of places close to each other at different time. Although we can find all time-delayed CAPs by setting arbitrary time delays, it is difficult to thoroughly find time-delayed CAPs because users have to repeatedly execute Miscela with changing the sets of time-delay offsets. Its computational cost is large, so it is better to automatically search for sets of time-delay offsets so that users can find timedelayed CAPs with any sets of time-delayed offsets. We here define our problem of CAP mining with automatic time-delay search.

Problem Definition (CAP Mining with Automatic Time-delay Search) Given sensor set S over time domain T, minimum support , evolving rate , distance threshold
, the maximum number of CAP attributes , and the maximum time-delay offset M , the CAP mining with automatic time-delay search discovers all the correlated attribute patterns which contain two to attributes with any patterns of the set of time-delay offsets whose values are between − M and M .
Given the maximum time-delay offset M , the number of patterns of the sets of time-delayed offsets is (2 M + 1) m (recall that m is the number of attributes), so a naive approach takes a significantly large time. That is, it takes (2 M + 1) m multiplied by the average execution time of Miscela. Therefore, we need an efficient search method for the sets of time-delay offsets that generate time-delayed CAPs.
We develop Miscela with automatic time-delay search based on apriori manner [1]. Our method executes Miscela for all patterns of two attributes and sets of time-delayed offsets, and then combines these attributes and the sets of timedelayed offsets if these parameters generate the time-delayed CAPs. The apriori manner can prune unnecessary parameters of attributes and time-delay offsets that do not generate the time-delayed CAPs. Furthermore, we initially reduce the number of sets of time-delayed offsets by pruning the sets that generate the same timedelay CAPs. For example, the sets of time delay offsets {0, 0} and {−1, −1}, and {1, 0} and {0, −1} generate the same results, respectively, because they extract the same co-evolving timestamps. We prune set of time-delayed offsets  1) ) times). It is the same computation cost with the naive method. The computation cost is significantly large theoretically, but empirically Miscela with automatic time-delay search efficiently computes CAPs by pruning a large number of candidates.

Experiments
In this section, we evaluate the efficiency of Miscela and the usefulness of the CAP mining. To the best of our knowledge, no existing methods can directly apply the CAP mining when a given sensor set contains multiple attributes. Hence, we compare Miscela with an SCP mining method Assembler [17] which is the state-of-the-art algorithm for the SCP mining. Assembler discovers SCPs which contains all necessary sensor sets for the CAP mining but also contains unnecessary ones. Then, we filter the SCPs to remove unnecessary sensor sets during the search process. Since the difference between Miscela and Assembler is their CAP search, we compare the response time of the CAP search of Miscela with that of Assembler by changing the following parameters: -The maximum number of CAP attributes , -The evolving rate , -The minimum support , -The set of time-delay offsets .
We additionally evaluate the efficiency of automatic searches.
The algorithms of Miscela and Assembler are implemented in Python. The experiments are conducted on a computer with Intel Xenon E7-8860v4 2.20GHz CPU.

Experimental setup
Our experiments use three real sensor datasets; (1) 5 attributes daily sensor data collected in Santander, Spain from March 1, 2016 to September 30, 2016, (2) 6 attributes daily sensor data collected in China from September 1, 2016 to August 31, 2018, and (3) 13 attributes daily sensor data collected in China from September 1, 2016 to August 31, 2018. We obtained the Santander dataset from FESTIVAL 1 . and the two China datasets from emphenvicloud.cn. 2 . Table 2 shows the number of sensors, the number of timestamps, and the type of attributes in three datasets. We set distance threshold as 80 m for the Santander dataset and as 200 km for the China datasets, respectively. We chose these distance thresholds so as to divide sensors in each dataset into around 20 connected components. For all the datasets, we use = 50 %, = 2, = 500, and = {0, … , 0} (i.e. no time delay) as default parameters. Vertex ordering is calculated based on sensor identifiers which are assigned in the datasets (i.e., almost random order).

Efficiency of Miscela
In this section, we evaluate the efficiency of Miscela. We show the response time of Miscela and Assembler by changing three parameters , , and . In addition, we show the response time of Miscela by changing . In summery, Miscela is always faster than Assembler throughout the experiments. Miscela is up to 20.84 times faster than Assembler. The results of the experiments for the time-delayed setting show that the response time of Miscela does not depend on but depends on the number of found CAPs.

The maximum number of CAP attributes
We describe the experimental results by changing the maximum number of CAP attributes . We vary from 2 to the number of attributes in each dataset (e.g., from 2 to 5 for the Santander dataset). Figure 5 shows the result of response time in the three datasets. We observe that Miscela is always faster than Assembler in all the datasets for any . Moreover, the result of China 13 dataset indicates that Miscela is more efficient with smaller . This is because Miscela more frequently skips processing the connected sub-graphs which have no CAPs when we set smaller . From the viewpoint of the time complexity of the CAP search (see Sect. 3.7), the computation cost of the CAP search increases as increases. Table 3 shows the number of CAP computations for Santander dataset with varying . We confirm that the time complexity of CAP search depends on .

Evolving rate "
We describe the experimental result by changing the evolving rate . We measure the response time of both methods with = 30%, 40%, 50%, 60%, and 70%. Figure 6 shows that Miscelais is faster than Assembler in all the datasets for any . The response time increases as the evolving rate increases. The setting of the large makes the number of extracted evolving timestamps to be large. As a result of this, the height of CAP search tree h becomes large, which makes the computation cost of the CAP search large, because more CAPs are discovered with a larger number of evolving timestamps. Table 4 shows the number of CAP computations for Santander dataset with varying . We observe the number of CAP computations increases as increases, and this result confirms the correctness of the time complexity of the CAP search given by Theorem 2.

Minimum support Ã
We describe the experimental result by changing the minimum support . We measure the response time of both methods with = 400, 450, 500, 550, and 600. Figure 7 shows the result of the response time. It shows that Miscela is faster than Assembler in all the cases. In particular, the response time of Miscela is 20.84 times smaller compared with that of Assembler in China 6 dataset with = 400. We can see in the result that the response time of CAP search increases with decreasing the minimum support. This is because the CAP search with smaller minimum support discovers more CAPs, which makes the computation cost of the CAP search larger because CAP search trees become high. indicates that the smaller minimum support causes the smaller computation cost of the CAP search.

Time delay offset
We describe the experimental result by changing the set of time delay offsets . We measure the response time of Miscela with a = −1, 0, and +1 for each attribute. More concretely, we set −3 to +3 to a single attribute and set zero to the other attributes. Figure 8 shows the response time of Miscela depends on the number of founded CAPs, so it does not change unless the number of found CAPs becomes large. We can see in the result of Santander that the response time of Miscela is very small when Temp. ≠ 0 compared with Temp. = 0. As shown in Table 6, this is because the number of found CAPs is 1423 and 1399 when we set Temp. = −1 and +1, respectively, while the number of found CAPs is 27,971 when Temp. = 0. In the other experiments, we observe that the response times of Miscela are almost same when we set any time delay offset because the numbers of found CAPs are also almost same. We confirm from these result that the response time of Miscela does not change much for the time delay unless the number of found CAPs significantly changes.

Efficiency of Miscela with automatic time-delay search
We evaluate the efficiency of Miscela extended with automatic time-delay search. Figure 9 shows response time of Miscela with automatic time-delay search and the naive approach, varying with the maximum time-delay offsets. In the response time of the naive approach, we show the estimated response time since it did not finish over 24 h. The estimated response time is computed as the average response time of Miscela times (2 M + 1) − (2 M ) . From this result, we can see that our algorithm is much more efficient than the naive approach. In particular, as increases, the response time of our algorithm does not increase exponentially. This is because our algorithm can effectively prune sets of time-delay offsets that do not generate time-delayed CAPs. Figure 10 shows the number of executions of Miscela varying with the maximum time-delay offset M . We can see that the automatic time-delay search drastically reduces the number of executions of Miscela.
In summery, Miscela with automatic time-delay search enables us to efficiently discover all time-delay CAPs with any time delays.

Examples of meaningful CAPs
We here show meaningful CAPs that are found in our experiments. We describe three examples; (1) simultaneous CAPs in Santander, (2) simultaneous and

Simultaneous CAPs in Santander
We first show a CAP between temperature and light in Santander. Figure 11 shows the locations of the sensors and the measurements of the sensors. The CAP is represented as C + 1 ,+ 2 a 1 ,a 2 , where a 1 = temperature and a 2 = light, G a 1 = {s 1 , s 2 , s 3 , s 4 , s 5 , s 6 } and G a 2 = {s 7 , s 8 , s 9 , s 10 , s 11 }. The light and temperature sensors are spatially close and their measurements increase/decrease simultaneously, which means there is no time delay between the changes of the values of temperature and light on these sensors. These sensors are located in a downtown area, and this pattern indicates that the downtown receives a lot of sunshine during the day time. Conversely, if some sensors that are not found in CAPs, this indicates that these sensors are located in the shade area.

Simultaneous and time-delayed CAPs in China
Next, we show a CAP between rainfall and NO 2 in China 13 dataset, which is described in Sect. 1 (see Fig. 1).
The CAP for s 1 and s 2 is a simultaneous CAP and a negative correlation pattern [i.e., if the measurement value of s 1 increases (resp. decreases), the measurement value of s 2 decreases (resp. increases)]. This indicates that the volume of NO 2 decreases/increases while the amount of rainfall increases/decreases. This is reasonable because meteorological particles are affected by environmental conditions (e.g., rain and wind) at the same location.
On the other hand, the CAP for s 1 and s 3 is a time-delayed CAP and a negative correlation pattern. This indicates that the volume of NO 2 decreases/increases a day later the amount of rainfall increases/decreases. Besides, we did not find any CAPs between the NO 2 and rainfall sensors located at the same location with 1 day time delay. These indicate that the rain affects the reduction of NO 2 at the location 80 km east 1 day later. This is an interesting result because our Fig. 11 The CAP in Santander algorithm found the phenomena based on data science without using physical simulations or other meteorological techniques and knowledge.

The difference between amounts of simultaneous and time-delayed CAPs in China
We finally show the amounts of simultaneous and time-delayed CAPs that PM2.5 correlates with the other attributes in China 13 dataset. Table 7 shows the number of CAPs in cases of PM2.5 = 0 and PM2.5 = +1 and a = 0 for the other attributes a. We observe that the PM2.5 are often correlated with the other attributes when PM2.5 = 0 (e.g. there are 999 CAPs among PM2.5 and CO) but did not correlate with the other meteorological sensors. Since meteorological data does not co-evolve with PM2.5, the volume of PM2.5 does not highly depend on meteorology such as weather. We here have to note that no or rare CAPs are also meaningful for environmental studies. In case of PM2.5 = +1, the number of CAPs becomes much small. This indicates that the volume of particles typically increase/decrease simultaneously. Specially, there is no CAPs among PM2.5 and O 3 on PM2.5 = +1 even though there are many CAPs on PM2.5 = 0. Some attributes related to air quality (e.g. SO 2 , NO 2 , and CO) correlate with PM2.5 when PM2.5 = +1, that is, the amounts of particles increases/decreases 1 day later after the value of the PM2.5 increases/decreases like the CAP in Sect. 5.4.2.

Related work
The CAP mining is one of the pattern mining tasks which aim to extract similar and frequent patterns in the time series data. We review two similar tasks; motif discovery and co-evolving mining task. Motif discovery in time series data extracts a pair of subsequences whose distance is smaller than a given threshold . The subsequence is called motif. Lonardi and Patel [8] first introduced the top-K motif discovery task that discovers the K subsequences that have the largest numbers of matches among time series data. Chiu et al. [3] developed an algorithm that discovers approximate motifs in linear time. Mueen et al. [11] proposed an algorithm that efficiently discovers exact motifs with the linear ordering heuristic and the early abandoning strategy. Motif discovery in multi-dimensional time series has also been studied. Tanaka et al. [14,15] proposed  an algorithm that transforms multi-dimensional time series data into a sequence of symbols using principal component analysis. Minnen et al. [10] studied the problem of mining sub-dimensional motifs that across only a subset of the dimension. These techniques are not suitable for the CAP mining because they do not consider the locations of sensors and the various patterns of attributes. Co-evolving mining task aims at discovering sets of sensors whose measurements co-evolve frequently. Trasarti et al. [16] studied the problem of discovering regions which show a similar deviation of population density by using mobile phone data. Their method extracts vertical changes by calculating the same hour of different days. In contrast, CAP search extracts horizontal frequent changes in different sensors. Matsubara et al. [9] proposed a spatially co-evolving framework Funnel to discover both the county-level and the state-level properties of different diseases. However, it is designed specifically for epidemic data instead of general urban sensor data. Zhang et al. [17] proposed a problem called the SCP mining, which aims to discover sensors which are spatially close each other and frequently co-evolving in their measurements. They proposed an efficient algorithm called Assembler which is the state-of-the-art of the SCP mining. Although Assembler can discover CAPs, there is large redundancy in their processing because they extract unnecessary correlated sensors that do not have CAPs. Here, to define the correlation among time series sensor data, they adopted the co-evolution instead of the standard Pearson correlation. The Pearson correlation is only for two variables while the co-evolution can be used for multiple variables. Since the SCP mining aims to find the correlated patterns among multiple sensors, the co-evolution is more suitable compared to the Pearson correlation. The CAP mining also targets multiple variables, we also adopt the co-evolution to define the correlation. Cheng et al. [2] studied discovering dynamic co-evolving zones in time series data. They proposed the divide-andconquer strategy to discover the relationship between the co-evolving zones of the different time period. Hassani et al. [6] proposed a method for constructing physical clusters of sensor nodes based on both spatial and measurement similarities to make groups which record similar measurement over a time period. These algorithms do not target the CAP mining. We show that Miscela is more efficient than Assembler which is the state-of-the-art for the SCP mining.

Conclusion
In this paper, we introduced a problem called correlated attribute pattern (CAP) mining, which discovers the correlated patterns among attributes in multi-attributes sensor set. We are motivated to discover not only the simultaneous correlated patterns among attributes but also the correlated patterns with time delay. Hence, we defined the CAP mining as discovering time-delayed correlated patterns among attributes. We proposed a efficient method, Miscela, for the CAP mining. Miscela effectively prunes the unnecessary computations for the CAP mining. Furthermore, we developed Miscela with automatic time-delay search to efficiently find all CAPs with any time delay. We conducted experiments using three real sensor datasets. The experiments proved that Miscela can more efficiently discover CAPs in multi-attribute sensor sets compared with the state-of-the-art the SCP mining method. The results showed that the CAP mining obtains several meaningful patterns.
There are several major directions for further investigation. First, CAPs highly depend on parameters, so users may repeat Miscela several times to discover meaningful CAPs. Thus, it is interesting to re-use prior results or compute reusable intermediate results. Second, CAPs can extract regional characteristics in certain areas. For example, two attributes do not co-evolve in an area though the two attributes often co-evolve in other areas. This indicates CAPs can be useful to discover outliers of areas. Third, we discover CAPs only for spatially close sensors, but attributes on distinct sensors may often co-evolve. Thus, we would like to extend the CAP mining for discovering co-evolution among attributes on distinct sensors.