PARASOL: a hybrid approximation approach for scalable frequent itemset mining in streaming data

Here, we present a novel algorithm for frequent itemset mining in streaming data (FIM-SD). For the past decade, various FIM-SD methods in one-pass approximation settings that allow to approximate the support of each itemset have been proposed. They can be categorized into two approximation types: parameter-constrained (PC) mining and resource-constrained (RC) mining. PC methods control the maximum error that can be included in the approximate support based on a pre-defined parameter. In contrast, RC methods limit the maximum memory consumption based on resource constraints. However, the existing PC methods can exponentially increase the memory consumption, while the existing RC methods can rapidly increase the maximum error. In this study, we address this problem by introducing a hybrid approach of PC-RC approximations, called PARASOL. For any streaming data, PARASOL ensures to provide a condensed representation, called a Δ-covered set, which is regarded as an extension of the closedness compression; when Δ = 0, the solution corresponds to the ordinary closed itemsets. PARASOL searches for such approximate closed itemsets that can restore the frequent itemsets and their supports while the maximum error is bounded by an integer, Δ. Then, we empirically demonstrate that the proposed algorithm significantly outperforms the state-of-the-art PC and RC methods for FIM-SD.


Introduction
Streaming data analysis is a central issue in many domains such as computer system monitoring (Du and Li 2016), online text analysis (Iwata et al. 2010;Hoffman et al. 2010), financial and economic analyses (Zhao et al. 2013;Zhu and Shasha 2002), and medical and health record data analysis (Ginsberg et al. 2009;Keogh et al. 2001). Streaming data is an infinite and continuous sequence of data, and, nowadays, is generated and collected rapidly. The sudden emergence of an intensive bursty event, called concept drift, in streaming data can make it difficult to extract meaningful information from such data. Therefore, there is a strong and growing need for powerful methods that are robust against concept drift in large-scale streaming data analysis.
Frequent itemset mining in streaming data (FIM-SD) (Han et al. 2007; Mala and Dhanaseelan 2011;Lee et al. 2014) is the most fundamental and well-used task in streaming data analysis. It is used to find frequently occurring itemsets (in this study, sets of nonnegative integers) in streaming data (in this study, a sequence of itemsets). FIM-SD must exhibit two important properties; (i) the real time property, which is the ability to process a huge volume of itemsets continuously arriving at high speed and simultaneously (onthe-fly) output the detected frequent itemsets (FIs); and (ii) memory efficiency, which is the ability to enumerate FIs while managing an exponential number of candidate FIs with limited memory. Concept drift suddenly introduces huge itemsets that must be processed, which demands a huge amount of memory. This makes it difficult to design scalable and efficient FIM-SD. Thus, the development of a space-efficient FIM-SD that is especially robust against concept drift is an important open challenge that must be solved to enable large-scale streaming data analysis.
One reasonable approach for memory efficiency is to compress the FIs. There are two major classes of solutions: closed FIs, which have no frequent supersets with the same support (i.e., occurrence frequency), and maximal FIs, which are not contained in any other FI. The maximal FIs can restore the original FIs, though their supports are lost. In contrast, the closed FIs can exactly restore every FI and its support, though memory efficiency is bounded by limitation of lossless compression. For an intermediate solution between the closedness and maximality, various lossy condensed representations have been proposed in Xin et al. (2005), Cheng et al. (2006Cheng et al. ( , 2008, Shin et al. (2014) and Quadrana et al. (2015).
A common approach among them is using an error parameter (0 ≤ < 1) to control the maximum error that can be included in the approximate supports. In the following, we call this approach parameter-constrained (PC) approximation. The value of is fixed in advance, however it is difficult to keep the initial value when concept drift emerges beyond user's expectation. Let us explain this situation using MOA-IncMine (Quadrana et al. 2015), which is the state-of-the-art PC method for FIM-SD. This method incrementally manages approximate closed FIs, called semi-frequent closed itemsets (SFCIs), for each segment of streaming transactions. Figure 1 shows the number of SFCIs stored at each segment of 1,000 transactions generated from a part of the Yahoo! Hadoop grid logs dataset. 1 Then, we observe that the SFCIs rapidly increase on the way to process an intermediate segment. When is less than or equal to 0.1, the process terminates due to out-of-memory exception (with 2GB JavaVM Heap size). Such sudden and intensive increase of memory consumption is a common drawback of PC mining in streaming data.
There is an alternative approximation approach, called resource-constrained (RC) approximation, working under resource (ex. memory and response time) constraint. The Skip LC-SS (Yamamoto et al. 2014) is a representative RC method which works in a constant O(k) space and returns approximate FIs, where k is a size constant associated with the available memory resource. By setting a relevant k, the Skip LC-SS can avoid the outof-memory; this represents a significant advantage. However, it involves another drawback: due to the combinatorial explosion of candidate FIs, the maximum error becomes extremely high in accordance with k, rendering some solutions in the output useless (Yamamoto et al. 2014).
In this study, we tackle the limitation of those PC and RC mining. First, to address the maximum error issue in RC mining, we introduce a lossy condensed representation of the FIs, called -covered set. The notion of -cover is regarded as an extension of closedness compression and allows the original FIs to be compressed while bounding the maximum error by an integer, . Secondly, to address the memory consumption issue in PC mining, we introduce a hybrid approach of PC and RC approximation. As shown in Fig. 1, sudden and intensive memory consumption can be considered a temporary phenomenon. Indeed, normal PC approximation is sufficient except for within a brief period. From the viewpoint of memory efficiency, it is reasonable to dynamically switch between PC and RC approximations. We call this hybrid approach PARASOL (an acronym for Parameter-and Resource-constrained Approximation for Soft and Lazy mining).
In this paper, we propose the first PARASOL method to find a -covered set of the FIs. The proposed method is based on two key techniques: incremental intersection, which is an incremental way to compute the closed itemsets (Borgelt et al. 2011;Yen et al. 2011), and minimum entry deletion, which is a space-saving technique for PC and RC approximation (Manku and Motwani 2002;Metwally et al. 2005;Yamamoto et al. 2014). We firstly show that the output obtained from these existing techniques composes a -covered set of the FIs. Based on this finding, the proposed method controls the memory consumption and maximum error using an error parameter and takes an RC approximation using a size constant k only in certain scenarios. Consequently, the following characteristics are exhibited: On-the-fly manner: it can process any transaction that has L items in O(k) space in almost O(kL) time. Anytime feature: it can monitor the maximum error, , and return the output, T , at anytime. Quality of output: foreveryFIα, T contains some superset β of α such that the difference between the frequencies of α and β is at most .
We also propose a post-processing technique, called -compression, which enables to reduce the size of the raw output for more concise solution. Then, we develop a novel data structure, called weeping tree, to efficiently carry out the three key operations (i.e., incremental intersection, minimal entry deletion, and -compression). Finally, we empirically show that the proposed method outperforms the existing PC and RC methods when applied to various real datasets.
The remainder of the paper is organized as follows. Section 2 presents a brief review of the related works. Section 3 is a preliminary. Section 4 describes the baseline algorithm used to compute a -covered set, and Section 5 describes PARASOL and -compression. The data structure is discussed in Section 6. The experimental results are summarized in Section 7. Then, we conclude in Section 8.

Related works
Here, we clarify our contribution through the comparison with the related works. There are many previous works (Xin et al. 2005;Cheng et al. 2006Cheng et al. , 2008Song et al. 2007;Boley et al. 2009Boley et al. , 2010Liu et al. 2012;Quadrana et al. 2015;Hu and Imielinski 2017) on PC mining that deal with lossy condensed representations of the FIs. However, most of those previous works, expect for Song et al. (2007), Cheng et al. (2008), and Quadrana et al. (2015), are oriented to a transactional database that tolerates multiple scanning and allows us to assume a stable distribution of occurrences. It is infeasible to apply them for SD where concept drift can emerge. Song et al. (2007) proposed an online PC approximation algorithm called CLAIM to compute relaxed closed itemsets. Like the notion of a -covered set, their solution (i.e., relaxed closed itemsets) can bound the maximum error in the approximate support of any FI. In this sense, CLAIM has the anytime feature. However, its update process involves exhaustive computation including set enumeration and generate-and-test based search. In contrast, the proposed method always performs the update processing in O(kL) time for any transaction of length L. Cheng et al. (2008) also proposed an online PC approximation algorithm called IncMine to compute semi-frequent closed itemsets (SFCIs). IncMine incrementally maintains SCFIs for each segment (i.e., each set of transactions) in two steps: computing the SFCIs in the current segment followed by updating whole set of SFCIs based on the newly computed ones. IncMine utilizes various pruning techniques and efficient data structure to avoid redundant computation in the update process. To our knowledge, IncMine is one of the state-of-the-art PC methods for condensed representation mining. Indeed, IncMine has been implemented in a massive online analysis (MOA) platform which enables it to be widely distributed (Quadrana et al. 2015). However, as shown in Fig. 1, a large number of SFCIs emerges temporarily, causing sudden and intensive memory consumption. In contrast, the proposed method based on PARASOL never issues the out-of-memory. Besides, it is unknown if the obtained SFCIs can restore the support of each FI within a bounded error.
StreamMining (Jin and Agrawal 2005) is a PC approximation method that seeks the FIs. Based on an error parameter, , it deletes itemsets that are not promising as they have supports less than or equal to × i for each timestamp i. The accuracy of the solution (i.e., the maximum error in the approximate supports) can be directly controlled by . However, this involves a combinatorial explosion of candidates itemsets to be managed, causing huge memory consumption. In contrast, the proposed method focuses the solution on acovered set of the FIs. Then, the memory consumption becomes much smaller than the one of StreamMining.
CloStream (Yen et al. 2011) is a non-constrained (NC) online method that computes the closed FIs exactly. The update process is based on the incremental intersection that never requires set enumeration computation. However, the memory consumption increases as the whole stream becomes large. This is due to the limitation of closedness compression. In contrast, the proposed method can control the memory consumption using the space-saving technique based on minimal-entry deletion (Manku and Motwani 2002;Metwally et al. 2005;Yamamoto et al. 2014). On the other hand, the proposed method is incomplete for finding the closed itemsets. This is a drawback of the proposed method. Note that there exists the notion of strong closedness, called -closed set (Boley et al. 2009). The -closed set is defined as a characteristic subset of the closed FIs, capturing the interesting patterns. Then, the proposed method is complete for finding this -closed set (i.e., any -covered set contains the -closed set), as detailed in the Appendices A and B.
Skip LC-SS (Yamamoto et al. 2014) is an RC method that seeks FIs based on minimalentry deletion. Given a size constant k, it can process any transaction of length L in O(k) space and O(kL) time, keeping the top-k itemsets with respect to their approximate supports. However, Skip LC-SS must increment the maximum error by one, whenever a transaction t i that satisfies 2 |t i | > k is processed; hence, it suffers from a rapid increase in in accordance with k. By introducing the condensed representation (i.e., -covered set), the proposed method enables to reduce the maximum error, as demonstrated in the experiment later.
Hence, our important contribution lies in proposing the first online PARASOL method that seeks a -covered set of the FIs. It can process any transaction of length L in O(k) space and almost in O(kL) time, unlike to the existing online PC and NC methods (Manku and Motwani 2002;Karp and Shenker 2003;Jin and Agrawal 2005;Chi et al. 2004;Jiang and Gruenwald 2006;Song et al. 2007;Li et al. 2008;Cheng et al. 2008;Yen et al. 2011;Shin et al. 2014;Quadrana et al. 2015). Compared with the existing RC method (Yamamoto et al. 2014), our solution is more accurate in the sense that is much smaller, while the support of every FI can be restored within an error bounded by .
We also highlight the issue on data structure to efficiently perform the key operations of the proposed method. Especially, we focus on a binomial spanning tree, called weeping tree. Unlike the previously proposed data structures, such as prefix trees (Borgelt et al. 2011;Shin et al. 2014) and vertical format index (Yen et al. 2011), the weeping tree captures not only each itemset but its occurrence sequence in the transactions. As a result, the weeping tree exhibits several interesting features that are essential to prune redundant update process. To our knowledge, this is the first attempt to demonstrate the applicability of binomial spanning trees in the context of FIM-SD.

Notation and terminology
Let I = {1, 2, . . . , N} be the universal set of items. Itemset t is a non-empty subset of I , i.e., t ⊂ I . Data stream S n is the sequence of itemsets t 1 , t 2 , . . . , t n for which t i ⊂ I for i = 1, 2, ..., n where n denotes the timestamp for which an output is requested by the user. Each t i is called transaction at timestamp i. The number of items in t i (i.e., the cardinality) is denoted as |t i | and is referred to as the length of t i . L denotes the maximum length of the transactions in S n . For itemset α ⊂ I and timestamp i, tran(α, i) denotes the family of itemsets each of which includes α as a subset at timestamp i (i.e., tran (α, i) Given a minimum support threshold σ (0 ≤ σ ≤ 1), if sup(α, n) > σ n, then α is frequent with respect to σ in S n . F n denotes the family of FIs with respect to σ at timestamp n in S n , i.e., F n = {α ⊂ I | sup(α, n) > σ n}. FI α is closed if there is no α's proper superset whose support is equal to α's support in F n . On the other hand, FI α is maximal if there is no α's proper superset in F n .
Next, we define two key notions to describe a condensed representation of the FIs as follows: Definition 1 ( -cover) Let α and β be two itemsets. If α ⊆ β and sup(α, i) ≤ sup(β, i)+ for a non-negative integer , then α is -covered by β at timestamp i; this state is denoted by α i β.
Definition 2 ( -covered set) Let P and Q be two families of itemsets. If ∀α ∈ P ∃β ∈ Q such that α i β for a non-negative integer , then Q is a -covered set of P at timestamp i.
A -covered set is a generalization of the closed itemsets. The closed itemsets of the family F n of FIs compose a 0-covered set of F n . Note also that each FI and its support can be recovered from a -covered set within an error bounded by ; in the following, this property is called -deficiency. Let us consider again the 2-covered set Q 2 , in Example 1.
The objective is to find a -covered set of F n for a non-negative integer, , while processing each transaction t i (1 ≤ i ≤ n) only once. As explained before, PC mining and RC mining approaches for FIM-SD have been established. PC mining controls according to an error parameter, , such that ≤ × i for each timestamp i; however, in the worst case, an exponential number of candidate itemsets must be stored with this approach. Indeed, it is not feasible for any PC method to solve this problem in O(k) space for a constant k.
Hence, we aim to design a novel method by integrating PC and RC mining approaches so that is kept as small as possible based on a given value of k, which bounds the memory consumption. Herein, we present a baseline method to find a -covered set of F n for a given k in O(kLn) time and O(k) space, where L is the maximum length of transactions and n is the end timestamp.

Baseline algorithm for online -covered set mining
The baseline algorithm is built on two key techniques: incremental intersection and minimum entry deletion, which are described in detail in this section.
Incremental intersection (Borgelt et al. 2011;Yen et al. 2011) is used for computing the closed itemsets. It is based on the following cumulative and incremental features of the closed itemsets. Let C i be the family of closed itemsets in S i .
Theorem 1 (Borgelt et al. 2011;Yen et al. 2011) Given C i−1 at timestamp i − 1 and transaction t i at timestamp i, C i is given as follows: Proof This recursive relationship has been revealed in the literature (Borgelt et al. 2011;Yen et al. 2011). We also give a brief sketch of the proof in Appendices A and B.
Theorem 1 ensures that C i can be computed from the intersection of each itemset in C i−1 with t i . Figure 3 illustrates how C i is incrementally generated until i = 3. C 1 consists of the first transaction. C 2 newly contains α 2 and α 3 , which correspond to the second transaction and its intersection with α 1 , respectively. Finally, C 3 and C 4 consist of the seven and fifteen closed itemsets, respectively.
An incremental intersection never unfolds a transaction to intermediate subsets. We just intersect each stored itemset by the transaction. Then, the update operation never involves the subset enumeration of itemsets. Besides, we do not need to check if the generated one is a closed itemset by Theorem 1. In this sense, incremental intersection can simply but efficiently compute only the closed itemsets. On the other hand, lossless compression based on closedness cannot necessarily control an exponential increase of the closed itemsets; in the worst case, (2 L ) closed itemsets can be generated. In Example 2, there is a total of Hence, it is required to embed a space-saving technique into incremental intersection. In this paper, we introduce the notion of minimum entry deletion which has been proposed in RC mining (Metwally et al. 2005;Yamamoto et al. 2014). Given a size constant, k, minimal entry deletion is performed when the number of stored itemsets exceeds k to keep only the top-k itemsets; in other words, the itemset with the lowest approximate support must be deleted iteratively as long as the number of stored itemsets is greater than k. The maximum value in the approximate supports of the deleted itemsets is then maintained as the maximum error, . It is reasonable for space-saving to embed the same scheme into the update process by incremental intersection.
The baseline algorithm manages two types of information for each stored itemset: its approximate support count and maximum error count. They are represented as a tuple entry of the form α, c, , which represents the stored itemset, the approximate support count, and maximum error count, respectively. In the following, we call c and the frequency count and error count of α, respectively. Besides, the itemset, frequency count, and error count of an entry, e, are often denoted by α e , c e , and e , respectively.
Example 3 Consider S 1 4 again. Let k = 3. The number of C 3 is beyond k = 3 at timestamp i = 3. As shown in Fig. 4, we remove four closed itemsets in order of increasing frequency count; the highest count of those that are removed is two. The next transaction, t 4 , is stored in the entry t 4 , 3, 2 , meaning that the support of t 4 is between one and three (i.e., 1 ≤ sup(t 4 , 4) ≤ 3).
In the following, T i denotes the collection of entries used at timestamp i, k(i) denotes the number of entries in T i , and (i) denotes the maximum error at timestamp i. A minimum entry in T i is an entry whose frequency count is the lowest of those in T i . Then, the baseline algorithm is described in Algorithm 1. The updating process is composed of entry addition and entry deletion for each timestamp. The addition operation is carried out by the function intersect (T i−1 , t i ), which performs the incremental intersection. The deletion operation is realized by the function delete(T i ), which performs the minimum entry deletion.

Fig. 4 Example of minimal entry deletion
Example 4 Consider T 4 again as shown in Fig. 4. T 4 corresponds to the output of Line 5 in Algorithm 1 at i = 4 for the input stream S 1 4 and k = 3. By the deletion process of Line 6, (4) becomes three. Now, we assume that the top three entries in T 4 remain as shown in Fig. 5. Next, let the transaction arriving at i = 5 be t 5 = {1, 3, 5}. Then, by the addition process, we first add the entry t 5 , 3, 3 to T 4 , since there is no entry for t 5 in T 4 . After that, the intersection of each stored itemset in T 4 with t 5 is computed and stored in C. In total, C consists of three entries. Then, T 5 is obtained by updating T 4 with C by way of Lines 26-33 in the intersect function. Finally, the minimum entry, {2, 5}, 3, 0 , is deleted from T 5 to ensure that k(5) ≤ k. Next, we clarify the quality of the output by the baseline algorithm.

Theorem 2 Algorithm 1 outputs a (n)-covered set of the FIs wrt σ , provided that (n) ≤ σ n.
This implies that for every FI, α, T n must contain an entry, e, in which α is (n)-covered by α e . To demonstrate this feature, we introduce the notion of a representative entry.
Definition 3 Let α be an itemset. Given T i , T α i is used to denote the set of entries in T i that contains such an itemset β that α ⊆ β. If T α i is not empty, a representative entry, r, for α is defined as an entry that has the maximum frequency count of those in T α i (i.e., r = argmax e∈T α i (c e )).
Then, for every FI, α, and a representative entry, r, of α, we claim that α is (n)-covered by α r . This claim can be explained as shown in Fig. 5: First, we prove the following proposition and two lemmas.
Proposition 1 Let e be an entry in T i . Then, e is a representative entry for α e .
Proof. Suppose that e is not a representative entry. Then, there exists another entry, e ∈ T i , such that α e ⊆ α e and c e < c e . This contradicts the anti-monotonicity in T i , which is proved by using the mathematical induction, following the below argument. First, since T 1 contains only one entry, T 1 satisfies the anti-monotonicity property. Assume that T i satisfies this property (i.e., for every two entries, e and e , if α e ⊆ α e then c e ≥ c e holds). Then, by the cumulative feature of the incremental intersection, the output of the function intersect (T i , t i+1 ) also exhibits anti-monotonicity. Moreover, this property is preserved by the deletion process. Therefore, T i+1 also exhibits the anti-monotonicity.

Lemma 1 Let α be an itemset. Given
Proof We use the mathematical induction. T 1 consists of only one entry, t 1 , 1, 0 . Since k > 0, this entry is not deleted. Accordingly, this lemma is true in the case of T 1 . Assume that T i satisfies the claim of the lemma. Let α be an itemset whose representative entry, r i+1 , exists in T i+1 . Now, we consider two cases for the existence of a representative entry r i for α in T i .
Case 1: r i exists. Based on the assumption, we have c r and T α i+1 = ∅ (i.e., the maximum frequency count in T α i is preserved in T α i+1 by the minimum entry deletion); thus, since sup(α, i + 1) = sup(α, i), c r i+1 − r i ≤ sup(α, i + 1) ≤ c r i+1 . Otherwise, since α ⊆ t i+1 , c r i+1 = c r i +1 holds because T i+1 contains the entry for α r i ∩t i+1 which has a frequency count of c r i + 1. Since sup(α, i + 1) = sup(α, i) + 1, Case 2: r i does not exist. This means that α or its supersets never appear or have been deleted before the time i. Hence, sup(α, i) is at most (i), because (i) is the upper bound of the frequencies of the deleted itemsets. Since r i+1 exists but r i does not, r i+1 corresponds to t i+1 , 1 + (i), (i) . Hence, α ⊆ t i+1 holds. Accordingly, 1 ≤ sup(α, i + 1) ≤ 1 + (i) holds. Since c r i+1 = 1 + (i) and In both cases, T i+1 also satisfies the claim of the lemma.
Lemma 2 For every itemset, α, such that sup(α, i) > (i), there exists a representative entry r ∈ T i for α.
Proof We also use the mathematical induction. The lemma is obviously true in the case of T 1 since (1) = 0 holds. Assume that T i satisfies the claim of this lemma; we show the induction step by contradiction. Assume that ∃α such that sup(α, i + 1) > (i + 1) but there is no representative entry for α in T i+1 . We can consider two cases: whether or not a representative entry, r, has existed in T i+1 before the deletion operation (i.e., in the intermediate T i+1 at the moment when the incremental intersection has been completed, as depicted in the middle T i+1 in Fig. 6).
Case 1: such an r exists in T i+1 but was deleted. By Lemma 1, we have c r − r ≤ sup(α, i + 1) ≤ c r . Since r was deleted, c r ≤ (i + 1) should hold. Thus, sup(α, i + 1) ≤ (i + 1) also holds. However, this is a contradiction. Case 2: such an r does not exist in T i+1 . Here, T i+1 captures the entry table before the deletion operation. Thus, T α i ⊆ T α i+1 holds. Since there is no representative entry for α in T i+1 , it holds that T α i+1 = ∅ and T α i = ∅. Note that T i+1 should contain the entry for t i+1 . Hence, α ⊆ t i+1 because T α i+1 = ∅. Therefore, sup(α, i) = Fig. 6 Sketch of updating T i to T i+1 with t i+1 sup(α, i + 1). Since sup(α, i + 1) > (i + 1) and (i + 1) ≥ (i), it should hold that sup(α, i) > (i). Then, by the assumption for T i , there should exist a representative entry for α in T i . However, this contradicts the conclusion that T α i = ∅. In both cases, there is a contradiction. Therefore, for every α such that sup(α, i + 1) > (i + 1), there exists a representative entry for α in T i+1 . Now, we prove Theorem 2 as follows: Proof of Theorem 2 Suppose that the condition of σ (i.e., (n) ≤ σ n) is satisfied. Then, by Lemma 2, for every FI α wrt σ , there exists a representative entry, r, in T n for α. By Lemma 1, c r − r ≤ sup(α, n) ≤ c r holds. By Proposition 1, r is also a representative entry for α r . Thus, we have c r − r ≤ sup(α r , n) ≤ c r . Hence, sup(α, n) ≤ sup(α r , n) + r holds. Since α r ⊇ α, α is r -covered by α r . Since r ≤ (n), we can claim that for every FI α, there exists an entry, r , such that α is (n)-covered by α r . Note that c r > σn holds because sup(α, n) > σ n. Hence, r is contained in the output because of Lines 9-11 in Algorithm 1.
Next, we clarify the complexity of the baseline algorithm. Incremental intersection generates at most k(i) + 1 new entries for the intersections of the entries in T i with t i+1 as well as t i+1 itself; these new entries are added to C in Lines 16 and 21 of the intersect function. Thus, the total number of stored entries is at most 2 × k(i) + 1, and k(i) is always bounded by the size constant k. Hence, the complexity of the baseline algorithm can be described as follows: Theorem 3 For a size constant k and the maximum transaction length L, Algorithm 1 processes every transaction in O(kL + k log k) time and O(k) space.
Proof First, the number of stored entries is at most 2k + 1 for every time point. Thus, the space required to store them is O(k). In addition, it takes at most O(2L) time to compute the intersection of a stored itemset with a transaction. Thus, the incremental intersection is performed in O(kL) time. It takes at most O(log k) time to delete a minimum entry and add a new entry by using a heap structure. Hence, the total time required to update the table is at most O(kL + k log k).
Based on Theorems 2 and 3, Algorithm 1 can extract a -covered set of the FIs wrt σ ≥ n for an integer and can process each transaction in O(kL) time and O(k) space for a size constant k (k < 2 L ). Therefore, Algorithm 1 satisfies the three characteristics (i.e., on-the-fly manner, anytime feature, and quality of output) presented in Section 1.

Parasol: a hybrid of PC and RC approximation
We observe in Fig. 1 that a sudden and intensive memory consumption may occur as a temporary phenomenon. In other words, the PC approximation is feasible unless all of the memory is consumed during this short period. Hence, it is reasonable to take a hybrid of PC and RC approximation; the memory consumption is generally controlled by using an error parameter and an RC approximation using a size constant k is taken only when the number of used entries exceeds k. We call this hybrid approximation approach PARASOL (an acronym for Parameter-and Resource-constrained Approximation for Soft and Lazy mining).
Our proposed PARASOL method can be realized just by replacing the delete function in Algorithm 1 with the following function: The main difference appears in the condition of Line 3. Unless sudden and intensive memory consumption occurs (i.e., k < k(i) holds), the proposed PARASOL method performs the minimal-entry deletion in the PC approximation manner. For this ordinary scenario, it always ensures (i) ≤ × i. Otherwise, it performs the minimal-entry deletion in the RC approximation manner due to the constant space constraint. For this exceptional scenario, it allows (i) to exceed × i. Since sudden and intensive memory consumption is a temporal phenomenon, the RC approximation is only required for a short period. Therefore, the error ratio, (i) i gradually converges to after this short period. This self-sustained recovery of the error ratio is a unique and advantageous characteristic of the proposed PARASOL method, and will be empirically demonstrated in experiment later.

Furthermore improvement
Note that the solution to our problem setting is not unique: there can exist many -covered sets of F. Then, it is reasonable to extract a concise -covered set. Here, we propose a fast post-processing technique to reduce the original output to a more concise one based on the maximum error, (n).
First, we explain the intuition using T 5 (the right-most table in Fig. 5) in Example 4. Since (5) = 3, the three itemsets stored in T 5 compose a 3-covered set of the FIs. Interestingly, even if {1, 5} was deleted from T 5 , the two remaining itemsets could compose a 3-covered set. Let us consider the entry r = {1, 5}, 4, 0 , and let α be an itemset whose representative entry is r. By Lemma 1, the support of α is at most c r . Since c r is the same as the frequency count for {1, 3, 5}, α is also 3-covered by {1, 3, 5}. Therefore, there is no need to keep r in T 5 for restoring the support of α when = 3.
There is another choice point to delete the entry for {5} from T 5 . In exchange for deletion, we need to update the entry for {1, 5} by {1, 5}, 5, 1 (i.e., incrementing the frequency and error counts by one). In this way, the support of {5} can be restored from the entry for {1, 5}.
In general, it is infeasible to identify such itemsets to be deleted from an original (n)covered set so that the remaining set keeps the (n)-deficiency. However, some of them can be simply detected based on the following corollary: Corollary 1 For every entry e ∈ T n , if there exists r ∈ T n such that α e α r and c e ≤ c r − r + (n), then the itemsets of T n − {e} compose a (n)-covered set of the FIs wrt σ , provided that (n) ≤ σ n.
Proof Let e be an entry in T n and α an arbitrary itemset whose representative entry is e. Suppose that there exists an entry r such that α e α r and c e ≤ c r − r + (n). We claim that α is (n)-covered by α r . By Lemma 1, it holds that sup(α, n) ≤ c e , followed by sup(α, n) ≤ c r − r + (n). Since c r − r ≤ sup(α r , n), we have sup(α, n) ≤ sup(α r , n) + (n). Since α ⊆ α e and α e α r , we also have α α r . Hence, α is (n)covered by α r . The claim implies that even if e was removed, the support of α can be recovered using r, while (n) remains as it is. Now, we propose a post-processing technique, called -compression, which incrementally deletes the entry from T n according to Corollary 1, as described in Algorithm 2. In exchange for deleting e, r is updated by increasing it by the difference between c e and c r , and c r is updated to c e in Lines 3 and 4.
Note that the output produced by Algorithm 2 does not necessarily compose a minimal (n)-covered set of the FIs. The counter example is derived by using T 5 of Example 4. After applying Algorithm 2, T 5 contains two itemsets {1, 3, 5} and {5} (alternatively, {1, 3, 5} and {1, 5}). However, only {1, 3, 5} is enough to compose a 3-covered set of the FIs (σ n is now assumed to be greater than or equal to 3). Thus, the output is not a minimal 3-covered set. A simple implementation of Algorithm 2 is that for each entry e, we search T n for a corresponding r. This can be achieved in O(k 2 ) time, since |k(n)| ≤ k.

Data structure
Here, we address the issue of the data structure that is needed to efficiently realize the three key operations of the proposed algorithm: incremental intersection, deletion, andcompression. Of these, incremental intersection incurs the majority of the computational cost. This operation imposes the traverse of every entry, e, to compute its intersection with a transaction, t i , for each timestamp i. This computation is often redundant: for example, if α e has no common items with t i , it is redundant to compute the intersection of α e with t i .
Based on this observation, Yen et al. (2011) proposed an indexing data structure, called cid list, corresponding to the vertical format of the stored itemsets: for each item, x, cid list (x) maintains the indexes of the entries corresponding to the itemsets that contain x. Using cid list, we can focus only on the entries whose indexes are contained in x∈t i cid list (x) and compute their intersections with t i . However, the computational cost of updating cid list is relatively high: the entire cid list is dynamically changed by addition and deletion operations. This overhead becomes especially high for dense datasets since most itemsets stored in T i have some of the same items as t i . Borgelt et al. (2011) proposed a fast two-pass FIM method, called ISTA, based on incremental intersection. In this implementation, the prefix tree (as well as patricia) was introduced to efficiently maintain T i and perform the incremental intersection. Although it is reasonable to represent T i with such a concise data structure, it is not directly applicable in the one-pass approximation setting that is used here. For example, we cannot use the item support as static information, while it is available in the transactional database that allows multi-pass scanning. This information is crucial to constructing a compact trie by sorting items in a pre-processing step. Note that the trie size can be directly affected by the order of (sorted) transactions. Indeed, ISTA constructs the trie with 306 nodes by treating it as a retail problem (No. 3 in Table 1), compared to 244,938 nodes when the pre-processing technique is not applied. Besides, in the context of the SD that emerges concept drift, it is not appropriate to assume a static distribution.
Thus, it is necessary to design a more suitable data structure for our proposed algorithm, which enables to prune redundant computations in incremental intersections, as well as quickly access both the minimum entries and -covered entries required for PARASOL deletion and -compression, respectively.

Weeping tree
Now, we propose a variation of the binomial spanning tree (Johnsson and Ho 1989;Chang 2005), called weeping tree, in which a collection of entries, T n , can be represented in a binary n-cube as follows: Let e be an entry in T n . By Theorem 1, α e corresponds to the intersection of a certain set S of transactions. This set can be represented as a binary address (x 1 , x 2 , . . . , x n ) where each x j is one if S contains t j and zero otherwise. Every α e has its own binary address. Thus, T n can be represented as a set of binary addresses, denoted by V (n), each of which identifies α e for an entry e ∈ T n . Each binary address can be described by an integer, x = n j =1 (x j × 2 n−j ). Then, V (n) corresponds to a subset of {1, . . . , 2 n − 1}. Let x and y be two integers (1 ≤ x, y ≤ 2 n − 1) with the addresses (x 1 , x 2 , . . . , x n ) and (y 1 , y 2 , . . . , y n ), respectively. p(x) is the position in x that satisfies two conditions: x p(x) = 1 and x j = 0 for each j (p(x) + 1 ≤ j ≤ n). In other words, p(x) is the least significant set bit of x. We say that x covers y if y j = x j for each j where 1 ≤ j ≤ p(x). For example, if x = 12 and y = 15 in 4-cube with addresses (1100) and (1111) then, p(x) = 2, y 1 = x 1 , and y 2 = x 2 so x covers y. We assume that zero covers every integer. Now, we define the binomial spanning tree of V (n), following the notion in the literature (Chang 2005). Definition 4 (Binomial spanning tree) Let V (n) be a subset of {1, . . . , 2 n − 1}, x be an integer such that 0 ≤ x ≤ 2 n − 1, and C(x) be the set of integers in V (n) each of which is covered by x. The binomial spanning tree of V (n) is the tree in which the root node, r, is zero and the other nodes are V (n). The children of each node, x, correspond to the following set: {y | y ∈ C(x) and ∃y ∈ C(x) s.t. y = y and y ∈ C(y )}.
The siblings y (1) , y (2) , . . . , y (m) are sorted in the descending order (i.e., y (j ) is the left sibling of y (j +1) ). We say that a non-root node, y, is a descendant of node x if y ∈ C(x), a precursor of x if y ∈ C(x) and y > x, and a successor of x if y < x. The precursors and successors of x are denoted at P (x) and S(x), respectively.
The weeping tree at time n, denoted by W (n), is the binomial spanning tree of V (n) obtained by associating each node, x, with its corresponding entry e (i.e., α e is the intersection of the transactions indicated by the address x).
One crucial feature of the weeping tree is that it captures inclusion relationships among the stored itemsets in T n . Fig. 7 The weeping tree W (4) wrt S 2 4 Proposition 2 Let two nodes x and y be associated with two entries e x and e y . If x covers y, then α e y ⊆ α e x Proof The address of y can be written as (x 1 , . . . , x p(x) , y p(x)+1 , . . . , y n ), since x covers y. Accordingly, α e y is written as ( x j =1,1≤j ≤p(x) t j ) ∩ I where I = y j =1,p(x)+1≤j ≤n t j . Since p(x) is the position of the least significant set bit, α e x = x j =1,1≤j ≤p(x) t j holds, followed by α e y = α e x ∩ I . Proposition 2 has three useful implications. First, it is applicable for pruning the intersection computations. Suppose that during the updating process at time i, an entry, e, is found such that α e ⊆ t i . Since every descendant of e must be included in t i , it is not necessary to compute the intersections for these descendants. Proposition 2 also implies that every minimum entry must be located in the shallowest layer in the tree due to the anti-monotonicity of T i . This feature is useful for the minimum entry deletion (in practice, it is reasonable to use min-heap for the shallowest layer).
Finally, Proposition 2 is applicable to the pairwise checking involved in -compression. Suppose that we found a parent entry, e p , and its child entry, e c such that C e c ≤ C e p − e p + (n). Then, α e c must be (n)-covered by α e p according to Corollary 1. Hence, a quick check can be done to determine if each child is (n)-covered by its parent. Note that a bruteforce approach requires O(k 2 ) time for -compression while the quick pairwise checking can be completed in O(k) time. Thus, it is useful as a pre-processing step precedingcompression.

Weeping tree updating
Here, we explain how to incrementally update W (i) at each time i. Suppose that an itemset, α, is newly stored in W (i + 1). An address (x 1 , . . . , x i , 1) is then assigned to α, where (x 1 , . . . , x i ) is the address of the node, r, in W (i) that corresponds to a representative entry of α if such an r exists; if there is no such node, r is the root node. Thus, the entry for α is newly located as a child of node r.

) in T (3).
Since r is given the address (101) at time i = 3, the address of α becomes (1011), and the entry for α is located as a child of r.
We can interpret the meaning of the address assigned to each entry. Let e be an entry of node x with the address (x 1 , . . . , x n ). We denote the least and greatest significant bit sets of x as p(x) and q(x), respectively. Then, e is written as α, (q(x) − 1) + B, (q(x) − 1) , where α = x j =1,q(x)≤j ≤p(x) t j , (q(x) − 1) is the maximum error at time q(x) − 1 and B is the bit count of x (See Fig. 8). This observation leads to the following proposition: Proposition 3 Consider two nodes, x and y, of entries e x and e y , respectively, in a weeping tree. If α e y ⊆ α e x , then y is either a descendant or a precursor of x.
Proof We derive a contradiction in the case that α e y ⊆ α e x and y < x. Let v be the address obtained by the bitwise OR operation between x and y. Since y < x and x ≤ v, y < v. We write v as (v 1 , . . . , v n ) and denote by α v the intersection v j =1,1≤j ≤n t j . Since α e y ⊆ α e x , we have α v = α e y . Thus, node v should not appear in the tree, since its duplicate never occurs in T n . Without losing generality, this implies that v has been deleted at some time, which is referred to as time m. Accordingly, the tree never contains such a node, u, with an address (u 1 , . . . , u n ) such that q(u) = q(v); moreover, the bit count of the m-prefix (u 1 , . . . , u m ) is lower than the bit count of the m-prefix (v 1 , . . . , v m ). This is because every node with such an address has been deleted at time m, along with v (i.e., u has a lower frequency count than v). Hence, no node can have an address in which the m-prefix matches (u 1 , . . . , u m ) or (v 1 , . . . , v m ). Next, we consider the address of node x. Since y < x and v is obtained by a bitwise OR operation between x and y, we have q(x) = q(v). In addition, (x 1 , . . . , x m ) is either equal to (v 1 , . . . , v m ) or has a lower bit count than (v 1 , . . . , v m ).
Hence, x should not appear in the tree. This is a contradiction.
For example, consider node 6 for the itemset {1, 4, 5} as shown in Fig. 7. There are three nodes 7, 14, 15 that have subsets of this itemset and each of these nodes is either a descendant or a precursor of the node 6.
Proposition 3 is useful for pruning the computation for incremental intersection. Suppose that for some entry C, t i ⊂ α C holds. Thus, the intersection computations for every successor of C with t i can be skipped as they do not store any subset of t i (See Fig. 9). The weeping tree can be used to perform the incremental intersection by traversing the weeping tree in a depth-first, left-to-right manner. Algorithm 3 sketches the process for updating a node x in W (i) with an itemset E. Note that E is initially a transaction.
In the algorithm, a node, x, is identified with its associated entry, e x . Here, α x , c x and x are the itemset, frequency, and error count of e x , respectively. Given the transaction t i+1 and W (i), the next tree, W (i + 1), is obtained by calling the function update(root, t i+1 , W (i)). Note that W (0) is defined as the initial tree consisting of the root node.
A few characteristics of the update(x, E, W (i)) algorithm should be noted: -Line 4 means that α y ⊆ E. By Proposition 2, the descendants of y are included by E. Thus, the frequency count of each node, z ∈ C(y), can be simply incremented without computing the intersection except for α y itself. This is called descendant-intersectskipping (DIS). -In Line 9, we continue the updating process. In the recursive call, the intersection, I , is used instead of the original itemset, E. It follows that α y ∩ E = α y ∩ I for each child y of y since I = α y ∩ E. By reducing E to I (i.e., I is a subset of E), the computational cost of the recursive call after Line 9 is reduced. This pruning technique is called masking. -IfI = ∅, every descendant of y has no items in common E so the descendants need not be updated. This is called descendant-update-skipping (DUS). -Line 11 checks if E ⊆ α y or not. If so, every right sibling of y need not be updated. This follows from the observation in Proposition 3 that the entry for any subset of E to be updated never appears in the successors of y. This is called successor-update-skipping (SUS). -Finally, if there is no entry for E in W (i), the new entry for E is added as the right-most child of x. Note that if x is the root node, we set c x = 0 and x = (i).  Fig. 10. The function update (root, t 4 , W (3)) is called to derive W (4) from W (3). For the left-most child e 1 , the intersection, I 1 = {1, 2, 5} of e 1 with t 4 is computed. Since I 1 = ∅, we call update(e 1 , I 1 , W (3)) as shown in Line 9. For the left-most child, e 2 , of e 1 , the intersection I 2 = {1, 2} of e 2 with I 1 is computed using I 1 by masking. Since I 2 = α e 2 , DIS is applied in Lines 5-7. Then, c e 5 is simply incremented by one. Moving to the right sibling, e 4 , the intersection I 4 = {2} of e 4 with I 2 is computed. Since I 4 = ∅, update(e 4 , I 4 , W (3)) is called. Since e 4 does not have any children, the algorithm checks if there exists the entry for I 4 then backtrack to the second call (i.e., update(e 1 , I 1 , W (3))). Since there is no sibling of e 4 , the algorithm returns to Line 15 and a new entry, e 7 , for I 1 is added as the right most child of e 1 . After backtracking to the first call, SU S is applied in Line 11 since I 1 = t 4 . Thus, the updating of the two right-most sibling nodes, e 3 and e 6 , is skipped and the algorithm proceeds to Line 15. Since an entry for t 4 exists, the updated tree, W (4), is returned as the output.
In this way, the update function realizes the incremental intersection. Note that every node in W (i) is visited at most once, which implies that Algorithm 3 efficiently runs update(root, t i+1 , W (i)) to return W (i + 1) in O(kL) time.
Next, we show how PARASOL realizes the minimum entry deletion in the weeping tree. As explained before, the minimum entries to be deleted are allocated in the shallowest layer relative to the root. Recall Example 5, in which was set to 0.25 and PARASOL was used to delete the entries with frequency counts of one at time i = 4. These minimum entries can be quickly accessed by applying min-heap to the shallowest layer. The reduced weeping tree is obtained by reconnecting the children of the deleted nodes with the root as shown in Fig. 11.
The weeping tree can be used as a pre-processing step for the -compression in such a way that we find such a child node e that c e ≤ c r − r + (n) where r is the parent node of e. By Corollary 1, e can be deleted from the tree.  Figure 11 shows the reduced tree obtained by searching the weeping tree W (4) in a bottom up manner from the left-most leaf to the root; this process results in the removal of four nodes (7, 11, 13, and 15). Note that -compression requires O(k 2 ) time to completely check every pair of nodes. Thus, it is reasonable to carry out the -compression in a two-step procedure; first checking each pair of parent-child nodes in a one-time scan and subsequently performing the brute-force search of the remaining nodes.

Experiment
Now, we empirically evaluate the performances of the baseline algorithm, PARASOL, and -compression, respectively. They have been implemented with the weeping tree (in C language 2 ). Ten real datasets were collected from Yamamoto et al. (2014) (Nos. 1-2), the FIMI repository 3 (Nos. 3-9), and a new benchmark (No. 10) created from the Yahoo! Hadoop grid logs dataset 4 as shown in Table 1. Note that n, |I |, L, and L ave are the end time, the number of different types of items, the maximum transaction length, and the average transaction length, respectively.
First, the baseline algorithm was compared with the state-of-the-art RC method, Skip LC-SS (Yamamoto et al. 2014), on the existing real datasets (Nos. 1-9). Then, the scalability of the baseline algorithm was demonstrated on the synthesized datasets generated by the IBM market basket generator. Next, the performance of PARASOL was compared with those of the baseline algorithm and the state-of-the-art PC method, MOA-IncMine (Quadrana et al. 2015), when applied to the large-scale dataset (No. 10). Finally, we evaluated the effectiveness of -compression in reducing the output. Table 2 describes the overall improvement that has been realized by the baseline algorithm, compared with Skip LC-SS, for each dataset in terms of the maximum error, (n), the error ratio, (n) n , and the average updating time (msec). In this experiment, the size constant, k, was fixed as 12,000. The results show that the baseline algorithm drastically reduced the error. Notably, in datasets 4, 5, 6, 8, and 9, the error ratio with Skip LC-SS reached one, meaning that the FIs could not be found for any σ (0 < σ < 1), while the baseline provided a (n)-covered set of the FIs for which (n) ≤ σ n. Moreover, the updating time of the baseline algorithm was lower than the one of Skip LC-SS.
Skip LC-SS was compared with the baseline algorithm with a range of size constants for the dataset 7, on which Skip LC-SS performed the best in terms of the error ratio. The results are summarized in Fig. 12a.
The data shows that the baseline algorithm (bold lines) performed better than Skip LC-SS in terms of maximum error and execution time for all values of k. Moreover, the execution time decreased linearly and (n) increased as k was decreased.
Next, the scalability of the baseline algorithm was evaluated by varying the maximum transaction length, L, as it is applied to the synthesis data. This dataset was generated by the IBM market basket generator; its stream size, n, was 10,000 and the number of types of items, |I |, was about 24,000. Figure 12b describes the average updating time and the error ratio, (n) n , as functions of L. As the time complexity for the updating process is O(kL), the execution time linearly increased as L was increased. In an ordinary problem setting, we need to focus the solution space on 2 L itemsets for L, resulting in a rapid increase in the error count. In contrast, the proposed solution can tolerate such large transactions that always contain 5,000 items. In addition, the change in the error ratio as a function of k in the range of 1,000,000 to 7,000,000 was evaluated with the dataset 6, for which the error ratio achieved by the baseline algorithm was the largest. The results are shown in Fig. 12c. Then, the error ratio linearly decreased while the execution time increased as k was increased. Note that a simple open MP parallelization was used for this computation. Such a parallelization scheme can be used in RC approximation with a fixed memory resource (Yamamoto et al. 2016) but the issue of efficient implementation of such a scheme is beyond the scope of this paper. i , respectively, which are obtained by applying PARASOL to the hadoop dataset (No. 10) with respect to k = 10, 000 and = 0.015. The results show that PARASOL violates the original value due to the RC approximation that was applied earlier in the time series period. However, after this period, the error ratio recovers to . This autonomous recover of is an interesting characteristic of PARASOL. Note that PARASOL finally derives T n of 1,626 entries while the baseline algorithm always uses the 10,000 entries. Another characteristic of PARASOL lies in this observation that T n becomes smaller than k due to the PC approximation. In contrast, MOA-IncMine generates 770,995 SFCIs for some intermediate segment when = 0.1 and a segment size of w = 1, 000; thus, it could not finish updating the SFCIs (this was true for other values of w, as well). The drastic decrease in performance due to streaming transactions (or segments) is a common drawback among the existing PC methods. PARASOL can avoid it and remains sustainable by switching between PC and RC approximations. Table 3 compares the performance of MOA-IncMine with that of PARASOL withcompression when applied to the other datasets in terms of execution time and number of entries in the output. Given = 0.1, MOA-IncMine was run with 1 GB Java heap size for six hours (the segment size was fixed as the default (1,000)). PARASOL was run with k = 100, 000 and = 0.1.
The results show that MOA-IncMine results in out-of-memory (OM) and time-out (TO) exceptions when applied to datasets 4, 5, 6, and 8. On the other hand, PARASOL successfully processed all of these datasets, however the error ratio ( (n) n ), exceeded the initial value of . In datasets, 1, 3, 7, MOA-IncMine was faster than PARASOL. These datasets were relatively sparse, implying that MOA-IncMine may tend to be more powerful for sparse datasets, although the speed also depends on the segment size (as well as the size constant k for PARASOL). The output of PARASOL was generally smaller than that of MOA-IncMine due to -compression as shown by number of used entries, k(n).   Table 4 shows the pruning effect of using the weeping tree structure (parameters were set as k = 12, 000 and = 0). The average ratio between traversing nodes and total nodes for each timestamp (i.e., Visiting ratio) and the average ratio between nodes that have nonempty intersections and total nodes for each timestamp (i.e., Non-empty ratio) are shown. The standard indexing technique (i.e., cid list) forces us to compute all the non-empty intersections. Interestingly, the weeping tree succeeds in reducing this search space by the indexing technique in datasets 2, 4, 5, 6, and 8. Those datasets are relatively dense. Note that the execution time with the weeping tree was faster than cid list for every dataset. This is due to the additional cost for maintaining the indexing data structure for each timestamp. The #Num − and #Num + columns show the number of entries after pre-processing and number of entries that were output by the -compression, respectively. For example in dataset 6, the ordinal 12,000 entries were reduced to 3,418 by pre-processing and 535 of these entries remained in the output after the second filtering step. The Time − and Time + columns show the execution time of the -compression with and without the pre-processing step, respectively. It is reasonable for -compression to apply the pre-processing with the weeping tree.

Conclusion and future work
In this study, we have proposed a novel solution for FIM-SD that involves seeking adeficient condensed representation, from which every FI and its support can be restored, while bounding the maximum error by an integer, . While the existing FIM-SD methods are limited to strictly PC and RC methods, we have mitigated the drawbacks of each approach by introducing a unified PC and RC-approximation scheme called PARASOL. We furthermore introduce a post-filtering technique called -compression and a novel data structure called the weeping tree. Experimental trials on ten datasets show that the proposed technique outperforms the existing FIM-SD methods. Moreover, the proposed algorithm is scalable in terms of the transactional length; in other words, it can tolerate any bursty transaction without running into an OM exception. This feature is favorable for analyzing large volumes of streaming transactions that consistently contain many items. Such large stream-ing transactions are often encountered in surveillance domains of sensor networks and cloud servers. In addition, along with the recent success achieved with deep learning, it is now necessary to analyze the cognitive correlations among mid-level objects for explanatory domains. We believe that integrating modern online learning with the solution presented here will give rise to a new methodology for streaming data analysis. From a technical standpoint, it will be important to introduce novel parallel-processing techniques to further improve the scalability of this technique. In addition, it will be fruitful to study how such techniques can be efficiently embedded into the proposed weeping tree structure.