Cardinality Statistics Based Maximal Frequent Itemsets Mining

Abstract

Extracting frequent itemsets is an important task in many data mining applications. Since the result set of all the frequent itemsets are likely to be undesirably large, condensed representations, such as maximal and closed frequent itemsets are used. The set of frequent closed itemsets uniquely determines the exact frequency of all itemsets, yet it can be orders of magnitudes smaller than the set of all frequent itemsets. But whenever there are very long patterns present in the data, it is often impractical to generate the entire set of closed frequent itemsets. The only recourse is to mine the maximal frequent itemsets in the domain with very long patterns. In this paper, we propose a new approach for mining all maximal frequent itemsets which introduces and makes use of the compact data structure: Reduced Transaction Pattern List (RTPL), for representing the database. Our implementation exploits the advantages of combining RTPL representation with statistical information of cardinality of each item in database. We devise pruning strategy to substantially reduce the combinatorial search space by making use of statistical information of items at two levels. Our experiments using synthetic and real-world standard benchmark dataset shows that the proposed algorithm outperforms algorithms not using cardinality statistical information.