Enumerating Maximal Frequent Sets Using Irredundant Dualization
In this paper, we give a new algorithm for enumerating all maximal frequent sets using dualization. Frequent sets in transaction data has been used for computing association rules. Maximal frequent sets are important in representing frequent sets in a compact form, thus many researchers have proposed algorithms for enumerating maximal frequent sets. Among these algorithms, some researchers proposed algorithms for enumerating both maximal frequent sets and minimal infrequent sets in a primal-dual way by using a computation of the minimal transversal for a hypergraph, or in other words, hypergraph dualization. We give an improvement for this kind of algorithms in terms of the number of queries of frequency and the space complexity. Our algorithm checks each minimal infrequent set just once, while the existing algorithms check more than once, possibly so many times. Our algorithm does not store the minimal infrequent sets in memory, while the existing algorithms have to store them. The main idea of the improvement is that minimal infrequent sets computed from maximal frequent sets by dualization is still minimal infrequent even if we add a set to the current maximal frequent sets. We analyze the query complexity and the space complexity of our algorithm theoretically, and experimentally evaluate the algorithm to show that the computation time on average is in the order of the multiplication of the number of maximal frequent sets and the number of minimal infrequent sets.
Unable to display preview. Download preview PDF.
- [Agrawal96]Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., Verkamo, A.I.: Fast Discovery of Association Rules. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, ch. 12, pp. 307–328 (1996)Google Scholar
- [Bayardo98]Bayardo Jr., R.J.: Efficiently Mining Long Patterns from Databases. In: Proc. of the 1998 ACM-SIGMOD, pp. 85–93 (1998)Google Scholar
- [Gunopulos97a]Gunopulos, D., Khardon, R., Mannila, H., Toivonen, H.: Data mining, Hypergraph Transversals, and Machine Learning. In: Proc. of PODS 1997, pp. 209–216 (1997)Google Scholar
- [Gunopulos97b]Gunopulos, D., Mannila, H., Saluja, S.: Discovering All Most Specific Sentences using Randomized Algorithms. In: Proc. of ICDT 1997, pp. 215–229 (1997)Google Scholar
- [Kavvadias99]Kavvadias, D.J., Stavropoulos, E.C.: Evaluation of an Algorithm for the Transversal Hypergraph Problem. Algorithm Engineering, 72–84 (1999)Google Scholar
- [Mannila96]Mannila, H., Toivonen, T.: On an Algorithm for Finding All Interesting Sentences. In: Cybernetics and Systems, The Thirteen European Meeting on Cybernetics and Systems Research, vol. II, pp. 973–978 (1996)Google Scholar
- [Satoh90]Satoh, K.: Formalizing Soft Constraints by Interpretation Ordering. In: Proc. of ECAI 1990, pp. 585–590 (1990)Google Scholar
- [Uno02]Uno, T.: A Practical Fast Algorithm for Enumerating Minimal Set Coverings. SIGAL83, Information Processing Society of Japan, 9–16 (2002) (in Japanese)Google Scholar