Interactive Discovery of Statistically Significant Itemsets
Frequent Itemset Mining (FIM) is a fundamental data mining task, which consists of finding frequent sets of items in transaction databases. However, traditional FIM algorithms can find lot of spurious patterns. To address this issue, the OPUS-Miner algorithm was proposed to find statistically significant patterns, called productive itemsets. Though, this algorithm is useful, it cannot be used for interactive data mining, that is the user cannot guide the search toward items of interest using queries, and the database is assumed to be static. This paper addresses this issue by proposing a novel approach to process targeted queries to check if some itemsets of interest to the user are non redundant and productive. The approach relies on a novel structure called Query-Tree to efficiently process queries. An experimental evaluation on several datasets of various types shows that thousands of queries are processed per second on a desktop computer, making it suitable for interactive data mining, and that it is up to 22 times faster than a baseline approach.
KeywordsItemset mining Productive itemsets Query-Tree Pattern
- 2.Webb, G.I., Vreeken, J.: Efficient discovery of the most interesting associations. ACM Trans. Knowl. Discov. Data 8(3), 15 (2014)Google Scholar
- 5.Fournier-Viger, P., Mwamikazi, E., Gueniche, T., Faghihi, U.: MEIT: memory efficient itemset tree for targeted association rule mining. In: Motoda, H., Wu, Z., Cao, L., Zaiane, O., Yao, M., Wang, W. (eds.) ADMA 2013, Part II. LNCS (LNAI), vol. 8347, pp. 95–106. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-53917-6_9CrossRefGoogle Scholar
- 6.Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: Proceedings of 20th International Conference on Very Large Databases, pp. 487–499. Morgan Kaufmann, Santiago de Chile (1994)Google Scholar
- 7.Llinares-López, F., Sugiyama, M., Papaxanthos, L., Borgwardt, K.: Fast and memory-efficient significant pattern mining via permutation testing. In: Proceedings of 21th ACM International Conference on Knowledgs Discovery and Data Mining, pp. 725–734. ACM (2015)Google Scholar