Advertisement

Data Mining and Knowledge Discovery

, Volume 28, Issue 3, pp 773–807 | Cite as

A time-efficient breadth-first level-wise lattice-traversal algorithm to discover rare itemsets

  • Luigi Troiano
  • Giacomo Scibelli
Article

Abstract

In this paper we face the problem of searching for rare itemsets. A main issue regards the strategy to adopt in exploring the power set lattice. Assuming a power set lattice with full set at the top and empty set at the bottom, the most of the algorithms adopt a bottom-up exploration, i.e. moving from smaller to larger sets. Although this approach is advantageous in the case of frequent itemsets, it might not be worth being used for rare itemsets, as they occur on the top of the lattice. We propose Rarity, a top-down breadth-first level-wise algorithm. Experimental results and comparisons are illustrated in order to provide a quantitative characterization of algorithm performances and complexity. Application to some UCI benchmark and real world datasets is provided. An algorithm parallelization is outlined. Experiments showed that this approach takes advantage of finding all rare non-zero itemsets in less time than other solutions, at expenses of higher memory demand.

Keywords

Data mining methods and algorithms Rare itemsets 

References

  1. Adda M, Wu L, Feng Y (2007) Rare itemset mining. In: Proceedings of 6th International Conference on Machine Learning and Applications, ICMLA ’07. IEEE Computer Society, Washington, DC, pp 73–80Google Scholar
  2. Agrawal R, Imieliński T, Swami A (1993) Database mining: a performance perspective. IEEE Trans Knowl Data Eng 5(6):914–925CrossRefGoogle Scholar
  3. Agrawal R, Imieliński T, Swami A (1993) Mining association rules between sets of items in large databases. ACM SIGMOD Int Conf Manag Data 22:207–216CrossRefGoogle Scholar
  4. Agrawal R, Mannila H, Srikant R, Toivonen H, Inkeri Verkamo A (1996) Fast discovery of association rules. In: Advances in knowledge discovery and data mining. AAAI/MIT Press, CambridgeGoogle Scholar
  5. Agrawal R, Shafer JC (1996) Parallel mining of association rules. IEEE Trans Knowl Data Eng 8(6):962–969CrossRefGoogle Scholar
  6. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: 20th VLDB ConferenceGoogle Scholar
  7. Bastide Y, Taouil R, Pasquier N, Stumme G, Lakhal L (2000) Mining frequent patterns with counting inference. SIGKDD Explor Newsl 2(2):66–75CrossRefGoogle Scholar
  8. Brin S, Motwani R, Ullman JD, Tsur S (1997) Dynamic itemset counting and implication rules for market basket data. In: SIGMOD ’97: Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data. ACM, New York, pp 255–264Google Scholar
  9. Burdick D, Calimlim M, Gehrke J (2001) Mafia: a maximal frequent itemset algorithm for transactional databases. In: Proceedings of the 17th International Conferences on Data Engineering. IEEE Computer Society, Washington, DC, pp 443–452Google Scholar
  10. Forina M (1991) Wine dataset. http://archive.ics.uci.edu/ml/datasets/wine. Accessed 5 Nov 2012
  11. Haglin DJ, Manning AM (2007) On minimal infrequent itemset mining. In: DMIN. CSREA Press, Las Vegas, pp 141–147Google Scholar
  12. Han J, Pei J, Yin Y, Mao R (2004) Mining frequent patterns without candidate generation: a frequent-pattern tree approach. In: Mannila H (ed) Data mining and knowledge discovery. Kluwer, New York, pp 53–87Google Scholar
  13. Koh YS, Rountree N (2005) Finding sporadic rules using apriori-inverse. In: PAKDD. Springer, New York, pp 97–106Google Scholar
  14. Koh YS, Rountree N, O’Keefe RA (2008) Mining interesting imperfectly sporadic rules. Knowl Inf Syst 14(2):179–196CrossRefGoogle Scholar
  15. Liu B, Hsu W, Ma Y (1999) Mining association rules with multiple minimum supports. In: KDD ’99: Proceedings of 5th ACM SIGKDD International Conferences on Knowledge Discovery and Data Mining. ACM, New York, pp 337–341Google Scholar
  16. Mannila H, Toivonen H, Verkamo I (1994) Efficient algorithms for discovering association rules. In: KDD ’94: Proceedings of the AAAI Workshop on Knowledge Discovery in Databases. AAAI Press, Seattle, pp 181–192Google Scholar
  17. Nakai K (1996a) Ecoli dataset. http://archive.ics.uci.edu/ml/datasets/ecoli. Accessed 5 Nov 2012
  18. Nakai K (1996b) Yeast dataset. http://archive.ics.uci.edu/ml/datasets/yeast. Accessed 5 Nov 2012
  19. Park JS, Chen M-S, Yu PS (1995) Efficient parallel data mining for association rules. In: CIKM ’95: Proceedings of 4th International Conference on Information and Knowledge Management. ACM, New York, pp 31–36Google Scholar
  20. Pasquier N, Bastide Y, Taouil R, Lakhal L (1999) Closed set based discovery of small covers for association rules. In: Proceedings 15emes Journees Bases de Donnees Avancees. BDA, pp 361–381Google Scholar
  21. Pei J, Han J, Lu H, Nishio S, Tang S, Yang D (2001) H-mine: hyper-structure mining of frequent patterns in large databases. In: ICDM ’01: Proceedings of the 2001 IEEE International Conferences on Data Mining. Washington, DC, pp 441–448Google Scholar
  22. Piatetsky-Shapiro G, Frawley WJ (eds) (1991) Knowledge discovery in databases. AAAI/MIT Press, CambridgeGoogle Scholar
  23. Savasere A, Omiecinski E, Navathe SB (1995) An efficient algorithm for mining association rules in large databases. In VLDB ’95: Proceedings of 21st International Conferences on Very Large Data Bases. Morgan Kaufmann, San Francisco, pp 432–444Google Scholar
  24. Shenoy P, Haritsa JR, Sudarshan S, Bhalotia G, Bawa M, Shah D (2000) Turbo-charging vertical mining of large databases. SIGMOD Rec 29(2):22–33CrossRefGoogle Scholar
  25. Song M, Rajasekaran S (2006) A transaction mapping algorithm for frequent itemsets mining. IEEE Trans Knowl Data Eng 18(4):472–481CrossRefGoogle Scholar
  26. Szathmary L, Napoli A, Kuznetsov SO (2007) ZART: a multifunctional itemset mining algorithm. In: Proceedings of the 5th International Conferences on Concept Lattices and Their Applications (CLA ’07). Montpellier, pp 26–37Google Scholar
  27. Szathmary L, Napoli A, Valtchev P (2007) Towards rare itemset mining. In: ICTAI ’07: Proceedings of 19th IEEE International Conferences on Tools with Artificial Intelligence. Washington, DC, pp 305–312Google Scholar
  28. Troiano L, Scibelli G, Birtolo C (2009) A fast algorithm for mining rare itemsets. In: ISDA’09, pp 1149–1155Google Scholar
  29. Tsang S, Koh YS, Dobbie G (2011) Rp-tree: rare pattern tree mining. In: Proceedings of CLA, pp 277–288Google Scholar
  30. Uno T, Asai T, Uchida Y, Arimura H (2003) Lcm: an efficient algorithm for enumerating frequent closed item sets. In: FIMI03: Proceedings of Workshop on Frequent Itemset Mining ImplementationsGoogle Scholar
  31. Uno T, Kiyomi M, Arimura H (2004) Lcm ver. 2: efficient mining algorithms for frequent/closed/maximal itemsets. In: FIMI ’04, Proceedings of the IEEE ICDM Workshop on Frequent Itemset Mining ImplementationsGoogle Scholar
  32. Uno T, Kiyomi M, Arimura H (2005) Lcm ver. 3: collaboration of array, bitmap and prefix tree for frequent itemset mining. In: Proceedings of 1st International Workshop on Open Source Data Mining: Frequent Pattern Mining Implementations, ACM, New York, pp 77–86Google Scholar
  33. Weiss GM (2004) Mining with rarity: a unifying framework. SIGKDD Explor Newsl 6(1):7–19CrossRefGoogle Scholar
  34. Yang G (2004) The complexity of mining maximal frequent itemsets and maximal frequent patterns. In: KDD ’04: Proceedings of 10th ACM SIGKDD International Conferences on Knowledge Discovery and Data Mining. New York, pp 344–353Google Scholar
  35. Yun H, Ha D, Hwang B, Ryu KH (2003) Mining association rules on significant rare data using relative support. J Syst Softw 67(3):181–191CrossRefGoogle Scholar
  36. Zaki MJ, Gouda K (2003) Fast vertical mining using diffsets. In: KDD ’03: Proceedings of 9th ACM SIGKDD International Conferences on Knowledge Discovery and Data Mining. New York, pp 326–335Google Scholar
  37. Zaki MJ, Parthasarathy S, Ogihara M, Li W (1997) New algorithms for fast discovery of association rules. Technical report, RochesterGoogle Scholar

Copyright information

© The Author(s) 2013

Authors and Affiliations

  1. 1.Department of EngineeringUniversity of SannioBeneventoItaly

Personalised recommendations