Mining All Non-derivable Frequent Itemsets

  • Toon Calders
  • Bart Goethals
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2431)

Abstract

Recent studies on frequent itemset mining algorithms resulted in significant performance improvements. However, if the minimal support threshold is set too low, or the data is highly correlated, the number of frequent itemsets itself can be prohibitively large. To overcome this problem, recently several proposals have been made to construct a concise representation of the frequent itemsets, instead of mining all frequent itemsets. The main goal of this paper is to identify redundancies in the set of all frequent itemsets and to exploit these redundancies in order to reduce the result of a mining operation. We present deduction rules to derive tight bounds on the support of candidate itemsets. We show how the deduction rules allow for constructing a minimal representation for all frequent itemsets. We also present connections between our proposal and recent proposals for concise representations and we give the results of experiments on real-life datasets that show the effectiveness of the deduction rules. In fact, the experiments even show that in many cases, first mining the concise representation, and then creating the frequent itemsets from this representation outperforms existing frequent set mining algorithms.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    R. Agrawal, T. Imilienski, and A. Swami. Mining association rules between sets of items in large databases. In Proc. ACM SIGMOD Int. Conf. Management of Data, pages 207–216, Washington, D. C., 1993.Google Scholar
  2. 2.
    R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In Proc. VLDB Int. Conf. Very Large Data Bases, pages 487–499, Santiago, Chile, 1994.Google Scholar
  3. 3.
    Y. Bastide, R. Taouil, N. Pasquier, G. Stumme, and L. Lakhal. Mining frequent patterns with counting inference. ACM SIGKDD Explorations, 2(2):66–74, 2000.CrossRefGoogle Scholar
  4. 4.
    R. J. Bayardo. Efficiently mining long patterns from databases. In Proc. ACM SIGMOD Int. Conf. Management of Data, pages 85–93, Seattle, Washington, 1998.Google Scholar
  5. 5.
    J.-F. Boulicaut and A. Bykowski. Frequent closures as a concise representation for binary data mining. In Proc. P a KDD Pacific-Asia Conf. on Knowledge Discovery and Data Mining, pages 62–73, 2000.Google Scholar
  6. 6.
    J.-F. Boulicaut, A. Bykowski, and C. Rigotti. Approximation of frequency queries by means of free-sets. In Proc. PKDD Int. Conf. Principles of Data Mining and Knowledge Discovery, pages 75–85, 2000.Google Scholar
  7. 7.
    A. Bykowski and C. Rigotti. A condensed representation to find frequent patterns. In Proc. PODS Int. Conf. Principles of Database Systems, 2001.Google Scholar
  8. 8.
    T. Calders. Deducing bounds on the frequency of itemsets. In EDBT Workshop DTDM Database Techniques in Data Mining, 2002.Google Scholar
  9. 9.
    S. Hettich and S. D. Bay. The UCI KDD Archive. [http://kdd.ics.uci.edu]. Irvine, CA: University of California, Department of Information and Computer Science, 1999.Google Scholar
  10. 10.
    D. E. Knuth. Fundamental Algorithms. Addison-Wesley, Reading, Massachusetts, 1997.MATHGoogle Scholar
  11. 11.
    M. Kryszkiewicz. Concise representation of frequent patterns based on disjunctionfree generators. In Proc. IEEE Int. Conf. on Data Mining, pages 305–312, 2001.Google Scholar
  12. 12.
    H. Mannila and H. Toivonen. Multiple uses of frequent sets and condensed representations. In Proc. KDD Int. Conf. Knowledge Discovery in Databases, 1996.Google Scholar
  13. 13.
    N. Pasquier, Y. Bastide, R. Taouil, and L. Lakhal. Discovering frequent closed itemsets for association rules. In Proc. ICDT Int. Conf. Database Theory, pages 398–416, 1999.Google Scholar
  14. 14.
    J. Pei, J. Han, and R. Mao. Closet: An efficient algorithm for mining frequent closed itemsets. In W. Chen, J. F. Naughton, and P. A. Bernstein, editors, ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, Dallas, TX, 2000.Google Scholar
  15. 15.
    M. J. Zaki and C. Hsiao. ChARM: An efficient algorithm for closed association rule mining. In Technical Report 99-10, Computer Science, Rensselaer Polytechnic Institute, 1999.Google Scholar
  16. 16.
    Z. Zheng, R. Kohavi, and L. Mason. Real world performance of association rule algorithms. In Proc. KDD Int. Conf. Knowledge Discovery in Databases, pages 401–406. ACM Press, 2001.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Toon Calders
    • 1
  • Bart Goethals
    • 2
  1. 1.University of AntwerpBelgium
  2. 2.University of LimburgBelgium

Personalised recommendations