Machine Learning

, Volume 54, Issue 1, pp 67–92 | Cite as

ART: A Hybrid Classification Model

  • Fernando Berzal
  • Juan-Carlos Cubero
  • Daniel Sánchez
  • José María Serrano

Abstract

This paper presents a new family of decision list induction algorithms based on ideas from the association rule mining context. ART, which stands for ‘Association Rule Tree’, builds decision lists that can be viewed as degenerate, polythetic decision trees. Our method is a generalized “Separate and Conquer” algorithm suitable for Data Mining applications because it makes use of efficient and scalable association rule mining techniques.

supervised learning classification decision lists decision trees association rules Data Mining 

References

  1. Aggarwal, C. C., Sun, Z., & Yu, P. S. (1998). Online algorithms for finding profile association rules. In Proceedings of the 1998 ACM CIKM 7th International Conference on Information and Knowledge Management (pp. 86–95). Bethesda, Maryland, USA.Google Scholar
  2. Aggarwal, C. C., & Yu, P. S. (1998a). A new framework for itemset generation. In Proceedings of the Seventeenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (pp. 18–24). Seattle, Washington.Google Scholar
  3. Aggarwal, C. C., & Yu, P. S. (1998b). Mining large itemsets for association rules. In Bulletin of the IEEE Computer Society Technical Committee on Data Engineering.Google Scholar
  4. Agrawal, R., Imielinski, T., & Swami, A. (1993). Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data (pp. 207–216). Washington, D.C.Google Scholar
  5. Agrawal, R., & Srikant, R. (1994). Fast algorithms for mining association rules. In VLDB '94, Proceedings of 20th International Conference on Very Large Data Bases (pp. 487–499). Santiago de Chile, Chile.Google Scholar
  6. Ali, K., Manganaris, S., & Srikant, R. (1997). Partial classification using association rules. In Proceedings of the 3rd International Conference on Knowledge Discovery in Databases and Data Mining (pp. 115–118). Newport Beach, California, USA.Google Scholar
  7. Aumann, Y., & Lindell, Y. (1999). A statistical theory for quantitative association rules. In Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 261–270). San Diego, California, USA.Google Scholar
  8. Berzal, F., Cubero, J. C., Marín, N., & Serrano, J. M. (2001). TBAR: An efficient method for association rule mining in relational databases. Data & Knowledge Engineering, 37:1, 47–64.Google Scholar
  9. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth, California, USA.Google Scholar
  10. Brin, S., Motwani, R., Ullman, J. D., & Tsur, S. (1997). Dynamic itemset counting and implication rules for market basket data. In Proceedings of the ACM SIGMOD International Conference on Management of Data (pp. 255–264). Tucson, Arizona, USA.Google Scholar
  11. Chan, P. K. (1989). Inductive learning with BCT. In Proceedings of the 6th International Workshop on Machine Learning (pp. 104–108). Ithaca, NY.Google Scholar
  12. Clark, P., & Boswell, R. (1991). Rule induction with CN2: Some recent improvements. In Y. Kodratoff (ed.), Machine Learning—EWSL-91 (pp. 151–163). Berlin: Springer-Verlag.Google Scholar
  13. Clark, P., & Nibblett, T. (1989). The CN2 induction algorithm. Machine Learning Journal (Kluwer Academic Publishers), 3:4, 261–183.Google Scholar
  14. Cohen, W. (1995). Fast effective rule induction. In Proc. 12th International Conference on Machine Learning (pp. 115–123). Morgan Kaufmann.Google Scholar
  15. Domingos, P. (1996). Linear-time rule induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96) (pp. 96–101).Google Scholar
  16. Domingos, P. (1998). Occam's two Razors: The sharp and the blunt. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining (KDD-98) (pp. 37–43). New York City, USA.Google Scholar
  17. Domingos, P. (1999). The role of Occam's Razor in knowledge discovery. Data Mining and Knowledge Discovery, 3:4, 409–425.Google Scholar
  18. Dong, G., & Li, J. (1999). Efficient mining of emerging patterns: Discovering trends and differences. In Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 43–52). San Diego, CA USA.Google Scholar
  19. Dong, G., Zhang, X., Wong, L., & Li, J. (1999). CAEP: Classification by aggregating emerging patterns. In Proceedings of the Second International Conference on Discovery Science (pp. 30–42). Tokyo, Japan.Google Scholar
  20. Elder IV, J. F. (1995). Heuristic Search for Model Structure: The Benefits of Restraining Greed. AI & Statistics-95, Ft. Lauderdale, Florida, pp. 199–210.Google Scholar
  21. Freitas, A. A. (2000). Understanding the crucial differences between classification and discovery of association rules—A position paper. SIGKDD Explorations, 2:1, 65–69.Google Scholar
  22. Fürnkranz, J., & Widmer, F. (1994). Incremental reduced error pruning. In Machine Learning: Proceedings of the 11th Annual Conference. New Brunswick, New Jersey: Morgan Kaufmann.Google Scholar
  23. Gehrke, J., Ganti, V., Ramakrishnan, R., & Loh, W.-Y. (1999a). BOAT—optimistic decision tree construction. In Proceedings of the 1999 ACM SIGMOD international conference on Management of Data (pp. 169–180). Philadelphia, PA, USA.Google Scholar
  24. Gehrke, J., Loh, W.-Y., & Ramakrishnan, R. (1999b). Classification and regression: Money can grow on trees. In Tutorial Notes for ACM SIGKDD 1999 International Conference on Knowledge Discovery and Data Mining (pp. 1–73). San Diego, California, USA.Google Scholar
  25. Gehrke, J, Ramakrishnan, R., & Ganti, V. (2000). RainForest-A framework for fast decision tree construction of large datasets. Data Mining and Knowledge Discovery, 4:2/3, 127–162.Google Scholar
  26. Giordana, A., & Neri, F. (1996). Search-intensive concept induction. Evolutionary Computation, 3:4, 375–416.Google Scholar
  27. Han, J., Pei, J., & Yin, Y. (2000). Mining frequent patterns without candidate generation. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data (pp. 1–12). Dallas, TX, USA.Google Scholar
  28. Han, J. L., & Plank, A. W. (1996). Background for association rules and cost estimate of selected mining algorithms. In CIKM '96, Proceedings of the Fifth International Conference on Information and Knowledge Management (pp. 73–80). Rockville, Maryland, USA.Google Scholar
  29. Hidber, C. (1999). Online association rule mining. In Proceedings of the 1999 ACM SIGMOD international conference on Management of Data (pp. 145–156). Philadelphia, PA, USA.Google Scholar
  30. Hipp, J., Güntzer, U., & Nakhaeizadeh, G. (2000). Algorithms for association rule mining—A general survey and comparison. SIGKDD Explorations, 2:1, 58–64.Google Scholar
  31. Houtsma, M., & Swami, A. (1993). Set-oriented mining for association rules. IBM Research Report RJ9567, IBM Almaden Research Center, San Jose, California.Google Scholar
  32. Joshi, M. V., Agarwal, R. C., & Kumar, V. (2001). Mining needles in a haystack: Classifying rare classes via two-phase rule induction. In Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data (pp. 91–101). Santa Barbara, California.Google Scholar
  33. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., & Griswold, W. G. (2001). Getting started with Aspect. J. Communications of the ACM, 44:10, 59–65.Google Scholar
  34. Liu, B., Hsu, W., & Ma, Y. (1998). Integrating classification and association rule mining. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining (KDD-98) (pp. 80–86). New York City, USA.Google Scholar
  35. Liu, B., Hu, M., & Hsu, W. (2000a) Intuitive representation of decision trees using general rules and exceptions. In Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI-2000). Austin, Texas.Google Scholar
  36. Liu, B., Hu, M., & Hsu, W. (2000b) Multi-level organization and summarization of the discovered rule. In Proceedings of the sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 208–217). Boston, MA, USA.Google Scholar
  37. Liu, B., Ma, Y., & Wong, C. K. (2000c). Improving an association rule based classifier. In Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD-2000). Lyon, France.Google Scholar
  38. Loh, W.-Y., & Shih, Y.-S. (1997). Split selection methods for classification trees. Statistica Sinica, 7, 815–840.Google Scholar
  39. Mehta, M., Agrawal, R., & Rissanen, J. (1996). SLIQ: A fast scalable classifier for data mining. Advances in Database Technology—Proceedings of the Fifth International Conference on Extending Database Technology (EDBT'96) (pp. 18–32). Avignon, France.Google Scholar
  40. Meretakis, D., & Wüthrich, B. (1999). Extending naïve Bayes classifiers using long itemsets. In Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 165–174). San Diego, CA, USA.Google Scholar
  41. Miller, R. J., & Yang, Y. (1997). Association rules over interval data. In Proceedings of the ACM SIGMOD Conference on Management of Data (pp. 452–461). Tucson, AZ, USA.Google Scholar
  42. Pagallo, G., & Haussler, D. (1990). Boolean feature discovery in empirical learning. Machine Learning, 5, 71–99.Google Scholar
  43. Park, J. S., Chen, M. S., & Yu, P. S. (1995). An effective hash-based algorithm for mining association rules. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data (pp. 175–186). San Jose, California.Google Scholar
  44. Park, J. S., Chen, M. S., & Yu, P. S. (1997). Using a hash-based method with transaction trimming for mining association rules. IEEE Transactions on Knowledge and Data Engineering, 9:5, 813–825.Google Scholar
  45. Quinlan, J. R. (1986a). Induction on decision trees. Machine Learning, 1, 81–106.Google Scholar
  46. Quinlan, J. R. (1986b). Learning Decision Tree Classifiers. ACM Computing Surveys, 28:1, 71–72.Google Scholar
  47. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.Google Scholar
  48. Rivest, R. L. (1987). Learning decision lists. Machine Learning Journal, 2:3, 229–246.Google Scholar
  49. Segal, R., & Etzioni, O. (1994). Learning decision lists using homogeneous rules. AAAI 1994, 12th National Conference on Artificial Intelligence (pp. 619–625). Seattle, WA, USA.Google Scholar
  50. Sestito, S., & Dillon, T. S. (1994). Automated Knowledge Acquisition. Sydney: Prentice Hall.Google Scholar
  51. Srikant, R., & Agrawal, R. (1996). Mining quantitative association rules in large relational tables. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data (pp. 1–12). Montreal, Quebec, Canada.Google Scholar
  52. Rastogi, R., & Shim, K. (1998). PUBLIC: A decision tree classifier that integrates building and pruning. In VLDB'98, Proceedings of 24th International Conference on Very Large Data Bases (pp. 404–415). New York City, New York, USA.Google Scholar
  53. Shafer, J. C., Agrawal, R., & Mehta, M. (1996). SPRINT: A scalable parallel classifier for data mining. In VLDB'96, Proceedings of 22nd International Conference on Very Large Data Bases (pp. 544–555). Mumbai (Bombay), India.Google Scholar
  54. Wang, K., Zhou, S., & He, Y. (2000). Growing decision trees on support-less association rules. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 265–269). Boston, MA, USA.Google Scholar
  55. Wang, K., Zhou, S., & Liew, S. C. (1999). Building hierarchical classifiers using class proximity. In VLDB'99, Proceedings of 25th International Conference on Very Large Data Bases (pp. 363–374). Edinburgh, Scotland, UK.Google Scholar
  56. Zheng, Z. (2000). Constructing X-of-N attributes for decision tree learning. Machine Learning, 40:1, 35–75.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Fernando Berzal
    • 1
  • Juan-Carlos Cubero
    • 1
  • Daniel Sánchez
    • 1
  • José María Serrano
    • 1
  1. 1.Dept. Computer Science and AIUniversity of GranadaSpain

Personalised recommendations