Skip to main content
SpringerLink
Log in
Book cover

Foundations of Rule Learning pp 1–17Cite as

Machine Learning and Data Mining

  • Chapter
  • First Online:

Part of the book series: Cognitive Technologies ((COGTECH))

Abstract

Machine learning and data mining are research areas of computer science whose quick development is due to the advances in data analysis research, growth in thedatabase industry and the resultingmarket needs for methods that are capable of extracting valuable knowledge from large data stores.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   59.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   79.95
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   84.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    This chapter is partly based on Lavrač & Grobelnik (2003).

  2. 2.

    The dataset is adapted from the well-known dataset Quinlan (1986).

  3. 3.

    The preference for simpler models is a heuristic criterion known as Occam’s razor, which appears to work well in practice. It is often addressed in the literature on model selection, but its utility has been the subject of discussion (Domingos, 1999; Webb, 1996).

  4. 4.

    Prolog is a programming language, enabling knowledge representation in first-order logic (Lloyd, 1987; Sterling & Shapiro, 1994). We will briefly return to learning in first-order logic in Sect. 1.7; a systematic treatment of relational rule learning can be found in Chap. 5.

  5. 5.

    The rules are taken from Kralj Novak, Lavrač, and Webb (2009).

References

  • Adamo, J.-M. (2000). Data mining for association rules and sequential patterns: Sequential and parallel algorithms. New York: Springer.

    Google Scholar 

  • Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., & Verkamo, A. I. (1995). Fast discovery of association rules. In U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, & R. Uthurusamy (Eds.), Advances in knowledge discovery and data mining (pp. 307–328). Menlo Park, CA: AAAI.

    Google Scholar 

  • Agrawal, R., & Srikant, R. (2000). Privacy-preserving data mining. In W. Chen, J. F. Naughton, & P. A. Bernstein (Eds.), Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data (SIGMOD-2000), Dallas, TX (pp. 439–450). New York: ACM.

    Google Scholar 

  • Aha, D. W., Kibler, D., & Albert, M. K. (1991). Instance-based learning algorithms. Machine Learning, 6, 37–66.

    Google Scholar 

  • Bennett, K. P., Buja, A., Freund, W. S. Y., Schapire, R. E., Friedman, J., & Hastie, T. et al. (2008). Responses to Mease and Wyner (2008). Journal of Machine Learning Research, 9, 157–194.

    Google Scholar 

  • Berthold, M. R., Cebron, N., Dill, F., Gabriel, T. R., Kötter, T., & Meinl, T., et al. (2009). KNIME—The Konstanz information miner. Version 2.0 and beyond. SIGKDD Explorations, 11, 26–31.

    Google Scholar 

  • Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford, UK: Clarendon.

    Google Scholar 

  • Blockeel, H., & De Raedt, L. (1998). Top-down induction of first-order logical decision trees. Artificial Intelligence, 101(1–2), 285–297.

    Article  MathSciNet  MATH  Google Scholar 

  • Boström, H. (1995). Covering vs. divide-and-conquer for top-down induction of logic programs. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), Montréal, QC (pp. 1194–1200). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.

    MathSciNet  MATH  Google Scholar 

  • Breiman, L. (2001a). Random forests. Machine Learning, 45(1), 5–32.

    Article  MATH  Google Scholar 

  • Breiman, L. (2001b). Statistical modeling: The two cultures. Statistical Science, 16(3), 199–231. With comments by D. R. Cox, B. Efron, B. Hoadley, and E. Parzen, and a rejoinder by the author.

    Google Scholar 

  • Breiman, L., Friedman, J. H., Olshen, R., & Stone, C. (1984). Classification and regression trees. Pacific Grove, CA: Wadsworth & Brooks.

    MATH  Google Scholar 

  • Cendrowska, J. (1987). PRISM: An algorithm for inducing modular rules. International Journal of Man-Machine Studies, 27, 349–370.

    Article  MATH  Google Scholar 

  • Chapman, P., Clinton, J., Kerber, R., Khabaza, T., Reinartz, T., & Shearer, C., et al. (2000). Crisp-Dm 1.0: Step-by-step data mining guide. SPSS. Available from http://www.the-modeling-agency.com/CRISP-DM.pdf.

  • Clark, P., & Boswell, R. (1991). Rule induction with CN2: Some recent improvements. In Proceedings of the 5th European Working Session on Learning (EWSL-91), Porto, Portugal (pp. 151–163). Berlin, Germany: Springer.

    Google Scholar 

  • Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3(4), 261–283.

    Google Scholar 

  • Dasarathy, B. V. (Ed.). (1991). Nearest neighbor (NN) norms: NN pattern classification techniques. Los Alamitos, CA: IEEE.

    Google Scholar 

  • Demšar, J., Zupan, B., & Leban, G. (2004). Orange: From experimental machine learning to interactive data mining. White Paper, Faculty of Computer and Information Science, University of Ljubljana. Available from http://orange.biolab.si/.

  • De Raedt, L. (2008). Logical and relational learning. Berlin, Germany: Springer.

    Book  MATH  Google Scholar 

  • Domingos, P. (1999). The role of Occam’s Razor in knowledge discovery. Data Mining and Knowledge Discovery, 3(4), 409–425.

    Article  Google Scholar 

  • Duda, R. O., Hart, P. E., & Stork, D. G. (2000). Pattern classification (2nd ed.). New York: Wiley.

    Google Scholar 

  • Džeroski, S., & Lavrač, N. (Eds.). (2001). Relational data mining: Inductive logic programming for knowledge discovery in databases. Berlin, Germany/New York: Springer.

    MATH  Google Scholar 

  • Everitt, B., & Hothorn, T. (2006). A handbook of statistical analyses using R. Boca Raton, FL: Chapman & Hall/CRC.

    Book  Google Scholar 

  • Fayyad, U. M., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, 17(3), 37–54.

    Google Scholar 

  • Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., & Uthurusamy, R. (Eds.). (1995). Advances in knowledge discovery and data mining. Menlo Park, CA: AAAI.

    Google Scholar 

  • Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139.

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman, J. H. (1998). Data mining and statistics: What’s the connection? In Computing Science and Statistics: Proceedings of the 29th Symposium on the Interface, Houston, TX. Fairfax Station, VA: Interface Foundation of North America.

    Google Scholar 

  • Friedman, J. H., & Fisher, N. I. (1999). Bump hunting in high-dimensional data. Statistics and Computing, 9(2), 123–143.

    Article  Google Scholar 

  • Fürnkranz, J. (1997). Pruning algorithms for rule learning. Machine Learning, 27(2), 139–171.

    Article  Google Scholar 

  • Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA data mining software: An update. SIGKDD Explorations, 11(1), 10–18.

    Article  Google Scholar 

  • Han, J., & Kamber, M. (2001). Data mining: Concepts and techniques. San Francisco: Morgan Kaufmann Publishers.

    Google Scholar 

  • Hastie, T., Tibshirani, R., & Friedman, J. H. (2001). The elements of statistical learning. New York: Springer.

    MATH  Google Scholar 

  • Kotsiantis, S., Zaharakis, I., & Pintelas, P. (2006). Supervised machine learning: A review of classification techniques. Artificial Intelligence Review, 26, 159–190.

    Article  Google Scholar 

  • Kralj Novak, P., Lavrač, N., & Webb, G. I. (2009). Supervised descriptive rule discovery: A unifying survey of contrast set, emerging pattern and subgroup mining. Journal of Machine Learning Research, 10, 377–403.

    MATH  Google Scholar 

  • Kramer, S. (1996). Structural regression trees. In Proceedings of the 13th National Conference on Artificial Intelligence (AAAI-96) (pp. 812–819). Menlo Park, CA: AAAI.

    Google Scholar 

  • Langley, P. (1996). Elements of machine learning. San Francisco: Morgan Kaufmann.

    Google Scholar 

  • Lavrač, N., & Džeroski, S. (1994a). Inductive logic programming: Techniques and applications. New York: Ellis Horwood.

    MATH  Google Scholar 

  • Lavrač, N., Džeroski, S., & Grobelnik, M. (1991). Learning nonrecursive definitions of relations with LINUS. In Proceedings of the 5th European Working Session on Learning (EWSL-91), Porto, Portugal (pp. 265–281). Berlin, Germany: Springer.

    Google Scholar 

  • Lavrač, N., & Grobelnik, M. (2003). Data mining. In D. Mladenić, N. Lavrač, M. Bohanec, & S. Moyle (Eds.), Data mining and decision support: Integration and collaboration (pp. 3–14). Boston: Kluwer.

    Chapter  Google Scholar 

  • Lavrač, N., Kok, J., de Bruin, J., & Podpečan, V. (Eds.). (2008). Proceedings of the ECML-PKDD-08 Workshop on Third Generation Generation Data Mining: Towards Service-Oriented Knowledge Discovery (SoKD-08), Antwerp, Belgium.

    Google Scholar 

  • Lavrač, N., Podpečan, V., Kok, J., & de Bruin, J. (Eds.). (2009). Proceedings of the ECML-PKDD-09 Workshop on Service-Oriented Knowledge Discovery (SoKD-09), Bled, Slovenia.

    Google Scholar 

  • Lloyd, J. W. (1987). Foundations of logic programming (2nd extended ed.). Berlin, Germany: Springer.

    Google Scholar 

  • Mease, D., & Wyner, A. (2008). Evidence contrary to the statistical view of boosting. Journal of Machine Learning Research, 9, 131–156.

    Google Scholar 

  • Michalski, R. S. (1969). On the quasi-minimal solution of the covering problem. In Proceedings of the 5th International Symposium on Information Processing (FCIP-69), Bled, Yugoslavia (Switching circuits, Vol. A3, pp. 125–128).

    Google Scholar 

  • Michalski, R. S. (1980). Pattern recognition and rule-guided inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2, 349–361.

    Article  MATH  Google Scholar 

  • Michalski, R. S., Carbonell, J. G., & Mitchell, T. M. (Eds.). (1983). Machine learning: An artificial intelligence approach (Vol. I). Palo Alto, CA: Tioga.

    Google Scholar 

  • Michalski, R. S., Carbonell, J. G., & Mitchell, T. M. (Eds.). (1986). Machine learning: An artificial intelligence approach (Vol. II). Los Altos, CA: Morgan Kaufmann.

    Google Scholar 

  • Michalski, R. S., Mozetič, I., Hong, J., & Lavrač, N. (1986). The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In Proceedings of the 5th National Conference on Artificial Intelligence (AAAI-86), Philadelphia (pp. 1041–1045). Menlo Park, CA: AAAI.

    Google Scholar 

  • Michie, D., Spiegelhalter, D., & Taylor, C. C. (Eds.). (1994). Machine learning, neural and statistical classification. New York: Ellis Horwood.

    MATH  Google Scholar 

  • Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., & Euler, T. (2006). Yale: Rapid prototyping for complex data mining tasks. In L. Ungar, M. Craven, D. Gunopulos, & T. Eliassi-Rad (Eds.), KDD ’06: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia (pp. 935–940). New York: ACM.

    Google Scholar 

  • Mitchell, T. M. (1997). Machine learning. New York: McGraw Hill.

    MATH  Google Scholar 

  • Muggleton, S. H. (Ed.). (1992). Inductive logic programming. London: Academic.

    MATH  Google Scholar 

  • Pagallo, G., & Haussler, D. (1990). Boolean feature discovery in empirical learning. Machine Learning, 5, 71–99.

    Article  Google Scholar 

  • Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Pechter, R. (2009). What’s PMML and what’s new in PMML 4.0. SIGKDD Explorations, 11, 19–25.

    Google Scholar 

  • Piatetsky-Shapiro, G. & Frawley, W. J. (Eds.). (1991). Knowledge discovery in databases. Menlo Park, CA: MIT.

    Google Scholar 

  • Quinlan, J. R. (1979). Discovering rules by induction from large collections of examples. In D. Michie (Ed.), Expert systems in the micro electronic age (pp. 168–201). Edinburgh, UK: Edinburgh University Press.

    Google Scholar 

  • Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.

    Google Scholar 

  • Quinlan, J. R. (1987a). Generating production rules from decision trees. In Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI-87) (pp. 304–307). Los Altos, CA: Morgan Kaufmann.

    Google Scholar 

  • Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5, 239–266.

    Google Scholar 

  • Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Ripley, B. D. (1996). Pattern recognition and neural networks. Cambridge, MA/New York Cambridge University Press.

    MATH  Google Scholar 

  • Rivest, R. L. (1987). Learning decision lists. Machine Learning, 2, 229–246.

    MathSciNet  Google Scholar 

  • Rumelhart, D. E., & McClelland, J. L. (Eds.). (1986). Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: Foundations. Cambridge, MA: MIT.

    Google Scholar 

  • Schapire, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5), 1651–1686.

    Article  MathSciNet  MATH  Google Scholar 

  • Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond. Cambridge, MA: MIT.

    Google Scholar 

  • Sterling, L., & Shapiro, E. (1994). The art of prolog—Advanced programming techniques (2nd ed.). Cambridge, MA: MIT.

    MATH  Google Scholar 

  • Torgo, L. (2010). Data mining with R: Learning with case studies (Data mining and knowledge discovery series). Boca Raton: Chapman & Hall/CRC.

    Google Scholar 

  • Vapnik, V. (1995). The nature of statististical learning theory. New York: Springer.

    Google Scholar 

  • Watanabe, L., & Rendell, L. (1991). Learning structural decision trees from examples. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), Sydney, NSW (pp. 770–776). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Webb, G. I. (1995). OPUS: An efficient admissible algorithm for unordered search. Journal of Artificial Intelligence Research, 5, 431–465.

    Google Scholar 

  • Webb, G. I. (1996). Further experimental evidence aganst the utility of Occam’s razor. Journal of Artificial Intelligence Research, 4, 397–417.

    MATH  Google Scholar 

  • Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques with Java implementations (2nd ed.). Amsterdam/Boston: Morgan Kaufmann Publishers.

    Google Scholar 

  • Zhang, C., & Zhang, S. (2002). Association rule mining: Models and algorithms. Berlin, Germany/New York: Springer.

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. FB Informatik, TU Darmstadt, Darmstadt, Germany

    Johannes Fürnkranz

  2. Rudjer Bošković Institute, Zagreb, Croatia

    Dragan Gamberger

  3. Department of Knowledge Technologies, Jožef Stefan Institute, Ljubljana, Slovenia

    Nada Lavrač

Authors
  1. Johannes Fürnkranz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dragan Gamberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nada Lavrač
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Fürnkranz, J., Gamberger, D., Lavrač, N. (2012). Machine Learning and Data Mining. In: Foundations of Rule Learning. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75197-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-75197-7_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-75196-0

  • Online ISBN: 978-3-540-75197-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation