Rule Quality Measure-Based Induction of Unordered Sets of Regression Rules

  • Marek Sikora
  • Adam Skowron
  • Łukasz Wróbel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7557)


This paper presents the algorithm for induction of unordered sets of regression rules. It uses sequential covering strategy and dynamic reduction to classification approach. The main focus is put on quality measures which control the process of rule induction. We examined the effectiveness of nine quality measures. Moreover, we propose and compare three schemes of the prediction of target attribute value of examples covered by more than one rule. We also show rule filtration algorithm for the reduction of the number of generated rules. All experiments were carried out on 35 benchmark datasets.


rule-based regression rule quality measures rule induction rule filtration prediction conflicts resolving 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kryszkiewicz, M.: Fast Discovery of Representative Association Rules. In: Polkowski, L., Skowron, A. (eds.) RSCTC 1998. LNCS (LNAI), vol. 1424, pp. 214–221. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  2. 2.
    Fürnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13, 3–54 (1999)zbMATHCrossRefGoogle Scholar
  3. 3.
    Stefanowski, J., Vanderpooten, D.: Induction of decision rules in classification and discovery-oriented perspectives. Int. J. Intell. Syst. 16(1), 13–27 (2001)zbMATHCrossRefGoogle Scholar
  4. 4.
    Grzymala-Busse, J.W., Ziarko, W.: Data mining based on rough sets. In: Data Mining: Opportunities and Challenges, pp. 142–173. IGI Global (2003)Google Scholar
  5. 5.
    Wojtusiak, J., Michalski, R.S., Kaufman, K.A., Pietrzykowski, J.: The AQ21 natural induction program for pattern discovery: Initial version and its novel features. In: Proceedings of the 18th IEEE International Conference on Tools with Artificial Intelligence, pp. 13–15 (2006)Google Scholar
  6. 6.
    Fawcett, T.: PRIE: a system for generating rulelists to maximize ROC performance. Data Mining and Knowledge Discovery 17(2), 207–224 (2008)MathSciNetCrossRefGoogle Scholar
  7. 7.
    An, A., Cercone, N.: Rule quality measures for rule induction systems: Description and evaluation. Computational Intelligence 17(3), 409–424 (2001)CrossRefGoogle Scholar
  8. 8.
    Bruha, I., Tkadlec, J.: Rule quality for multiple-rule classifier: Empirical expertise and theoretical methodology. Intell. Data Anal. 7(2), 99–124 (2003)zbMATHGoogle Scholar
  9. 9.
    Brzezinska, I., Greco, S., Slowinski, R.: Mining pareto-optimal rules with respect to support and confirmation or support and anti-support. Eng. Appl. Artif. Intell. 20(5), 587–600 (2007)CrossRefGoogle Scholar
  10. 10.
    Janssen, F., Fürnkranz, J.: On the quest for optimal rule learning heuristics. Machine Learning 78(3), 343–379 (2010)CrossRefGoogle Scholar
  11. 11.
    Sikora, M.: Rule Quality Measures in Creation and Reduction of Data Rule Models. In: Greco, S., Hata, Y., Hirano, S., Inuiguchi, M., Miyamoto, S., Nguyen, H.S., Słowiński, R. (eds.) RSCTC 2006. LNCS (LNAI), vol. 4259, pp. 716–725. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Sikora, M.: Decision Rule-Based Data Models Using TRS and NetTRS – Methods and Algorithms. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets XI. LNCS, vol. 5946, pp. 130–160. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Sikora, M., Wróbel, Ł.: Data-Driven Adaptive Selection of Rules Quality Measures for Improving the Rules Induction Algorithm. In: Kuznetsov, S.O., Ślęzak, D., Hepting, D.H., Mirkin, B.G. (eds.) RSFDGrC 2011. LNCS, vol. 6743, pp. 278–285. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Sikora, M., Wróbel, L.: Data-driven adaptive selection of rule quality measures for improving rule induction and filtration algorithms. International Journal of General Systems (to appear, 2012)Google Scholar
  15. 15.
    Torgo, L., Gama, J.: Regression by Classification. In: Borges, D.L., Kaestner, C.A.A. (eds.) SBIA 1996. LNCS, vol. 1159, pp. 51–60. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  16. 16.
    Quinlan, R.J.: Learning with continuous classes. In: 5th Australian Joint Conference on Artificial Intelligence, pp. 343–348. World Scientific, Singapore (1992)Google Scholar
  17. 17.
    Holmes, G., Hall, M., Frank, E.: Generating Rule Sets from Model Trees. In: Foo, N.Y. (ed.) AI 1999. LNCS, vol. 1747, pp. 1–12. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  18. 18.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth (1984)Google Scholar
  19. 19.
    Friedman, J.H., Popescu, B.E.: Predictive learning via rule ensembles. The Annals of Applied Statistics, 916–954 (2008)Google Scholar
  20. 20.
    Dembczyński, K., Kotłowski, W., Słowiński, R.: Solving Regression by Learning an Ensemble of Decision Rules. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 533–544. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  21. 21.
    Demšar, D.: Obravnavanje numeričnih problemov z induktivnim logičnim programiranjem. Master’s thesis, Faculty of Computer and Information Science, University of Ljubljana, Slovenia, 59, 64 (1999) (in Slovene)Google Scholar
  22. 22.
    Janssen, F., Fürnkranz, J.: Heuristic rule-based regression via dynamic reduction to classification. In: Walsh, T. (ed.) Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), pp. 1330–1335 (2011)Google Scholar
  23. 23.
    Ženko, B., Džeroski, S., Struyf, J.: Learning Predictive Clustering Rules. In: Bonchi, F., Boulicaut, J.-F. (eds.) KDID 2005. LNCS, vol. 3933, pp. 234–250. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  24. 24.
    Witten, I.H., Frank, E., Hall, M.A.: Data Mining: Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufmann, Amsterdam (2011)Google Scholar
  25. 25.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Garcia, S., Herrera, F.: An Extension on ”Statistical Comparisons of Classifiers over Multiple Data Sets” for all Pairwise Comparisons. Journal of Machine Learning Research 9, 2677–2694 (2008)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Marek Sikora
    • 1
    • 2
  • Adam Skowron
    • 1
  • Łukasz Wróbel
    • 1
  1. 1.Institute of Computer ScienceSilesian University of TechnologyGliwicePoland
  2. 2.Institute of Innovative Technologies EMAGKatowicePoland

Personalised recommendations