Developments on a Multi-objective Metaheuristic (MOMH) Algorithm for Finding Interesting Sets of Classification Rules

  • Beatriz de la Iglesia
  • Alan Reynolds
  • Vic J Rayward-Smith
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3410)


In this paper, we experiment with a combination of innovative approaches to rule induction to encourage the production of interesting sets of classification rules. These include multi-objective metaheuristics to induce the rules; measures of rule dissimilarity to encourage the production of dissimilar rules; and rule clustering algorithms to evaluate the results obtained.

Our previous implementation of NSGA-II for rule induction produces a set of cc-optimal rules (coverage-confidence optimal rules). Among the set of rules produced there may be rules that are very similar. We explore the concept of rule similarity and experiment with a number of modifications of the crowding distance to increasing the diversity of the partial classification rules produced by the multi-objective algorithm.


Association Rule Pareto Front Categorical Attribute Pareto Optimal Front Rule Induction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ali, S., Manganaris, K., Srikant, R.: Partial classification using association rules. In: Heckerman, D., Mannila, H., Pregibon, D., Uthurusamy, R. (eds.) Proceedings of the Third Int. Conf. on Knowledge Discovery and Data Mining, pp. 115–118. AAAI Press, Menlo Park (1997)Google Scholar
  2. 2.
    Bayardo, R., Agrawal, R.: Constraint based rule mining in large, dense databases. Data Mining and Knowledge Discovery Journal 4, 217–240 (2000)CrossRefGoogle Scholar
  3. 3.
    Bayardo, R., Agrawal, R.: Mining the most interesting rules. In: Proceedings of the 5th International Conference on Knowledge Discovery and Data Mining (KDD 1999), pp. 145–152. AAAI Press, Menlo Park (1999)CrossRefGoogle Scholar
  4. 4.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and regression trees. Wadsworth, Pacific Grove (1984)zbMATHGoogle Scholar
  5. 5.
    Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3, 261–284 (1989)Google Scholar
  6. 6.
    Cohen, W.: Fast effective rule induction. In: Proceedings of Twelfth International Conference on Machine Learning (ICML-1995), pp. 115–123. Morgan Kaufman, San Francisco (1995)Google Scholar
  7. 7.
    de la Iglesia, B., Richards, G., Philpott, M.S., Smith, V.J.R.: The application and effectiveness of a multi-objective metaheuristic algorithm for partial classification. European Journal of Operational Research (2004)( to appear)Google Scholar
  8. 8.
    Deb, K., Agrawal, S., Pratab, A., Meyarivan, T.: A Fast Elitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 849–858. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  9. 9.
    Freitas, A.A.: On objective measures of rule surprisingness. In: Żytkow, J.M. (ed.) PKDD 1998. LNCS, vol. 1510, Springer, Heidelberg (1998)Google Scholar
  10. 10.
    Freitas, A.A.: On rule interestingness measures. Knowledge-Based Systems Journal 12(5-6), 209–315 (1999)Google Scholar
  11. 11.
    Jaccard, P.: Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin de la Société Vaudoise de la Sciences Naturelles 37, 547–579 (1901)Google Scholar
  12. 12.
    Kaufman, L., Rousseuw, P.J.: Finding Groups in Data: An introduction to Cluster Analisys. Wiley Series in probability and mathematical statistics. John Wiley and Sons Inc., Chichester (1990)CrossRefGoogle Scholar
  13. 13.
    Merz, C.J., Murphy, P.M.: UCI repository of machine learning databases. Univ. California, Irvine (1998)Google Scholar
  14. 14.
    Piatetsky-Shapiro, G.: Discovery, Analysis, and Presentation of Strong Rules, ch. 13, pp. 229–248. AAAI/MIT Press (1991)Google Scholar
  15. 15.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
  16. 16.
    Reynolds, A.P., Richards, G., de la Iglesia, B., Smith, V.J.R.: Nugget clustering: A comparison of partitioning and hierarchical clustering algorithms. TBA (In preparation 2004)Google Scholar
  17. 17.
    Reynolds, A.P., Richards, G., Rayward-Smith, V.J.: The Application of K-medoids and PAM to the Clustering of Rules. In: Yang, Z.R., Yin, H., Everson, R.M. (eds.) IDEAL 2004. LNCS, vol. 3177, pp. 173–178. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Richards, G., Rayward-Smith, V.J.: The discovery of association rules from tabular databases comprising nominal and ordinal attributes. Intelligent Data Analysis 9(3) (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Beatriz de la Iglesia
    • 1
  • Alan Reynolds
    • 1
  • Vic J Rayward-Smith
    • 1
  1. 1.University of East AngliaNorwich, NorfolkUK

Personalised recommendations