Evaluating Learning Algorithms for a Rule Evaluation Support Method Based on Objective Rule Evaluation Indices

  • Hidenao Abe
  • Shusaku Tsumoto
  • Miho Ohsaki
  • Takahira Yamaguchi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4203)


In this paper, we present an evaluation of learning algorithms of a novel rule evaluation support method for post-processing of mined results with rule evaluation models based on objective indices. Post-processing of mined results is one of the key processes in a data mining process. However, it is difficult for human experts to completely evaluate several thousands of rules from a large dataset with noises. To reduce the costs in such rule evaluation task, we have developed the rule evaluation support method with rule evaluation models which learn from a dataset. This dataset comprises objective indices for mined classification rules and evaluations by a human expert for each rule. To evaluate performances of learning algorithms for constructing the rule evaluation models, we have done a case study on the meningitis data mining as an actual problem. Furthermore, we have also evaluated our method with five rule sets obtained from five UCI datasets.


Training Dataset Human Expert Rule Evaluation Objective Index Evaluation Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ali, K., Manganaris, S., Srikant, R.: Partial Classification Using Association Rules. In: Proc. of Int. Conf. on Knowledge Discovery and Data Mining KDD-1997, pp. 115–118 (1997)Google Scholar
  2. 2.
    Brin, S., Motwani, R., Ullman, J., Tsur, S.: Dynamic itemset counting and implication rules for market basket data. In: Proc. of ACM SIGMOD Int. Conf. on Management of Data, pp. 255–264 (1997)Google Scholar
  3. 3.
    Frank, E., Wang, Y., Inglis, S., Holmes, G., Witten, I.H.: Using model trees for classification. Machine Learning 32(1), 63–76 (1998)MATHCrossRefGoogle Scholar
  4. 4.
    Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: Proc. of the Fifteenth International Conference on Machine Learning, pp. 144–151 (1998)Google Scholar
  5. 5.
    Gago, P., Bento, C.: A Metric for Selection of the Most Promising Rules. In: Żytkow, J.M. (ed.) PKDD 1998. LNCS, vol. 1510, pp. 19–27. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications. Springer Series in Statistics, vol. 1. Springer, Heidelberg (1979)MATHGoogle Scholar
  7. 7.
    Gray, B., Orlowska, M.E.: CCAIIA: Clustering Categorical Attributes into Interesting Association Rules. In: Wu, X., Kotagiri, R., Korb, K.B. (eds.) PAKDD 1998. LNCS, vol. 1394, pp. 132–143. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  8. 8.
    Hamilton, H.J., Shan, N., Ziarko, W.: Machine Learning of Credible Classifications. In: Proc. of Australian Conf. on Artificial Intelligence AI-1997, pp. 330–339 (1997)Google Scholar
  9. 9.
    Hatazawa, H., Negishi, N., Suyama, A., Tsumoto, S., Yamaguchi, T.: Knowledge Discovery Support from a Meningoencephalitis Database Using an Automatic Composition Tool for Inductive Applications. In: Terano, T., Chen, A.L.P. (eds.) PAKDD 2000. LNCS, vol. 1805, pp. 28–33. Springer, Heidelberg (2000)Google Scholar
  10. 10.
    Hettich, S., Blake, C. L., Merz, C.J.: UCI Repository of machine learning databases, University of California, Department of Information and Computer Science, Irvine, CA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  11. 11.
    Hilderman, R.J., Hamilton, H.J.: Knowledge Discovery and Measure of Interest. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  12. 12.
    Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of 8th Annual Conference of the Cognitive Science Society, Amherest, MA (1986) REprinted in R.G.M.Morris (ed.)Google Scholar
  13. 13.
    Holte, R.C.: Very simple classification rules perform well on most commonly used datasets. Machine Learning 11, 63–91 (1993)MATHCrossRefGoogle Scholar
  14. 14.
    Klösgen, W.: Explora: A Multipattern and Multistrategy Discovery Assistant. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 249–271. AAAI/MIT Press, California (1996)Google Scholar
  15. 15.
    Ohsaki, M., Kitaguchi, S., Kume, S., Yokoi, H., Yamaguchi, T.: Evaluation of Rule Interestingness Measures with a Clinical Dataset on Hepatitis. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS (LNAI), vol. 3202, pp. 362–373. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Piatetsky-Shapiro, G.: Discovery, Analysis and Presentation of Strong Rules. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 229–248. AAAI/MIT Press, Cambridge (1991)Google Scholar
  17. 17.
    Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  18. 18.
    Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco (1993)Google Scholar
  19. 19.
    Rijsbergen, C.: Information Retrieval, ch. 7, Butterworths, London (1979), http://www.dcs.gla.ac.uk/Keith/Chapter.7/Ch.7.html
  20. 20.
    Smyth, P., Goodman, R.M.: Rule Induction using Information Theory. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 159–176. AAAI/MIT Press, Cambridge (1991)Google Scholar
  21. 21.
    Tan, P.N., Kumar, V., Srivastava, J.: Selecting the Right Interestingness Measure for Association Patterns. In: Proc. of Int. Conf. on Knowledge Discovery and Data Mining KDD-2002, pp. 32–41 (2002)Google Scholar
  22. 22.
    Witten, I.H., Frank, E.: DataMining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar
  23. 23.
    Yao, Y.Y., Zhong, N.: An Analysis of Quantitative Measures Associated with Rules. In: Zhong, N., Zhou, L. (eds.) PAKDD 1999. LNCS (LNAI), vol. 1574, pp. 479–488. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  24. 24.
    Zhong, N., Yao, Y.Y., Ohshima, M.: Peculiarity Oriented Multi-Database Mining. IEEE Trans. on Knowledge and Data Engineering 15(4), 952–960 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hidenao Abe
    • 1
  • Shusaku Tsumoto
    • 1
  • Miho Ohsaki
    • 2
  • Takahira Yamaguchi
    • 3
  1. 1.Department of Medical InformaticsShimane University, School of MedicineIzumo, ShimaneJapan
  2. 2.Faculty of EngineeringDoshisha UniversityKyo-Tanabe, KyotoJapan
  3. 3.Faculty of Science and TechnologyKeio UniversityKohoku Yokohama, KanagawaJapan

Personalised recommendations