Evaluating a Rule Evaluation Support Method Based on Objective Rule Evaluation Indices

  • Hidenao Abe
  • Shusaku Tsumoto
  • Miho Ohsaki
  • Takahira Yamaguchi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3918)


In this paper, we present an evaluation of novel rule evaluation support method for post-processing of mined results with rule evaluation models based on objective indices. Post-processing of mined results is one of the key issues in a data mining process. However, it is difficult for human experts to evaluate many thousands of rules from a large dataset with noises completely. To reduce the costs of rule evaluation task, we have developed the rule evaluation support method with rule evaluation models, which are obtained with objective indices of mined classification rules and evaluations of a human expert for each rule. To evaluate performances of learning algorithms for constructing rule evaluation models, we have done a case study on the meningitis data mining as an actual problem. Furthermore, we have also evaluated our method on four rulesets from the four kinds of UCI datasets.


Training Dataset Human Expert Human Evaluation Rule Evaluation Objective Index 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ali, K., Manganaris, S., Srikant, R.: Partial Classification Using Association Rules. In: Proc. of Int. Conf. on Knowledge Discovery and Data Mining, KDD 1997, pp. 115–118 (1997)Google Scholar
  2. 2.
    Brin, S., Motwani, R., Ullman, J., Tsur, S.: Dynamic itemset counting and implication rules for market basket data. In: Proc. of ACM SIGMOD Int. Conf. on Management of Data, pp. 255–264 (1997)Google Scholar
  3. 3.
    Frank, E., Wang, Y., Inglis, S., Holmes, G., Witten, I.H.: Using model trees for classification. Machine Learning 32(1), 63–76 (1998)CrossRefMATHGoogle Scholar
  4. 4.
    Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: Proc. of the Fifteenth International Conference on Machine Learning, pp. 144–151 (1998)Google Scholar
  5. 5.
    Gago, P., Bento, C.: A Metric for Selection of the Most Promising Rules. In: Żytkow, J.M. (ed.) PKDD 1998. LNCS, vol. 1510, pp. 19–27. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications. In: Springer Series in Statistics, vol. 1, Springer, Heidelberg (1979)Google Scholar
  7. 7.
    Gray, B., Orlowska, M.E.: CCAIIA: Clustering Categorical Attributes into Interesting Association Rules. In: Wu, X., Kotagiri, R., Korb, K.B. (eds.) PAKDD 1998. LNCS, vol. 1394, pp. 132–143. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  8. 8.
    Hamilton, H.J., Shan, N., Ziarko, W.: Machine Learning of Credible Classifications. In: Proc. of Australian Conf. on Artificial Intelligence AI 1997, pp. 330–339 (1997)Google Scholar
  9. 9.
    Hatazawa, H., Negishi, N., Suyama, A., Tsumoto, S., Yamaguchi, T.: Knowledge Discovery Support from a Meningoencephalitis Database Using an Automatic Composition Tool for Inductive Applications. In: Terano, T., Chen, A.L.P. (eds.) PAKDD 2000. LNCS, vol. 1805, pp. 28–33. Springer, Heidelberg (2000)Google Scholar
  10. 10.
    Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine (1998), http://www.ics.uci.edu/mlearn/MLRepository.html Google Scholar
  11. 11.
    Hilderman, R.J., Hamilton, H.J.: Knowledge Discovery and Measure of Interest. Kluwer Academic Publishers, Dordrecht (2001)CrossRefMATHGoogle Scholar
  12. 12.
    Hinton, G.E.: Learning distributed representations of concepts. In: Morris, R.G.M. (ed.) Proceedings of 8th Annual Conference of the Cognitive Science Society, Amherest, MA (1986) (reprinted)Google Scholar
  13. 13.
    Holte, R.C.: Very simple classification rules perform well on most commonly used datasets. Machine Learning 11, 63–91 (1993)CrossRefMATHGoogle Scholar
  14. 14.
    Klösgen, W.: Explora: A Multipattern and Multistrategy Discovery Assistant. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 249–271. AAAI/MIT Press (1996)Google Scholar
  15. 15.
    Ohsaki, M., Kitaguchi, S., Kume, S., Yokoi, H., Yamaguchi, T.: Evaluation of Rule Interestingness Measures with a Clinical Dataset on Hepatitis. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS (LNAI), vol. 3202, pp. 362–373. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Piatetsky-Shapiro, G.: Discovery, Analysis and Presentation of Strong Rules. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 229–248. AAAI/MIT Press (1991)Google Scholar
  17. 17.
    Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  18. 18.
    Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  19. 19.
    Rijsbergen, C.: Information Retrieval, Butterworths, London, vol. 7 (1979), http://www.dcs.gla.ac.uk/Keith/Chapter.7/Ch.7.html
  20. 20.
    Smyth, P., Goodman, R.M.: Rule Induction using Information Theory. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 159–176. AAAI/MIT Press (1991)Google Scholar
  21. 21.
    Tan, P.N., Kumar, V., Srivastava, J.: Selecting the Right Interestingness Measure for Association Patterns. In: Proc. of Int. Conf. on Knowledge Discovery and Data Mining KDD 2002, pp. 32–41 (2002)Google Scholar
  22. 22.
    Witten, I.H., Frank, E.: DataMining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar
  23. 23.
    Yao, Y.Y., Zhong, N.: An Analysis of QuantitativeMeasures Associated with Rules. In: Zhong, N., Zhou, L. (eds.) PAKDD 1999. LNCS (LNAI), vol. 1574, pp. 479–488. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  24. 24.
    Zhong, N., Yao, Y.Y., Ohshima, M.: Peculiarity Oriented Multi-Database Mining. IEEE Trans. on Knowledge and Data Engineering 15(4), 952–960 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hidenao Abe
    • 1
  • Shusaku Tsumoto
    • 1
  • Miho Ohsaki
    • 2
  • Takahira Yamaguchi
    • 3
  1. 1.Department of Medical InformaticsShimane University, School of MedicineShimaneJapan
  2. 2.Faculty of EngineeringDoshisha UniversityKyotoJapan
  3. 3.Faculty of Science and TechnologyKeio UniversityKanagawaJapan

Personalised recommendations