Comparison of Some Classification Algorithms Based on Deterministic and Nondeterministic Decision Rules

  • Paweł Delimata
  • Barbara Marszał-Paszek
  • Mikhail Moshkov
  • Piotr Paszek
  • Andrzej Skowron
  • Zbigniew Suraj
Chapter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6190)

Abstract

We discuss two, in a sense extreme, kinds of nondeterministic rules in decision tables. The first kind of rules, called as inhibitory rules, are blocking only one decision value (i.e., they have all but one decisions from all possible decisions on their right hand sides). Contrary to this, any rule of the second kind, called as a bounded nondeterministic rule, can have on the right hand side only a few decisions. We show that both kinds of rules can be used for improving the quality of classification. In the paper, two lazy classification algorithms of polynomial time complexity are considered. These algorithms are based on deterministic and inhibitory decision rules, but the direct generation of rules is not required. Instead of this, for any new object the considered algorithms extract from a given decision table efficiently some information about the set of rules. Next, this information is used by a decision-making procedure. The reported results of experiments show that the algorithms based on inhibitory decision rules are often better than those based on deterministic decision rules. We also present an application of bounded nondeterministic rules in construction of rule based classifiers. We include the results of experiments showing that by combining rule based classifiers based on minimal decision rules with bounded nondeterministic rules having confidence close to 1 and sufficiently large support, it is possible to improve the classification quality.

Keywords

rough sets classification decision tables deterministic decision rules inhibitory decision rules lazy classification algorithms (classifiers) nondeterministic decision rules rule based classifiers 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aha, D.W. (ed.): Lazy Learning. Kluwer Academic Publishers, Dordrecht (1997)MATHGoogle Scholar
  2. 2.
    Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules Between Sets of Items in Large Databases. In: Buneman, P., Jajodia, S. (eds.) Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, D.C., May 26-28, pp. 207–216. ACM Press, New York (1993)CrossRefGoogle Scholar
  3. 3.
    Bazan, J.G.: Discovery of Decision Rules by Matching New Objects Against Data Tables. In: Polkowski, L., Skowron, A. (eds.) RSCTC 1998. LNCS (LNAI), vol. 1424, pp. 521–528. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  4. 4.
    Bazan, J.G.: A Comparison of Dynamic and Non-Dynamic Rough Set Methods for Extracting Laws from Decision Table. In: Polkowski, L., Skowron, A. (eds.) Rough Sets in Knowledge Discovery, pp. 321–365. Physica-Verlag, Heidelberg (1998)Google Scholar
  5. 5.
    Bazan, J.G.: Methods of Approximate Reasoning for Synthesis of Decision Algorithms. Ph.D. Thesis. Warsaw University (1998) (in Polish)Google Scholar
  6. 6.
    Bazan, J., Skowron, A., Swiniarski, R.: Rough Sets and Vague Concept Approximation: From Sample Approximation to Adaptive Learning. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets V. LNCS, vol. 4100, pp. 39–62. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Bazan, J.G., Szczuka, M.S., Wojna, A., Wojnarski, M.: On the Evolution of Rough Set Exploration System. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 592–601. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Bramer, M.A.: Automatic Induction of Classification Rules from Examples Using NPRISM. In: Research and Development in Intelligent Systems XVI, pp. 99–121. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  9. 9.
    Cendrowska, J.: PRISM: An Agorithm for Inducing Modular Rules. International Journal of Man-Machine Studies 27(4), 349–370 (1987)CrossRefMATHGoogle Scholar
  10. 10.
    Data Mining Exploration System (Software), http://www.univ.rzeszow.pl/rspn
  11. 11.
    Delimata, P., Moshkov, M., Skowron, A., Suraj, Z.: Two Families of Classification Algorithms. In: An, A., Stefanowski, J., Ramanna, S., Butz, C.J., Pedrycz, W., Wang, G. (eds.) RSFDGrC 2007. LNCS (LNAI), vol. 4482, pp. 297–304. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Delimata, P., Moshkov, M.J., Skowron, A., Suraj, Z.: Comparison of Lazy Classification Algorithms Based on Deterministic and Inhibitory Decision Rules. In: Wang, G., Li, T., Grzymala-Busse, J.W., Miao, D., Skowron, A., Yao, Y. (eds.) RSKT 2008. LNCS (LNAI), vol. 5009, pp. 55–62. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Delimata, P., Moshkov, M.J., Skowron, A., Suraj, Z.: Inhibitory Rules in Data Analysis: A Rough Set Approach. In: Studies in Computational Intelligence, vol. 163. Springer, Heidelberg (2009)Google Scholar
  14. 14.
    Grzymala-Busse, J.W.: LERS - A Data Mining System. In: Maimon, O., Rokach, L. (eds.) The Data Mining and Knowledge Discovery Handbook, pp. 1347–1351. Springer, New York (2005)CrossRefGoogle Scholar
  15. 15.
    Holte, R.: Very Simple Classification Rules Perform Well on Most Commonly Used Data Sets. Machine Learning 11, 63–91 (1993)CrossRefMATHGoogle Scholar
  16. 16.
    Marszał-Paszek, B., Paszek, P.: Minimal Templates and Knowledge Discovery. In: Kryszkiewicz, M., Peters, J.F., Rybiński, H., Skowron, A. (eds.) RSEISP 2007. LNCS (LNAI), vol. 4585, pp. 411–416. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
  18. 18.
    Moshkov, M., Skowron, A., Suraj, Z.: On Maximal Consistent Extensions of Information Systems. In: Proceedings of the Conference Decision Support Systems, Zakopane, Poland, December 2006, vol. 1, pp. 199–206. University of Silesia, Katowice (2007)Google Scholar
  19. 19.
    Moshkov, M., Skowron, A., Suraj, Z.: Maximal Consistent Extensions of Information Systems Relative to Their Theories. Information Sciences 178, 2600–2620 (2008)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    UCI Repository of Machine Learning Databases, University of California, Irvine, http://www.ics.uci.edu/~mlearn/MLRepository.html
  21. 21.
    Nguyen, H.S.: Scalable Classification Method Based on Rough Sets. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 433–440. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  22. 22.
    Nguyen, T.T.: Handwritten Digit Recognition Using Adaptive Classifier Construction Techniques. In: Pal, S.K., Polkowski, L., Skowron, A. (eds.) Rough-Neural Computing: Techniques for Computing with Words, pp. 573–585. Springer, Heidelberg (2003)Google Scholar
  23. 23.
    Pawlak, Z.: Rough Sets – Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)MATHGoogle Scholar
  24. 24.
    Pawlak, Z., Skowron, A.: Rudiments of Rough Sets. Information Sciences 177, 3–27 (2007); Rough Sets: Some Extensions. Information Sciences 177, 28–40 (2007); Rough Sets and Boolean Reasoning. Information Sciences 177, 41–73 (2007)Google Scholar
  25. 25.
    Polkowski, L., Skowron, A. (eds.): Rough Sets in Knowledge Discovery 2: Applications, Case Studies and Software Systems. Studies in Fuzziness and Soft Computing, vol. 19. Physica-Verlag, Heidelberg (1998)MATHGoogle Scholar
  26. 26.
    Pulatova, S.: Covering (Rule-Based) Algorithms. In: Berry, M.W., Browne, M. (eds.) Lecture Notes in Data Mining, pp. 87–97. World Scientific, Singapore (2006)CrossRefGoogle Scholar
  27. 27.
    Rissanen, J.: Modeling by Shortest Data Description. Automatica 14, 465–471 (1978)CrossRefMATHGoogle Scholar
  28. 28.
    Rissanen, J.: Information and Complexity in Statistical Modeling. In: Springer Sciences and Business Media, LLC, New York (2007)Google Scholar
  29. 29.
  30. 30.
    Rough Set Exploration System, http://logic.mimuw.edu.pl/~rses
  31. 31.
    Skowron, A., Suraj, Z.: Rough Sets and Concurrency. Bulletin of the Polish Academy of Sciences 41, 237–254 (1993)MATHGoogle Scholar
  32. 32.
    Skowron, A., Swiniarski, R., Synak, P.: Approximation Spaces and Information Granulation. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets III. LNCS, vol. 3400, pp. 175–189. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  33. 33.
    Suraj, Z.: Some Remarks on Extensions and Restrictions of Information Systems. In: Ziarko, W.P., Yao, Y. (eds.) RSCTC 2000. LNCS (LNAI), vol. 2005, pp. 204–211. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  34. 34.
    Triantaphyllou, E., Felici, G. (eds.): Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques. In: Springer Science and Business Media, LLC, New York (2006)Google Scholar
  35. 35.
    Tsumoto, S.: Modelling Medical Diagnostic Rules Based on Rough Sets. In: Polkowski, L., Skowron, A. (eds.) RSCTC 1998. LNCS (LNAI), vol. 1424, pp. 475–482. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  36. 36.
    Wojna, A.: Analogy-Based Reasoning in Classifier Construction (Ph.D. Thesis). In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets IV. LNCS, vol. 3700, pp. 277–374. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  37. 37.
    Yao, J.T., Yao, Y.Y.: Induction of Classification Rules by Granular Computing. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 331–338. Springer, Heidelberg (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Paweł Delimata
    • 1
  • Barbara Marszał-Paszek
    • 2
  • Mikhail Moshkov
    • 3
  • Piotr Paszek
    • 2
  • Andrzej Skowron
    • 4
  • Zbigniew Suraj
    • 1
  1. 1.Chair of Computer ScienceUniversity of RzeszówRzeszówPoland
  2. 2.Institute of Computer ScienceUniversity of SilesiaSosnowiecPoland
  3. 3.Division of Mathematical and Computer Sciences and EngineeringKing Abdullah University of Science and TechnologyJeddahSaudi Arabia
  4. 4.Institute of MathematicsWarsaw UniversityWarsawPoland

Personalised recommendations