Multimodal Classification: Case Studies

  • Andrzej Skowron
  • Hui Wang
  • Arkadiusz Wojna
  • Jan Bazan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4100)


Data models that are induced in classifier construction often consist of multiple parts, each of which explains part of the data. Classification methods for such multi-part models are called multimodal classification methods. The model parts may overlap or have insufficient coverage. How to deal best with the problems of overlapping and insufficient coverage? In this paper we propose a hierarchical or layered approach to this problem. Rather than seeking a single model, we consider a series of models under gradually relaxing conditions, which form a hierarchical structure. To demonstrate the effectiveness of this approach we consider two classifiers that construct multi-part models – one based on the so-called lattice machine and the other one based on rough set rule induction, and we design hierarchical versions of the two classifiers. The two hierarchical classifiers are compared through experiments with their non-hierarchical counterparts, and also with a method that combines k-nearest neighbors classifier with rough set rule induction as a benchmark. The results of the experiments show that this hierarchical approach leads to improved multimodal classifiers.


hierarchical classification multimodal classifier lattice machine rough sets rule induction k-nearest neighbors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bazan, J.: A Comparison of Dynamic and non-Dynamic Rough Set Methods for Extracting Laws from Decision Table. In: Polkowski, L., Skowron, A. (eds.) Rough Sets in Knowledge Discovery, pp. 321–365. Physica-Verlag, Heidelberg (1998)Google Scholar
  2. 2.
    Bazan, J.G.: Classifiers based on two-layered learning. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 356–361. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Bazan, J.G., Nguyen, S.H., Nguyen, H.S., Skowron, A.: Rough set methods in approximation of hierarchical concepts. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 346–355. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Bazan, J.G., Son Nguyen, H., Hoa Nguyen, S., Synak, P., Wróblewski, J.: Rough Set Algorithms in Classification Problem. In: Polkowski, L., Tsumoto, S., Lin, T.Y. (eds.) Rough Set Methods and Applications, pp. 49–88. Physica-Verlag, Heidelberg (2000)Google Scholar
  5. 5.
    Bazan, J.G., Szczuka, M.S.: The Rough Set Exploration System. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets III. LNCS, vol. 3400, pp. 37–56. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Bazan, J.G., Szczuka, M.S., Wojna, A., Wojnarski, M.: On the Evolution of Rough Set Exploration System. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 592–601. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Bazan, J.G., Szczuka, M.S., Wróblewski, J.: A New Version of Rough Set Exploration System. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 397–404. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Behnke, S.: Hierarchical Neural Networks for Image Interpretation. LNCS (LNAI), vol. 2766. Springer, Heidelberg (2003)zbMATHCrossRefGoogle Scholar
  9. 9.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  10. 10.
    Cost, S., Salzberg, S.: A weighted nearest neighbor algorithm for learning with symbolic features. Machine Learning 10, 57–78 (1993)Google Scholar
  11. 11.
    Dietterich, T.G.: Ensemble Learning. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, 2nd edn., pp. 405–408. The MIT Press, Cambridge (2002)Google Scholar
  12. 12.
    Dudani, S.A.: The distance weighted k-nearest neighbor rule. IEEE Trans. Syst. Man Cyber. 6, 325–327 (1976)Google Scholar
  13. 13.
    Düntsch, I., Gediga, G.: Simple data filtering in rough set systems. International Journal of Approximate Reasoning 18(1–2), 93–106 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Fix, E., Hodges, J.L.: Discriminatory analysis, nonparametric discrimination: Consistency properties. Technical Report TR4, USAF School of Aviation Medicine, Randolph Field, TX (1951)Google Scholar
  15. 15.
    Góra, G., Wojna, A.G.: RIONA: a new classification system combining rule induction and instance-based learning. Fundamenta Informaticae 51(4), 369–390 (2002)zbMATHMathSciNetGoogle Scholar
  16. 16.
    Handl, J., Knowles, J.: Cluster generators: synthetic data for the evaluation of clustering algorithms,
  17. 17.
    Nguyen, S.H., Bazan, J.G., Skowron, A., Nguyen, H.S.: Layered learning for concept synthesis. In: Peters, J.F., Skowron, A., Grzymała-Busse, J.W., Kostek, B.z., Świniarski, R.W., Szczuka, M.S. (eds.) Transactions on Rough Sets I. LNCS (LNAI), vol. 3100, pp. 187–208. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Pawlak, Z.: Rough Sets. Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)zbMATHGoogle Scholar
  19. 19.
    Poggio, T., Smale, S.: The Mathematics of Learning: Dealing with Data. Notices of the AMS 50(5), 537–544 (2003)zbMATHMathSciNetGoogle Scholar
  20. 20.
    Quinlan, R.: Improved Use of Continuous Attributes in C4.5. Journal of Artificial Intelligence Research 4, 77–90 (1996)zbMATHGoogle Scholar
  21. 21.
    Quinlan, R.: Rulequest research data mining tools,
  22. 22.
    RSES: Rough set exploration system, Institute of Mathematics, Warsaw University, Poland,
  23. 23.
    Skowron, A.: Boolean reasoning for decision rules generation. In: Komorowski, J., Raś, Z.W. (eds.) ISMIS 1993. LNCS (LNAI), vol. 689, pp. 295–305. Springer, Heidelberg (1993)Google Scholar
  24. 24.
    Skowron, A., Wang, H., Wojna, A., Bazan, J.G.: A Hierarchical Approach to Multimodal Classification. In: Ślęzak, D., Yao, J., Peters, J.F., Ziarko, W.P., Hu, X. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3642, pp. 119–127. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  25. 25.
    Snedecor, G.W., Cochran, W.G.: Statisitical Methods, 8th edn. Iowa State University Press, Ames (2002)Google Scholar
  26. 26.
    Stanfill, C., Waltz, D.: Toward memory-based reasoning. Communication of ACM 29, 1213–1229 (1986)CrossRefGoogle Scholar
  27. 27.
    Stone, P.: Layered Learning in Multi-agent Systems: A Winning Approach to Robotic Soccer. MIT Press, Cambridge (2000)Google Scholar
  28. 28.
    Vapnik, V.N.: Statistical learning theory. Wiley, New York (1998)zbMATHGoogle Scholar
  29. 29.
    Wang, H.: Nearest neighbors by neighborhood counting. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(6) (June 2006)Google Scholar
  30. 30.
    Wang, H., Dubitzky, W., Düntsch, I., Bell, D.: A lattice machine approach to automated casebase design: Marrying lazy and eager learning. In: Proc. IJCAI 1999, Stockholm, Sweden, pp. 254–259 (1999)Google Scholar
  31. 31.
    Wang, H., Düntsch, I., Bell, D.: Data reduction based on hyper relations. In: Proceedings of KDD 1998, pp. 349–353. New York (1998)Google Scholar
  32. 32.
    Wang, H., Düntsch, I., Gediga, G.: Classificatory filtering in decision systems. International Journal of Approximate Reasoning 23, 111–136 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Wang, H., Düntsch, I., Gediga, G., Skowron, A.: Hyperrelations in version space. International Journal of Approximate Reasoning 36(3), 223–241 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6, 1–34 (1997)zbMATHMathSciNetGoogle Scholar
  35. 35.
    Wojna, A.: Analogy-Based Reasoning in Classifier Construction. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets IV. LNCS, vol. 3700, pp. 277–374. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Andrzej Skowron
    • 1
  • Hui Wang
    • 2
  • Arkadiusz Wojna
    • 3
  • Jan Bazan
    • 4
  1. 1.Institute of MathematicsWarsaw UniversityWarsawPoland
  2. 2.School of Computing and MathematicsUniversity of Ulster at Jordanstown, Northern IrelandUnited Kingdom
  3. 3.Institute of InformaticsWarsaw UniversityWarsawPoland
  4. 4.Institute of MathematicsUniversity of RzeszówRzeszówPoland

Personalised recommendations