Abstract
Data models that are induced in classifier construction often consist of multiple parts, each of which explains part of the data. Classification methods for such multi-part models are called multimodal classification methods. The model parts may overlap or have insufficient coverage. How to deal best with the problems of overlapping and insufficient coverage? In this paper we propose a hierarchical or layered approach to this problem. Rather than seeking a single model, we consider a series of models under gradually relaxing conditions, which form a hierarchical structure. To demonstrate the effectiveness of this approach we consider two classifiers that construct multi-part models – one based on the so-called lattice machine and the other one based on rough set rule induction, and we design hierarchical versions of the two classifiers. The two hierarchical classifiers are compared through experiments with their non-hierarchical counterparts, and also with a method that combines k-nearest neighbors classifier with rough set rule induction as a benchmark. The results of the experiments show that this hierarchical approach leads to improved multimodal classifiers.
Keywords
- hierarchical classification
- multimodal classifier
- lattice machine
- rough sets
- rule induction
- k-nearest neighbors
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bazan, J.: A Comparison of Dynamic and non-Dynamic Rough Set Methods for Extracting Laws from Decision Table. In: Polkowski, L., Skowron, A. (eds.) Rough Sets in Knowledge Discovery, pp. 321–365. Physica-Verlag, Heidelberg (1998)
Bazan, J.G.: Classifiers based on two-layered learning. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 356–361. Springer, Heidelberg (2004)
Bazan, J.G., Nguyen, S.H., Nguyen, H.S., Skowron, A.: Rough set methods in approximation of hierarchical concepts. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 346–355. Springer, Heidelberg (2004)
Bazan, J.G., Son Nguyen, H., Hoa Nguyen, S., Synak, P., Wróblewski, J.: Rough Set Algorithms in Classification Problem. In: Polkowski, L., Tsumoto, S., Lin, T.Y. (eds.) Rough Set Methods and Applications, pp. 49–88. Physica-Verlag, Heidelberg (2000)
Bazan, J.G., Szczuka, M.S.: The Rough Set Exploration System. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets III. LNCS, vol. 3400, pp. 37–56. Springer, Heidelberg (2005)
Bazan, J.G., Szczuka, M.S., Wojna, A., Wojnarski, M.: On the Evolution of Rough Set Exploration System. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 592–601. Springer, Heidelberg (2004)
Bazan, J.G., Szczuka, M.S., Wróblewski, J.: A New Version of Rough Set Exploration System. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 397–404. Springer, Heidelberg (2002)
Behnke, S.: Hierarchical Neural Networks for Image Interpretation. LNCS (LNAI), vol. 2766. Springer, Heidelberg (2003)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
Cost, S., Salzberg, S.: A weighted nearest neighbor algorithm for learning with symbolic features. Machine Learning 10, 57–78 (1993)
Dietterich, T.G.: Ensemble Learning. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, 2nd edn., pp. 405–408. The MIT Press, Cambridge (2002)
Dudani, S.A.: The distance weighted k-nearest neighbor rule. IEEE Trans. Syst. Man Cyber. 6, 325–327 (1976)
Düntsch, I., Gediga, G.: Simple data filtering in rough set systems. International Journal of Approximate Reasoning 18(1–2), 93–106 (1998)
Fix, E., Hodges, J.L.: Discriminatory analysis, nonparametric discrimination: Consistency properties. Technical Report TR4, USAF School of Aviation Medicine, Randolph Field, TX (1951)
Góra, G., Wojna, A.G.: RIONA: a new classification system combining rule induction and instance-based learning. Fundamenta Informaticae 51(4), 369–390 (2002)
Handl, J., Knowles, J.: Cluster generators: synthetic data for the evaluation of clustering algorithms, http://dbkweb.ch.umist.ac.uk/handl/generators/
Nguyen, S.H., Bazan, J.G., Skowron, A., Nguyen, H.S.: Layered learning for concept synthesis. In: Peters, J.F., Skowron, A., Grzymała-Busse, J.W., Kostek, B.z., Świniarski, R.W., Szczuka, M.S. (eds.) Transactions on Rough Sets I. LNCS (LNAI), vol. 3100, pp. 187–208. Springer, Heidelberg (2004)
Pawlak, Z.: Rough Sets. Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)
Poggio, T., Smale, S.: The Mathematics of Learning: Dealing with Data. Notices of the AMS 50(5), 537–544 (2003)
Quinlan, R.: Improved Use of Continuous Attributes in C4.5. Journal of Artificial Intelligence Research 4, 77–90 (1996)
Quinlan, R.: Rulequest research data mining tools, http://www.rulequest.com/
RSES: Rough set exploration system, Institute of Mathematics, Warsaw University, Poland, http://logic.mimuw.edu.pl/rses
Skowron, A.: Boolean reasoning for decision rules generation. In: Komorowski, J., Raś, Z.W. (eds.) ISMIS 1993. LNCS (LNAI), vol. 689, pp. 295–305. Springer, Heidelberg (1993)
Skowron, A., Wang, H., Wojna, A., Bazan, J.G.: A Hierarchical Approach to Multimodal Classification. In: Ślęzak, D., Yao, J., Peters, J.F., Ziarko, W.P., Hu, X. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3642, pp. 119–127. Springer, Heidelberg (2005)
Snedecor, G.W., Cochran, W.G.: Statisitical Methods, 8th edn. Iowa State University Press, Ames (2002)
Stanfill, C., Waltz, D.: Toward memory-based reasoning. Communication of ACM 29, 1213–1229 (1986)
Stone, P.: Layered Learning in Multi-agent Systems: A Winning Approach to Robotic Soccer. MIT Press, Cambridge (2000)
Vapnik, V.N.: Statistical learning theory. Wiley, New York (1998)
Wang, H.: Nearest neighbors by neighborhood counting. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(6) (June 2006)
Wang, H., Dubitzky, W., Düntsch, I., Bell, D.: A lattice machine approach to automated casebase design: Marrying lazy and eager learning. In: Proc. IJCAI 1999, Stockholm, Sweden, pp. 254–259 (1999)
Wang, H., Düntsch, I., Bell, D.: Data reduction based on hyper relations. In: Proceedings of KDD 1998, pp. 349–353. New York (1998)
Wang, H., Düntsch, I., Gediga, G.: Classificatory filtering in decision systems. International Journal of Approximate Reasoning 23, 111–136 (2000)
Wang, H., Düntsch, I., Gediga, G., Skowron, A.: Hyperrelations in version space. International Journal of Approximate Reasoning 36(3), 223–241 (2004)
Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6, 1–34 (1997)
Wojna, A.: Analogy-Based Reasoning in Classifier Construction. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets IV. LNCS, vol. 3700, pp. 277–374. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Skowron, A., Wang, H., Wojna, A., Bazan, J. (2006). Multimodal Classification: Case Studies. In: Peters, J.F., Skowron, A. (eds) Transactions on Rough Sets V. Lecture Notes in Computer Science, vol 4100. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11847465_11
Download citation
DOI: https://doi.org/10.1007/11847465_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-39382-5
Online ISBN: 978-3-540-39383-2
eBook Packages: Computer ScienceComputer Science (R0)