Abstract
This paper proposes a new method called FC-MNNC based on feature subset clustering for combining multiple NNCs to obtain better performance than that of using a single NNC. In FC-MNNC, the component NNCs based on the reasonably partitioned feature subsets are parallel and independently able to classify one pattern and the final decision is aggregated by the majority voting rule. Here, two methods are used to partition the feature set. In method I, GA is used for clustering features to form different feature subsets according to the accuracy of the combination classification. And method II is the transitive closure clustering method based on the pair-wise correlation between features. To demonstrate the performance of FC-MNNC, we select four UCI databases for our experiments. The experimental results show that: (i) in FC-MNNC, the performance of method II isn’t better than that of method I; (ii) the accuracy of FC-MNNC based on method I is better than that of the standard NNC and feature selection using GA in individual classifier; (iii) the performance of FC-MNNC based on method I is not worse than that of feature subset selection using GA in multiple NNCs; and (iv) FC-MNNC is robust against irrelevant features.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cho, S.-B., Kim, J.H.: Combining multiple neural networks by fuzzy integral for robust classification. IEEE trans. on SMC 25, 380–384 (1995)
Cho, S.-B., Kim, J.H.: Multiple network fusion using fuzzy logic. IEEE trans. on NN 6, 497–501 (1995)
Tumer, K., Ghosh, J.: Error correction and error reduction in ensemble classifiers. Connection science 8(314), 385–404 (1996)
Kittler, J., Hojjatoleslami, A., windeatt, T.: Strategies for combining classifiers employing shared and distinct representations. Pattern recognition letters 18, 1373–1377 (1997)
Bay, S.D.: Nearest neighbor classifiers from multiple feature subsets. Intelligent data analysis 3, 191–209 (1999)
Breiman, L.: Bagging predictors. Machine learning 24, 123–140 (1996)
Langley, P., Iba, W.: Average–case analysis of a nearest neighbor algorithm. In: Proc. of the thirteenth international joint conference on artificial intelligence, pp. 889–894 (1993)
Vishwath, P., Murty, M.N., Bhatnagar, C.: Fusion of multiple approximate nearest neighbor classifier for fast and efficient classification. Information fusion 5, 239–250 (2004)
Kuncheva, L.I., Jain, L.C.: Designing classifier fusion systems by genetic algorithms. IEEE trans. on evolutionary computation 4, 327–336 (2000)
Lam, L., Suen, C.Y.: Application of majority voting to pattern recognition: An analysis of its behavior and performance. IEEE trans. on SMC 27, 553–568 (1997)
Mitchell, T.M.: Machine learning. China Machine Press, Beijing (2003)
UCI repository of machine learning databases and domain theories. FTP address, http://www.ics.uci.edu/~mlearn
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, LJ., Hua, Q., Wang, XL., Chen, QC. (2006). Combination of Multiple Nearest Neighbor Classifiers Based on Feature Subset Clustering Method. In: Yeung, D.S., Liu, ZQ., Wang, XZ., Yan, H. (eds) Advances in Machine Learning and Cybernetics. Lecture Notes in Computer Science(), vol 3930. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11739685_56
Download citation
DOI: https://doi.org/10.1007/11739685_56
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33584-9
Online ISBN: 978-3-540-33585-6
eBook Packages: Computer ScienceComputer Science (R0)