Combination of Multiple Nearest Neighbor Classifiers Based on Feature Subset Clustering Method
This paper proposes a new method called FC-MNNC based on feature subset clustering for combining multiple NNCs to obtain better performance than that of using a single NNC. In FC-MNNC, the component NNCs based on the reasonably partitioned feature subsets are parallel and independently able to classify one pattern and the final decision is aggregated by the majority voting rule. Here, two methods are used to partition the feature set. In method I, GA is used for clustering features to form different feature subsets according to the accuracy of the combination classification. And method II is the transitive closure clustering method based on the pair-wise correlation between features. To demonstrate the performance of FC-MNNC, we select four UCI databases for our experiments. The experimental results show that: (i) in FC-MNNC, the performance of method II isn’t better than that of method I; (ii) the accuracy of FC-MNNC based on method I is better than that of the standard NNC and feature selection using GA in individual classifier; (iii) the performance of FC-MNNC based on method I is not worse than that of feature subset selection using GA in multiple NNCs; and (iv) FC-MNNC is robust against irrelevant features.
KeywordsFeature Selection Classification Accuracy Feature Subset Individual Classifier Neighbor Classifier
Unable to display preview. Download preview PDF.
- 1.Cho, S.-B., Kim, J.H.: Combining multiple neural networks by fuzzy integral for robust classification. IEEE trans. on SMC 25, 380–384 (1995)Google Scholar
- 2.Cho, S.-B., Kim, J.H.: Multiple network fusion using fuzzy logic. IEEE trans. on NN 6, 497–501 (1995)Google Scholar
- 7.Langley, P., Iba, W.: Average–case analysis of a nearest neighbor algorithm. In: Proc. of the thirteenth international joint conference on artificial intelligence, pp. 889–894 (1993)Google Scholar
- 10.Lam, L., Suen, C.Y.: Application of majority voting to pattern recognition: An analysis of its behavior and performance. IEEE trans. on SMC 27, 553–568 (1997)Google Scholar
- 11.Mitchell, T.M.: Machine learning. China Machine Press, Beijing (2003)Google Scholar
- 12.UCI repository of machine learning databases and domain theories. FTP address, http://www.ics.uci.edu/~mlearn