Abstract
Dimension disaster will directly impact on the efficiency and accuracy of K-nearest neighbor (KNN) classification algorithm. In order to reduce the effect, this chapter proposes an improved Entropy-KNN algorithm based on attribute reduction. It combines KNN algorithm with information entropy theory to reduce the attribute, and the test sample is classified by the average distance and the numbers on the respective class. The experimental results show that compared with traditional KNN algorithm, the proposed algorithm enormously raises the classification accuracy rate; meanwhile it also maintains the efficiency of category.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cover TM, Hart PE. Nearest neighbor pattern classification. IEEE Trans Inform Theory. 1967;13(1):21–7.
Zhang R, Ooi BC, Tan KL. Making the pyramid technique robust to query types and workloads. In: Proceedings of the 20th international conference on data engineering, Boston, MA, USA, vol. 20, pp. 313–24. 2004.
Hughes GF. On the mean accuracy of statistical pattern recognizers. IEEE Trans Inform Theory. 1968;14(1):55–63.
Wang GY, Yu H, Yang DC. Decision table reduction based on conditional information entropy. Chin J Comput. 2002;25(7):759–66.
Anil KG. On optimum choice of k in nearest neighbor classification. Comput Stat Data Anal. 2006;50(11):3113–23.
Lindenbaum M, Markovitch S, Rusakov D, Lindenbaum M, Markovitch S, Rusakov D. Selective sampling for nearest neighbor classifiers. Mach Learn. 2004;54(2):125–52.
Swiniarski RW, Skowron A. Rough set methods in feature selection and recognition. Pattern Recogn Lett. 2003;24(2):833–49.
Zhou C, Yan Y, Chen Q, Zhou C, Yan Y, Chen Q. Improving nearest neighbor classification with cam weighted distance. Pattern Recogn. 2006;39(4):635–45.
Kohavi R, John GH. Wrappers for feature subset selection. Artif Intell. 1997;97(1–2):273–324.
Wang ZM, Wang KJ. Improve KNN algorithm based on entropy method. Comput Eng Appl. 2009;45(30):129–31. In Chinese.
Acknowledgments
This work was supported by National Key Technology R&D Program of the Ministry of Science and Technology of China (No. 2014BAH24F02) and Engineering Research Center of Information Networks, Ministry of Education. It was partly supported by National public industry (food) special funds for scientific research (No. 201313009-08).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Wei, L., Zhao, X., Zhou, X. (2015). An Enhanced Entropy-K-Nearest Neighbor Algorithm Based on Attribute Reduction. In: Wong, W. (eds) Proceedings of the 4th International Conference on Computer Engineering and Networks. Lecture Notes in Electrical Engineering, vol 355. Springer, Cham. https://doi.org/10.1007/978-3-319-11104-9_45
Download citation
DOI: https://doi.org/10.1007/978-3-319-11104-9_45
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11103-2
Online ISBN: 978-3-319-11104-9
eBook Packages: EngineeringEngineering (R0)