Harmony-Based Feature Weighting to Improve the Nearest Neighbor Classification

  • Ali Adeli
  • Mehrnoosh Sinaee
  • M. Javad Zomorodian
  • Ali Hamzeh
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 177)


This paper introduces the use of Harmony Search with novel fitness function in order to assign higher weights to informative features while noisy irrelevant features are given low weights. The fitness function is based on the Area Under the receiver operating characteristics Curve (AUC). The aim of this feature weighting is to improve the performance of the k-NN algorithm. Experimental results show that the proposed method can improve the classification performance of the k-NN algorithm in comparison with the other important method in realm of feature weighting such as Mutual Information, Genetic Algorithm, Tabu Search and chi-squared (χ 2). Furthermore, on synthetic data sets, this method is able to allocate very low weight to the noisy irrelevant features which may be considered as the eliminated features from the data set.


AUC Harmony Search Feature weighting Noisy feature elimination k-NN 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blake, L., Merz, C.J.: UCI repository of machine learning databases, http://www.ics.uci.edu/~mlearn/MLRepository.html
  2. 2.
    Das, S., Mukhopadhyay, A., Roy, A., Abraham, A., Panigrahi, B.K.: Exploratory power of the harmony search algorithm: analysis and improvements for global numerical optimization. IEEE Transactions on Systems Man and Cybernetics Part B 41(1), 89–106 (2011)CrossRefGoogle Scholar
  3. 3.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis, vol. 7. Wiley (1973)Google Scholar
  4. 4.
    Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing, vol. 12. Springer (2003), http://www.mitpressjournals.org/doi/abs/10.1162/evco.2004.12.2.269
  5. 5.
    Fawcett, T.: Roc graphs, Notes and practical considerations for data mining researchers ROC graphs. Intelligent Enterprise 31 (HPL-2003-4) 28 (2003)Google Scholar
  6. 6.
    Guvenir, H.A., Akkus, A.: Weighted k nearest neighbor classification feature projections. In: Proc. of the Twelfth International Symposium on Computer and Information Sciences, ISCIS XII, pp. 44–51 (1997)Google Scholar
  7. 7.
    Han, E.-H(S.), Karypis, G., Kumar, V.: Text Categorization Using Weight Adjusted k-Nearest Neighbor Classification. In: Cheung, D., Williams, G.J., Li, Q. (eds.) PAKDD 2001. LNCS (LNAI), vol. 2035, pp. 53–65. Springer, Heidelberg (2001), http://dl.acm.org/citation.cfm?id=646419.693652 CrossRefGoogle Scholar
  8. 8.
    Jankowski, N.: Discrete feature weighting selection algorithm. In: Proceedings of the International Joint Conference on Neural Networks, vol. 1, pp. 636–641 (2003)Google Scholar
  9. 9.
    Mahdavi, M., Fesanghary, M., Damangir, E.: An improved harmony search algorithm for solving optimization problems. Applied Mathematics and Computation 188(2), 1567–1579 (2007)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Seeker, A., Freitas, A.: Wairs improving classification accuracy by weighting attributes in the airs classifier. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 3759–3765 (2007)Google Scholar
  11. 11.
    Tang, P.H., Tseng, M.H.: Medical data mining using BGA and RGA for weighting of features in fuzzy k-NN classification. In: International Conference on Machine Learning and Cybernetics, vol. 5, pp. 3070–3075 (2009)Google Scholar
  12. 12.
    Vivencio, D., Hruschka, E., Nicoletti, M., dos Santos, E., Galvao, S.: Feature-weighted k-nearest neighbor classifier. In: FOCI 2007, pp. 481–486 (2007)Google Scholar
  13. 13.
    Zomorodian, M.J., Adeli, A., Sinaee, M., Hashemi, S.: Improving Nearest Neighbor Classification by Elimination of Noisy Irrelevant Features. In: Horng, M.-F. (ed.) ACIIDS 2012, Part II. LNCS, vol. 7197, pp. 11–21. Springer, Heidelberg (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ali Adeli
    • 1
    • 2
  • Mehrnoosh Sinaee
    • 1
  • M. Javad Zomorodian
    • 1
    • 3
  • Ali Hamzeh
    • 1
  1. 1.Department of Computer Science & EngineeringShiraz UniversityShirazIran
  2. 2.Institute of Computer ScienceBojnurd Darolfonoun Technical CollegeBojnurdIran
  3. 3.Institute of Computer ScienceShiraz Bahonar Technical CollegeShirazIran

Personalised recommendations