An Effective Feature Selection Scheme via Genetic Algorithm Using Mutual Information

  • Chunkai K. Zhang
  • Hong Hu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3614)


In the artificial neural networks (ANNs), feature selection is a well-researched problem, which can improve the network performance and speed up the training of the network. The statistical-based methods and the artificial intelligence-based methods have been widely used to feature selection, and the latter are more attractive. In this paper, using genetic algorithm (GA) combining with mutual information (MI) to evolve a nearoptimal input feature subset for ANNs is proposed, in which mutual information between each input and each output of the data set is employed in mutation in evolutionary process to purposefully guide search direction based on some criterions. By examining the forecasting at the Australian Bureau of Meteorology, the simulation of three different methods of feature selection shows that the proposed method can reduce the dimensionality of inputs, speed up the training of the network and get better performance.


Root Mean Square Error Feature Selection Mutual Information Feature Subset Australian Bureau 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1, 131–156 (1997)CrossRefGoogle Scholar
  2. 2.
    Montgomery, D.C., Peck, E.A.: Introduction to Linear Regression Analysis. John Wiley & Sons, New York (1982)zbMATHGoogle Scholar
  3. 3.
    Sen, A., Serivastava, M.: Regression Analysis: Theory, Methods, and Applications. Springer, New York (1990)zbMATHGoogle Scholar
  4. 4.
    Holz, H.J., Loew, M.H.: Relative feature importance: A classifier-independent approach to feature selection. In: Gelsema, E.S., Kanal, L.N. (eds.) Pattern Recognition in Practice IV, pp. 473–487. Elsevier, Amsterdam (1994)Google Scholar
  5. 5.
    Wang, H., Bell, D., Murtagh, F.: Automatic approach to feature subset selection based on relevance. IEEE Trans. PAMI 21(3), 271–277 (1999)Google Scholar
  6. 6.
    Choi, B., Hendtlass, T., Bluff, K.: A comparison of neural network input vector selection techniques. In: Orchard, B., Yang, C., Ali, M. (eds.) IEA/AIE 2004. LNCS (LNAI), vol. 3029, pp. 1–10. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Kwak, N., Choi, C.-H.: Input feature selection by mutual information based on parzen window. IEEE Trans. PAMI 24(12), 1667–1671 (2002)Google Scholar
  8. 8.
    Huang, D., Chow, T.W.S.: Effective feature selection scheme using mutual information. Neurocomputing 63, 325–343 (2005)CrossRefGoogle Scholar
  9. 9.
    Fraser, A.M., Swinney, H.L.: Independent Coordinates for Strange Attractors from Mutual Information. Physical Review A 33(2), 1134–1140 (1986)zbMATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    Cibas, T., Soulie, F.F., Gallinari, P., Raudys, S.: Variable selection with neural networks. Neurocomputing 12, 223–248 (1996)zbMATHCrossRefGoogle Scholar
  11. 11.
    Siedlechi, W., Sklansky, J.: A note on genetic algorithms for large-scale feature selection. Pattern Recognition Letters 10, 335–347 (1989)CrossRefGoogle Scholar
  12. 12.
    Emmanouilidis, C., Hunter, A., Macintyre, J., Cox, C.: Selecting features in neurofuzzy modeling by multiobjective genetic algorithms. Artificial Neural Networks, 749–754 (1999)Google Scholar
  13. 13.
    Yang, J.H., Honavar, V.: Feature Subset Selection Using a Genetic Algorithm. IEEE Intelligent Systems 13(2), 44–49 (1998)CrossRefGoogle Scholar
  14. 14.
    Oh, I.-S., Lee, J.-S., Moon, B.-R.: Hybrid genetic algorithms for feature selection. IEEE Trans. Pattern Analysis and Machine Intelligence 26, 1424–1437 (2004)CrossRefGoogle Scholar
  15. 15.
    LeMarshall, J.: An Intercomparison of Temperature and Moisture Fields Derived from TIROS Operational Vertical Sounder Data by Different Retrieval Techniques. Part I: Basic Statistics. Journal of Applied Meteorology 27, 1282–1293 (1988)Google Scholar
  16. 16.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Chunkai K. Zhang
    • 1
  • Hong Hu
    • 1
  1. 1.Member IEEE, Department of Mechanical Engineering and AutomationHarbin Institute of Technology, Shenzhen Graduate SchoolShenzhenChina

Personalised recommendations