Abstract
Hybrid genetic algorithms (GA) and artificial neural networks (ANN) are not new in the machine learning culture. Such hybrid systems have been shown to be very successful in classification and prediction problems. However, little attention has been focused on this architecture as a feature selection method and the consequent significance of the ANN activation function and the number of GA evaluations on the feature selection performance. The activation function is one of the core components of the ANN architecture and influences the learning and generalization capability of the network. Meanwhile the GA searches for an optimal ANN classifier given a set of chromosomes selected from those available. The objective of the GA is to combine the search for optimum chromosome choices with that of finding an optimum classifier for each choice. The process operates as a form of co-evolution with the eventual objective of finding an optimum chromosome selection rather than an optimum classifier. The selection of an optimum chromosome set is referred to in this paper as feature selection. Quantitative comparisons of four of the most commonly used ANN activation functions against ten GA evaluation step counts and three population sizes are presented. These studies employ four data sets with high dimension and low significant datum instances. That is to say that each datum has a high attribute count and the unusual or abnormal data are sparse within the data set. Results suggest that the hyperbolic tangent (tanh) activation function outperforms other common activation functions by extracting a smaller, but more significant feature set. Furthermore, it was found that fitness evaluation sizes ranging from 20,000 to 40,000 within populations ranging from 200 to 300, deliver optimum feature selection capability. Again, optimum in this sense meaning a smaller but more significant feature set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beiko RG, Charlebois RL (2005) GANN: genetic algorithm neural networks for the detection of conserved combinations of features in DNA. BMC Bioinformatics 6:36
Bevilacqua V, Mastronardi G, Menolascina F, Paradiso A, Tommasi S (2006) Genetic algorithms and artificial neural networks in microarray data analysis: a distributed approach. Eng Lett Spec Issue Bioinformatics 13(3):335–343
Cartwright H (2008) Using artificial intelligence in chemistry and biology: A practical guide. In: Chapter Evolutionary Algorithms, CRC Press, Taylor & Francis Group, Boca Raton, London pp 113–172
Cho HS, Kim TS, Wee JW, Jeon SM, Lee CH (2003) cDNA microarray data based classification of cancers using neural networks and genetic algorithms. In Nanotech’03: Nanotechnology Conference and Trade Show, proceedings, vol 1
DeJong KA, Spears WM (1991) An analysis of the interacting roles of population size and crossover in genetic algorithms. In: Schwefel HP, Männer R (eds) PPSN’91: first Workshop on Parallel Problem Solving from Nature, proceedings, volume 496 of Lecture Notes in Computer Science, Springer, Berlin pp 38–47
Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, BloomfIeld CD, Lander ES (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439):531–536
Karzynski M, Mateos Á, Herrero J, Dopazo J (2003) Using a genetic algorithm and a perceptron for feature selection and supervised class learning in DNA microarray data. Artif Intell Rev 20(1–2):39–51
Khan J, Wei JS, Ringnér M, Saal LH, Ladanyi M, Westermann F, Berthold F, Schwab M, Antonescu CR, Peterson C, Meltzer PS (2001) Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat Med 7:673–679
Lin T-C, Liu R-S, Chao Y-T, Chen S-Y (2006) Multiclass microarray data classification using GA/ANN method. In Yang Q, Webb GI (eds) PRICAI’06: trends in artificial intelligence, ninth Pacific Rim international conference on artificial intelligence, proceedings, vol 4099 of Lecture Notes in Computer Science. Springer, pp 1037-1041
Mitchell TM (1997) Does machine learning really work? AI Mag 18(3):11–20
Ramasubramanian P, Kannan A (2006) A genetic-algorithm based neural network short-term forecasting framework for database intrusion prediction system. Soft Comput 10:699–714
Schierz A (2009) Virtual screening of bioassay data. J Cheminform 1:2
Schwarzer G, Vach W, Schumacher M (2000) On the misuses of artificial neural network for prognostic and diagnostic classification in oncology. Stat Med 19(4):541–561
Shenouda E (2006) A quantitative comparison of different MLP activation functions in classification. In: ISNN’06: advances in neural networks, third international symposium on neural networks, proceedings, Part I–III, vol 2971 of Lecture Notes in Computer Science. Springer, Berlin, pp 849–857
Taheri M, Mohebbi A (2008) Design of artificial neural networks using a genetic algorithm to predict collection efficiency in venturi scrubbers. J Hazard Mater 157(1):122–129
Verma B, Zhang P (2007) A novel neural-genetic algorithm to find the most significant combination of features in digital mammograms. Appl Soft Comput 7(2):612–625
Zorić G, Pandžić IS (2006) Real-time language independent lip synchronization method using a genetic algorithm. Signal Processing 86(12):3644–3656
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tong, D.L., Mintram, R. Genetic Algorithm-Neural Network (GANN): a study of neural network activation functions and depth of genetic algorithm search applied to feature selection. Int. J. Mach. Learn. & Cyber. 1, 75–87 (2010). https://doi.org/10.1007/s13042-010-0004-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-010-0004-x