Advertisement

Abstract

In this chapter we describe the basics of Genetic Algorithms and how they can be used to train Artificial Neural Networks. Supervised training of Multilayer Perceptrons for classification problems is considered. We also explain how the Genetic Algorithm can be hybridized with other algorithms and present two hybrids between it and two classical algorithms for the neural network training: Backpropagation and Levenberg-Marquardt. Several experiments over a set of six applications in the context of Bioinformatics are performed comparing the Genetic Algorithm, its hybrids, and the classical algorithms mentioned above. The testbed has been chosen from Proben1: breast cancer, diabetes, heart disease, gene, soybean, and thyroid. The results show that the genetic algorithm hybridized with Levenberg-Marquardt is a serious competitor for standard approaches.

Key words

Neural networks genetic algorithms hybridization bioinformatics medical applications Proben 1 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alander, J. T., 1994, Indexed bibliography of genetic algorithms and neural networks. Technical Report 94-1-NN, University of Vaasa, Department of Information Technology and Production Economics.Google Scholar
  2. Alba, E., 1993, Aplicación de los algoritmos genéticos para el diseño de redes neuronales, Informáticay Automática 26(2):22–35 (text in Spanish).Google Scholar
  3. Alba, E., Aldana, J. F., and Troya, J. M., 1993, Full automatic ANN design: a genetic approach, in: New Trends in Neural Computation, International Workshop on artificial Neural Networks, J. Mira, J. Cabestany, and A. Prieto, eds., Springer-Verlag, Sitges, Spain, pp. 399–404.Google Scholar
  4. Bäck, T., 1996, Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms, Oxford University Press, New York.Google Scholar
  5. Bäck, T., Fogel, D. B., Whitley, D., and Angeline, P. J., 2000, Mutation operators, in: Evolutionary Computation I. Basic Algorithms and Operators, T. Bäck, D. B. Fogel, and T. Michalewicz, eds., IOP Publishing Lt, pp. 237–255.Google Scholar
  6. Bennett, K. P., and Mangasarian, O. L., 1992, Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software. 1:23–34.CrossRefGoogle Scholar
  7. Blum, C, and Roli, A., 2003, Metaheuristics in combinatorial optimization: overview and conceptual comparison, ACM Computing Surveys. 35(3):268–308.CrossRefGoogle Scholar
  8. Booker, L. B., Fogel, D. B., Whitley, D., Angeline, P. J., and Eiben, A. E., 2000, Recombination, in: Evolutionary Computation 1. Basic Algorithms and Operators, Bäck, T., Fogel, D. B., and Michalewicz, T., eds., IOP Publishing Lt, pp. 256–307.Google Scholar
  9. Cantú-Paz, E., 2003, Pruning neural networks with distribution estimation algorithms, in: Proceedings of GECCO 2003 (LNCS 2723), E. Cantú-Paz et al., eds., Springer-Verlag, Chicago, USA, pp. 790–800.Google Scholar
  10. Cao, C, Wang, X., and Lu, M.-P., 2003, Direct torque control based on FNN and optimization, in: Proceedings of the Second International Conference on Machine Learning and Cybernetics, IEEE Computer Society Press, Xian, China, pp. 760–764.Google Scholar
  11. Cheng, W.-S., 2004, An application of adaptive-genetic neural algorithm to sinter’s BTP process, in: Proceedings of the Third International Conference on Machine Learning and Cybernetics, IEEE Computer Society Press, Shanghai, China, pp. 3356–3360.Google Scholar
  12. Cotta, C, Alba, E., Sagarna, R., and Larrañaga, P., 2001, Adjusting weights in artificial neural networks using evolutionary algorithms, in: Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation, P. Larrañaga, and J. A. Lozano, eds., Kluwer Academic Publishers, Boston, USA, pp. 357–373.Google Scholar
  13. Cotta, C, and Troya, J. M., 1998, On decision-making in strong hybrid evolutionary algorithms, in: Tasks and Methods in Applied Artificial Intelligence, Lecture Notes in Artificial Intelligence, vol. 1415, Del Pobil, A. P., Mira, J., and Ali, M., eds., Springer-Verlag, Berlin, Germany, 418–427.Google Scholar
  14. Davis, L., 1991, Handbook of Genetic Algorithms, Van Nostrand Reinhold, New York.Google Scholar
  15. Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J., Sandhu, S., Guppy, K., Lee, S., and Froelicher, V., 1989, International application of a new probability algorithm for the diagnosis of coronary artery disease, American Journal of Cardiology 64:304–310.PubMedCrossRefGoogle Scholar
  16. Erhard, W., Fink, T., Gutzmann, M. M., Rahn, C, Doering, A., and Galicki, M., 1998, The improvement and comparison of different algorithms for optimizing neural networks on the MasPar MP-2, in: Neural Computation — NC’98, M. Heiss, ed., ICSC Academic Press, Vienna, Austria, pp. 617–623.Google Scholar
  17. Eshelman, L. J., 2000, Genetic algorithms, in: Evolutionary Computation 1. Basic Algorithms and Operators, T. Bäck, D. B. Fogel, and T. Michalewicz, eds., IOP, pp. 64–80.Google Scholar
  18. Gengyin, L., Ming, Z., and Zhiyuan, Z., 2003, Research on power quality disturbance automatic recognition and location, in: Proceedings of the Power Engineering Society General Meeting, IEEE Computer Society Press, Toronto, Canada, pp. 687–691.CrossRefGoogle Scholar
  19. Gennari, J. H., Langley, P., and Fisher, D., 1989, Models of incremental concept formation, Artificial Intelligence 40:11–61.CrossRefGoogle Scholar
  20. Gilson, M., Py, J. S., Brault, J. J., and Sawan, M., 2003, Training recurrent pulsed networks by genetic and taboo methods, in: Proceedings of the Canadian Conference on Electrical and Computer Engineering, IEEE Computer Society Press, Montreal, CA, pp. 1857–1860.Google Scholar
  21. Hagan, M. T., and Menhaj, M. B., 1994, Training feedforward networks with the Marquardt algorithm, IEEE Transactions on Neural Networks 5(6):989–993.CrossRefGoogle Scholar
  22. Holland, J. H., 1975, Adaptation in Natural and Artificial Systems, the University of Michigan Press, Ann Arbor, Michigan.Google Scholar
  23. Islam, M. M., and Yao, X., 2003, A constructive algorithm for training cooperative neural network ensembles, IEEE Transactions on Neural Networks 14(4):820–834.CrossRefGoogle Scholar
  24. Ku, K. W. C, Mak, M. W., and Siu, W. C, 1999, Adding learning to cellular genetic algorithms for training recurrent neural networks, IEEE Transactions on Neural Networks 10(2):239–252.CrossRefGoogle Scholar
  25. Kuri-Morales, A. F., Ortiz-Posadas, M. R., Zenteno, D., and Peñaloza, R. P., 2003, Classification of sperm cells according to their chromosomic content using a neural network trained with a genetic algorithm, in: Proceedings of the 25th Annual International Conference of the IEEE EMBS, IEEE Computer Society Press, Cancun, Mexico, pp. 2253–2256.Google Scholar
  26. Land, W. H., and Albertelli, L. E., 1998, Breast cancer screening using evolved neural networks, in: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, IEEE Computer Society Press, San Diego, USA, pp. 1619–1624.Google Scholar
  27. Leung, F. H. F., Lam, H. K., Ling, S. H., and Tarn, P. K. S., 2003a, Tuning of the structure and parameters of a neural network using an improved genetic algorithm, IEEE Transactions on Neural Networks 14(l):79–88.CrossRefGoogle Scholar
  28. Leung, K. F., Leung, F. H. F., Lam, H. K., and Ling, S. H., 2004, On interpretation of graffiti digits and characters for ebooks: Neural-fuzzy network and genetic algorithm approach. IEEE Transactions on Industrial Electronics 51(2):464–471.CrossRefGoogle Scholar
  29. Leung, K., Leung, F., Lam, H., and Tarn, P., 2003b, Neural fuzzy network and genetic algorithm approach for cantonese speech command recognition, in: Proceedings of the 12th IEEE International Conference on Fuzzy Systems, IEEE Computer Society Press, St. Louis, USA, pp. 208–213.CrossRefGoogle Scholar
  30. Ling, S. H., Leung, F. H. F., Lam, H. K., Lee, Y.-S., and Tam, P. K. S., 2003, A novel genetic-algorithm-based neural network for short-term load forecasting, IEEE Transactions on Industrial Electronics 50(4):793–799.CrossRefGoogle Scholar
  31. Lisboa, P. J. G., Vellido, A., and Edisbury, B., 2000, Business Applications of Neural Networks. The State-of-the-Art of Real-World Applications, World Scientific.Google Scholar
  32. Mangasarian, O. L., Setiono, R., and Wolberg, W. H., 1990, Pattern recognition via linear programming: theory and application to medical diagnosis, in: Large-Scale Numerical Optimization, T. F. Coleman, and Y. Li, eds., SIAM Publications, Philadelphia, USA, pp. 22–31.Google Scholar
  33. McClelland, J. L., and Rumelhart, D. E., 1986, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. The MIT Press, Cambridge, USA.Google Scholar
  34. Moon, S.-W., and Kong, S.-G., 2001, Block-based neural networks, IEEE Transactions on Neural Networks 12(2):307–317.CrossRefGoogle Scholar
  35. Noordewier, M. O., Towell, G. G., and Shavlik, J. B., 1991, Training knowledge-based neural networks to recognize genes in DNA, in: Advances in Neural Information Processing Systems, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, eds., Morgan Kaufmann Publishers Inc., Denver, USA, pp. 530–536.Google Scholar
  36. Parker, G. B., and Lee, Z., 2003, Evolving neural networks for hexapod leg controllers, in: Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE Computer Society Press, Las Vegas, USA, pp. 1376–1381.Google Scholar
  37. Pattnaik, S. S., Khuntia, B., Panda, D. C, Neog, D. K., and Dutta, S. D. M., 2005, Application of a genetic algorithm in an artificial neural network to calculate the resonant frequency of a tunable single-shorting-post rectangular-patch antenna, International Journal of RF and Microwave Computer-Aided Engineering 15(1): 150–144.CrossRefGoogle Scholar
  38. Prechelt, L., 1994, Probenl—A set of neural network benchmark problems and benchmarking rules. Technical Report 21, Fakultät für Informatik Universität Karlsruhe, GE, 76128.Google Scholar
  39. Ragg, T., Gutjahr, S., and Sa, H., 1997, Automatic determination of optimal network topologies based on information theory and evolution, in: Proceedings of the 23rd EUROMICRO Conference, IEEE Computer Society Press, Budapest, pp. 549–555.Google Scholar
  40. Rosenblatt, F., 1962, Principles of Neurodynamics, Spartan Books, New York.zbMATHGoogle Scholar
  41. Rumelhart, D., Hinton, G., and Williams, R., 1986, Learning representations by backpropagation errors, Nature 323:533–536.CrossRefADSGoogle Scholar
  42. Schiffmann, W., Joost, M., and Werner, R., 1992, Optimization of the backpropagation algorithm for training multilayer perceptrons. Technical report, Institute of Physics, University of Koblenz, Koblenz, Germany.Google Scholar
  43. Smith, J. W., Everhart, J. E., Dickson, W. C, Knowler, W. C, and Johannes, R. S., 1988, Using the ADAP learning algorithm to forecast the onset of diabetes mellitus, in: Proceedings of the Twelfth Symposium on Computer Applications in Medical Care, IEEE Computer Society Press, Washington D.C., USA, pp. 261–265.Google Scholar
  44. Talbi, E.-G., 2002, A taxonomy of hybrid metaheuristics, Journal of Heuristics 8(2):541–564.CrossRefGoogle Scholar
  45. Tan, M., and Eshelman, L., 1988, Using weighted networks to represent classification knowledge in noisy domains, in: Proceedings of the 5th International Conference on Machine Learning, Ann Arbor, USA, pp. 121–134.Google Scholar
  46. Tarca, L. A., Grandjean, B. P. A., and Larachi, F., 2004, Embedding monotonicity and concavity in the training of neural networks by means of genetic algorithms application to multiphase flow, Computers and Chemical Engineering 28(9): 1701–1713.CrossRefGoogle Scholar
  47. Wang, K., Gelgele, H. L., Wang, Y., Yuan, Q., and Fang, M., 2003, A hybrid intelligent method for modelling the EDM process, International Journal of Machine Tools Manufacture 43(10):995–999.CrossRefGoogle Scholar
  48. Wolberg, W. H., 1990, Cancer diagnosis via linear programming, SIAM News 23(5): 1–18.Google Scholar
  49. Wolberg, W. H., and Mangasarian, O. L., 1990, Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proceedings of the National Academy of Sdences 87(23):9193–9196.zbMATHCrossRefADSGoogle Scholar
  50. Xiaoyun, S., Donghui, L., Kai, Z., Liweil, G., Ran, Z., and Jianye, L., 2004, Neural network with adaptive genetic algorithm for eddy current nondestructive testing, in: Proceedings of the 5th World Congress on Intelligent Control and Automation, IEEE Computer Society Press, Hangzhou, China, pp. 2034–2037.CrossRefGoogle Scholar
  51. Yao, X., 1999, Evolving artificial neural networks, Proceedings of the IEEE 87(9): 1423–1447.CrossRefGoogle Scholar
  52. Yao, X., and Liu, Y., 1997, A new evolutionary system for evolving artificial neural networks, IEEE Transactions on Neural Networks 8:694–713.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2006

Authors and Affiliations

  • Enrique Alba
    • 1
  • Francisco Chicano
    • 1
  1. 1.Department of Languages and Computer ScienceUniversity of MalagaSpain

Personalised recommendations