Abstract
It has been a controversial issue in the research of cognitive science and artificial intelligence whether signal-type representations (typically connectionist networks) or symbol-type representations (e.g., semantic networks, production systems) should be used. Meanwhile, it has also been recognized that both types of information representations might exist in the human brain. In addition, symbol-type representations are often very helpful in gaining insights into unknown systems. For these reasons, comprehensible symbolic rules need to be extracted from trained neural networks. In this paper, an evolutionary multi-objective algorithm is employed to generate multiple models that facilitate the generation of signal-type and symbol-type representations simultaneously. It is argued that one main difference between signal-type and symbol-type representations lies in the fact that the signal-type representations are models of a higher complexity (fine representation), whereas symbol-type representations are models of a lower complexity (coarse representation). Thus, by generating models with a spectrum of model complexity, we are able to obtain a population of models of both signal-type and symbol-type quality, although certain post-processing is needed to get a fully symbol-type representation. An illustrative example is given on generating neural networks for the breast cancer diagnosis benchmark problem.
Keywords
- Neural Network
- Mean Square Error
- Multiobjective Optimization
- Hide Neuron
- Rule Extraction
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abbass, H.A.: An evolutionary artificialneuralnet works approach for breast cancer diagnosis. Artificial Intelligence in Medicine 25(3), 265–281 (2002)
Abbass, H.A.: Speeding up back-propagation using multiobjective evolutionary algorithms. Neural Computation 15(11), 2705–2726 (2003)
Andrews, R., Diederich, J., Tickle, A.: A survey and critique of techniques for extracting rules from trained artificialneuralnet works. Knowledge Based Systems 8(6), 373–389 (1995)
Badre, D., Wagner, A.: Semantic retrieval, mnemonic control, and prefrontal cortex. Behavior and Cognitive Neuroscience Reviews 1(3), 206–218 (2002)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Burnham, K.P., Anderson, D.R.: Model Selection and Multimodel Inference, 2nd edn. Springer, New York (2002)
Coello Coello, C., Veldhuizen, D., Lamont, G.: Evolutionary algorithms for solving multi-objective problems. Kluwer Academic, New York (2002)
Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, Chichester (2001)
Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In: Parallel Problem Solving from Nature, vol. VI, pp. 849–858 (2000)
Duch, W., Adamczak, R., Grabczewski, K.: Extraction of logical rules from backpropagation networks. Neural Processing Letters 7, 1–9 (1998)
Duch, W., Setiono, R., Zurada, J.: Computational intelligence methods for rule-based data understanding. Proceedings of the IEEE 92(5), 771–805 (2004)
Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: A criticalanalysis. Cognition 28(3), 3–71 (1988)
Gabrieli, J., Poldrack, R., Desmond, J.: The role of left prefrontal cortex in langrange and memory. Proceedings of the national Academy of Sciences 95, 906–913 (1998)
Hüsken, M., Gayko, J.E., Sendhoff, B.: Optimization for problem classes –Neural networks that learn to learn. In: Yao, X., Fogel, D.B. (eds.) IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (ECNN 2000), pp. 98–109. IEEE Press, Los Alamitos (2000)
Igeland, C., Hüsken, M.: Improving the Rprop learning algorithm. In: Proceedings of the 2nd ICSC International Symposium on Neural Computation, pp. 115–121 (2000)
Ishibuchi, H., Yamamoto, T.: Evolutionary multiobjective optimization for generating an ensemble of fuzzy rule-based classifiers. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1077–1088 (2003)
Ishikawa, M.: Rule extraction by successive regularization. Neural Networks 13, 1171–1183 (2000)
Jin, Y.: Advanced Fuzzy Systems Design and Applications. Springer, Heidelberg (2003)
Jin, Y., Okabe, T., Sendhoff, B.: Neuralnet work regularization and ensembling using multi-objective evolutionary algorithms. In: Congress on Evolutionary Computation, pp. 1–8. IEEE, Los Alamitos (2004)
Jin, Y., Sendhoff, B.: Extracting interpretable fuzzy rules from RBF networks. Neural Processing Letters 17(2), 149–164 (2003)
Martin, A., Chao, L.: Semantic memory and the brain: Structure and process. Current Opinions in Neurobiology 11, 194–201 (2001)
Miller, D.A., Zurada, J.M.: A dynamical system perspective of structural learning with forgetting. IEEE Transactions on Neural Networks 9(3), 508–515 (1998)
Prechelt, L.: PROBEN1 - a set of neuralnet work benchmark problems and benchmarking rules. Technical Report 21/94, Fakultát für Informatik, Universität Karlsruhe (1994)
Quinlan, J.R.: C4.5 Programs for Machine Learning. Morgan Kaufmann, San Francisco (1992)
Reed, R.D., Marks II, R.J.: Neural Smithing. MIT Press, Cambridge (1999)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropgation learning: The RPROP algorithm. In: IEEE international Conference on Neural Networks, vol. 1, pp. 586–591. IEEE, New York (1993)
Setiono, R.: Generating concise and accurate classification rules for breast cancer disgnosis. Artificial Intelligence in Medicine 18, 205–219 (2000)
Setiono, R., Liu, H.: Symbolic representation of neural networks. IEEE Computer 29(3), 71–77 (1996)
Taha, I., Ghosh, J.: Symbolic interpretation of artificialneuralnet works. IEEE Transactions on Knowledge and Data Engineering 11(3), 448–463 (1999)
de Roselito Teixeira, A., Braga, A.P., Takahashi, R.H.C., Saldanha, R.R.: Improving generalization of MLPs with multi-objective optimization. Neurocomputing 35, 189–194 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jin, Y., Sendhoff, B., Körner, E. (2005). Evolutionary Multi-objective Optimization for Simultaneous Generation of Signal-Type and Symbol-Type Representations. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds) Evolutionary Multi-Criterion Optimization. EMO 2005. Lecture Notes in Computer Science, vol 3410. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-31880-4_52
Download citation
DOI: https://doi.org/10.1007/978-3-540-31880-4_52
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24983-2
Online ISBN: 978-3-540-31880-4
eBook Packages: Computer ScienceComputer Science (R0)