Abstract
Feed-forward neural network models approximate nonlinear functions connecting inputs to outputs. The cascade correlation (CC) learning algorithm allows networks to grow dynamically starting from the simplest network topology to solve increasingly more difficult problems. It has been demonstrated that the CC network can solve a wide range of problems including those for which other kinds of networks (e.g., back-propagation networks) have been found to fail. In this paper we show the mechanism and characteristics of nonlinear function learning and representations in CC networks, their generalization capabilities, the effects of environmental bias, etc., using a variety of knowledge representation analysis tools.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Buckingham, D. & Shultz, T.R. (1994). A connectionist model of the development of velocity, time, and distance concepts. Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society (pp. 507-511). Hillsdale, NJ: Lawrence Earlbaum Associates.
Cohn, D.A. (1994). Neural network exploration using optimal experiment design. In J.D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural infonnation processing systems 6 (pp. 679-686). San Mateo, CA: Morgan Kaufmann.
Cohn, D.A., Ghahramani, Z. & Jordan, M.I. (1995). Active learning with statistical models. In G. Tesauro, D.S. Touretzky, & T.K. Leen (Eds.), Advances in neural information processing systems 7 (pp. 705–712). Cambridge, MA: The MIT Press.
Fahlman, S.E. (1989). Faster-learning variations on back-propagation: An empirical study. In D.S. Touretzky, G.E. Hinton, & T.J. Sejnowski (Eds.), Proceedings of the 1988 Connectionist Models Summer School (pp. 33-51). San Mateo, CA: Morgan Kaufmann.
Fahlman, S.E. & Lebiere, C. (1990). The cascade correlation learning architecture. In D.S. Touretzky (Ed.), Advances in neural information processing systems 2 (pp. 524–532). San Mateo, CA: Morgan Kaufmann.
Friedman, J.H. (1991). Multivariate adaptive regression splines. The Annals of Statistics, 19, 19–1.
Geman, S., Bienenstock, E. & R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4, 4–1.
Harshman, R.A. (1972). Foundations of parafac procedure: Models and conditions for an “exploratory” multi-model factor analysis. UCLA Phonetic Lab Working Paper #16.
Hastie, T.J. & Tibshirani, R.J. (1990). Generalized additive models. London: Chapman and Halt.
Hinton, G.E. & Shallice, T. (1991). Lesioning an attractor network: Investigations of acquired dyslexia. Psychological Review, 98, 74–95.
Koene, R. & Takne, Y. (1998). Discriminant component pruning: Regularization and interpretation of multi-layered back-propagation networks. Neural Computation, in press.
Mareschal, D. & Shultz, T.R. (in press). Development of children’s seriation: a connectionist approach. Connection Science.
Moseholm, L., Taudorf, E. & Frosig, A. (1993). Pulmonary function changes in asthmatics associated with low-level SO2 and NO2 air pollution, weather, and medicine intake. Allergy; 48, 48–334.
Oshima-Takane, Y. (1988). Children learn from speech not addressed to them: The case of personal pronouns. Journal of Child Language, 15, 15–94.
Oshima-Takane, Y. (1992). Analysis of pronomial errors: A case study. Journal of Child Language, 19, 19–111.
Oshima-Takane, Y. & Benaroya, S. (1989). An alternative view of pronomial errors in autistic children. Journal of Autism and Developmental Disorders, 19, 19–73.
Oshima-Takane, Y., Goodz, E. & Derevensky, J.L. (1996). Birth order effects on early language development: Do secondborn children learn from overheard speech? Child Development, 67, 67–621.
Oshima-Takane, Y., Takane, Y. & Shultz, T.R. (1995). Learning of personal pronouns: Network models and analysis. Technical Report 2095, McGill Papers in Cognitive Science, McGill University, Montreal.
Oshima-Takane, Y., Takane, M. & Takane, Y. (1998). Learning of first, second, and third person pronouns. Proceedings of the Nineteenth Annual Conference of the Cognitive Science Society (pp. 800-805). Mahwar, NJ: Lawrence Earlbaum Associates.
Plate, T., Band, P., Bert, J. & Grace, J. (1997). Visualizing the function computed by a feedforward neural network. In Kosabov, N., Kozma, R., Ko, K., O’Shea, R., Coghill, G. & Gedeon, T. (Eds.) Progress in connectionist-based information systems (pp. 306-309). Berlin: Springer Verlag.
Prechelt, L. (1996). Investigation of the CasCor family of learning algorithms. Neural Neuworks, in press.
Reed, R. (1993). Pruning algorithms—a survey. IEEE Transactions on Neural Networks, 4, 740–746.
Roosen, C.B. (1995). Visualization and exploration of high-dimensional functions through functional ANOVA decomposition. Unpublished Doctoral Dissertation, Stanford University.
Shultz, T.R. (1998). A computational analysis of conservation. Developmental Science, 1, 1–103.
Shultz, T.R., Buckingham, D. & Oshima-Takane, Y. (1994). A connectionist model of learning personal pronouns in English. In Hanson, S.J., Petche, T., Kearn, M., & Rivest, R.J. (Eds.), Computational learning theory and natural learning systems, Vol. 2: Intesection between theory and experiment (pp. 347-362). Cambridge, MA: MIT Press.
Shultz, T.R. & Mareschal, D. (1997). Rethinking innateness, learning, and constructivism: Connectionist perspective on development. Cognitive Development, 12, 12–563.
Shultz, T.R., Mareschal, D. & Schmidt, W.C. (1994). Modeling cognitive development on balance scale phenomena. Machine Learning, 16, 16–57.
Shultz, T.R., Oshima-Takane, Y. & Takane, Y. (1995). Analysis of unstandardized contributions in cross connected networks. In G. Tesauro, D.S. Touretzky, & T.K. Leen (Eds.), Advances in neural information processing systems 7 (pp. 601-608). Cambridge, MA: MIT Press.
Shultz, T.R., Schmidt, W.C, Buckingham, D. & Mareschal, D. (1994). Modeling cognitive development with a generative connectionist algorithm. In Simon, T.J. & Haiford, G.S. (Ed.), Developing cognitive competence: New approaches to process modeling (pp. 205-261). Hillsdale, NJ: Lawrence Earlbaum Associates.
Sirois, S. & Shultz, T.R. (1998). Neural network modeling of developmental effects in discrimination shifts. Journal of Experimental Child Psychology, in press.
Takane, Y. (1995). Nonlinear PCA by neural network models. Proceedings of the Sixty-third Annual Meeting of the Japan Statistical Society, 258–260.
Takane, Y. (1998). Nonlinear multivariate analysis by neural network models. In C. Hayashi, N. Ohsumi, K. Yajima, Y. Tanaka, H.H. Bock & Y. Baba (Eds.), Data science, classification, and related methods: Studies in classification, data analysis, and knowledge organization (pp. 527-538). Tokyo: Springer Verlag.
Takne, Y., Oshima-Takane, Y. & Shultz, T.R. (1994). Approximations of nonlinear functions by feed-forward neural networks. Proceedings of the Eleventh Annual Meeting of the Japan Classification Society, 26–33.
Takane, Y., Oshima-Takane, Y. & Shultz, T.R. (1995). Network analyses: The case of first and second person pronouns. Proceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics. 3594–3599.
Takane, Y. & Shibayama, T. (1991). Principal component analysis with external information on both subjects and variables. Psychometrika, 56, 56–97.
Tetewsky, S.J., Shultz, T.R. & Takane, T. (1995). Training regimens and function compatibility: Implications for understanding the effects of knowledge on concept learning. Proceedings of the Seventeenth Annual Conference of the Cognitive Science Society (pp. 304-309), Hillsdale, NJ: Lawrence Earlbaum Associates.
Author information
Authors and Affiliations
Corresponding author
Additional information
The work reported in this paper has been supported by a team grant from Fonds pour la Formation de Chercheurs et l’Aide a la Recherche to the authors. We thank David Buckingham, Heungsun Hwang, Francois Rivest and Sylvain Sirois for their helpful comments on an earlier draft of this paper.
About this article
Cite this article
Takane, Y., Oshima-Takane, Y. & Shultz, T.R. Analysis of Knowledge Representations in Cascade Correlation Networks. Behaviormetrika 26, 5–28 (1999). https://doi.org/10.2333/bhmk.26.5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.2333/bhmk.26.5