Abstract
We consider the modeling process of a “biological” agent by combining the concepts of neuroinformatics and deterministic chaos. We assume that an agent observes a target process as a stochastic symbolic process, which is restricted by grammatical constraints. Our main hypothesis is that an agent would learn the target model by reconstructing an equivalent quasi-stochastic process on its deterministic neural dynamics. We employed a recurrent neural network (RNN), which is regarded as an adjustable deterministic dynamical system. Then, we conducted an experiment to observe how the RNN learns to reconstruct the target process, represented by a stochastic finite state machine in the simulation. The result revealed the capability of the RNN to evolve, by means of learning, toward chaos, which is able to mimic a target's stochastic process. We precisely analyzed the evolutionary process as well as the internal representation of the neural dynamics obtained. This analysis enabled us to clarify an interesting mechanism of the self-organization of chaos by means of neural learning, and also showed how grammar can be embedded in the evolved deterministic chaos.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Crutchfield JP (1991) Semantics and thermodynamics. In: Casdagli M, Eubank S (eds) Non-linear moedling. Addison-Wesley, Reading, pp 1–36
Crutchfield JP, Young K (1989) Inferring statistical complexity. Phys Rev Lett 63:105–108
Elman J (1990) Finding structure in time. Cogn Sci 14:179–211
Friedman EJ (1991) Structure and uncomputability in one-dimensional maps. Complex Systems 5:335–350
Giles CL, Miller CB, Chen D, Chen HH, Sun GZ, Lee YC (1992) Learning and extracting finite state automata with second order recurrent neural networks. Neural Comp 4:393–405
Hao B-L (1989) Elementary symbolic dynamics and chaos in dissipative system. World Scientific, Singapore
Jordan M (1986) Attractor dynamics and parallelism in a connectionist sequence machine. In: Proc of Ninth Annual conference of Cognitive Science Society. Lawrence Earlbaum, New York, pp 531–546
Pineda FJ (1987) Generalization of back-propagation to recurrent neural networks. Phys Rev Lett 59:2229–2232
Pollack JB (1991) Higher order recurrent networks and grammatical inference. Machine Learning 7:227–252
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McLelland JL (eds) Parallel distributed processing. MIT Press, Cambridge, Mass, pp 318–362
Sato M (1990) A learning algorithm to teach spatiotemperal patterns to recurrent neural networks. Biol Cybern 62:259–263
Sato M, Murakami Y, Joe K (1990) Learning chaotic dynamics by recurrent neural networks. In: Proc. of Int. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Japan, pp 601–604
Wiggins S (1990) Introduction to applied nonlinear dynamical systems and chaos. Springer, Berlin Heidelberg New York, pp 420–438
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Tani, J., Fukumura, N. Embedding a grammatical description in deterministic chaos: an experiment in recurrent neural learning. Biol. Cybern. 72, 365–370 (1995). https://doi.org/10.1007/BF00202792
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF00202792