Increased complexity training
The training strategy used in connectionist learning has not received much attention in the literature. We suggest a new strategy for backpropagation learning, increased complexity training, and show experimentally that it leads to faster convergence compared to both the conventional training strategy using a fixed set, and to combined subset training. Increased complexity training combined with an incremental increase in the success ratio required on the training set produced even quicker convergence.
Unable to display preview. Download preview PDF.
- [Cottrell 91]Cottrell, G.W., and Tsung, F.S., “Learning Simple Arithmetic Procedures”, High-Level Connectionist Models, eds. J.A. Barnden, J.B. Pollack, in the series Advances in Connectionist and Neural Computation Theory, Vol. 1, pp.305–321, 1991.Google Scholar
- [Elman 90]Elman, J.L., “Finding Structure in Time”, Cognitive Science, Vol. 14, pp. 179–211, 1990.Google Scholar
- [Fahlman 89]Faldman, S.E., “Faster Learning Variations on Back-propagation: An Empirical Study”, Proceedings of the 1988 Connectionist Models Summer School, Morgan Kaufmann Publishers, pp. 38–50, 1989.Google Scholar
- [Hoekstra 92]Hoekstra, J., “Is Counting with Artificial Neural Networks a Problem?”, Proceedings of the 1st IFIP Working Group-10.6 Workshop, INPG, Grenoble, France, pp. 27–30, 2–3 March 1992.Google Scholar
- [Rumelhart 86]Rumelhart, D.E., Hinton G.E., Williams R.J., “Learning Internal Representations by Error Propagation”, Parallel Distributed Processing: Vol. 1, Foundations, eds. Rumelhart, D.E., McClelland, J.L., Cambridge, MA: MIT Press, 1986.Google Scholar
- [Silva 90]Silva, F.M., and Almeida, L.B., “Speeding up Backpropagation”, Advanced Neural Computers, ed. R. Eckmiller, Elsevier Science Publishers B.V., North Holland, pp. 151–158, 1990.Google Scholar
- [Waibel 89]Waibel, A., “Consonant Recognition by Modular Construction of Large Phonemic Time-Delay Neural Networks”, Advances in Neural Information Processing Systems 1, ed. D.S. Touretzky, Morgan Kaufmann Publishers, pp. 215–223, 1989.Google Scholar