Using a general purpose meta neural network to adapt a parameter of the quickpropagation learning rule
This paper proposes a refinement of an application independent method of automating learning rule parameter selection which uses a form of supervisor neural network, known as a Meta Neural Network, to alter the value of a learning rule parameter during training. The Meta Neural Network is trained using data generated by observing the training of a neural network and recording the effects of the selection of various parameter values. The Meta Neural Network is then combined with a normal learning rule to augment its performance. This paper investigates the combination of training sets for different Meta Neural Networks in order to improve the performance of a Meta Neural Network system. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of the Quickpropagation learning rule.
Unable to display preview. Download preview PDF.
- Schiffmann, W. Optimisation of the Backpropagation Algorithm for training multilayer perceptrons. Technical report, Dept of Physics, University of Kobelnz, Germany.ftp://archive.cis.ohio-state.edu/pub/neuroprose/schiff.bp_speedup.ps.Z (1993)Google Scholar
- Riedmiller, M. Advanced supervised learning in multi-layered perceptrons: From Backpropagation to adaptive learning algorithms. Computer Standards and Interfaces. vol 16 part 3. 265–278. (1994)Google Scholar
- Fahlman, S. Faster-Learning variations on Back-Propagation: An Empirical Study. Proceedings of the 1988 Connectionist Models Summer School. Morgan Kaufmann. (1988)Google Scholar
- Prechelt, L. PROBEN1: A set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Dept. of Informatics, University of Karlsruhe, Germany. ftp://ftp.ira.uka.de/pub/neuron/proben1.tar.gz. (1994)Google Scholar
- McCormack, C. A study of the Adaptation of Learning Rule Parameters using a Meta Neural Network. 13th European Meeting on Systems and Cybernetic Research, Vienna. Vol 2, 1043–1048. (1996)Google Scholar
- McCormack, C. Parameter Adaptation using a Meta Neural Network. Proceedings of the World Conference on Neural Networks, Washington D.C., Vol 3, 147–150. (1995)Google Scholar