Neural Processing Letters

, Volume 2, Issue 4, pp 26–30 | Cite as

On the search for new learning rules for ANNs

  • Samy Bengio
  • Yoshua Bengio
  • Jocelyn Cloutier


In this paper, we present a framework where a learning rule can be optimized within a parametric learning rule space. We define what we callparametric learning rules and present a theoretical study of theirgeneralization properties when estimated from a set of learning tasks and tested over another set of tasks. We corroborate the results of this study with practical experiments.


Neural Network Artificial Intelligence Complex System Nonlinear Dynamics Practical Experiment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    D. Chalmers. The evolution of learning: An experiment in genetic connectionism, inProceedings of the 1990 Connectionist Models Summer School, J. Hinton, ed., San Mateo, CA, Morgan Kaufmann, 1990.Google Scholar
  2. [2]
    Y.Bengio, S. Bengio. Learning a synaptic learning rule,Tech. Rep. 751, Dép. d'Informatique et de Recherche Opérationnelle, Université de Montréal, Montreal (QC) Canada, 1990.Google Scholar
  3. [3]
    V. N. Vapnik.Estimation of dependencies based on empirical data, Springer-Verlag, New-York, NY, USA, 1982.Google Scholar
  4. [4]
    J. Holland.Adaptation in Natural and Artificial Systems, University of Michigan Press, 1975.Google Scholar
  5. [5]
    S. Kirkpatrick, C. D. Gelatt, M.P. Vecchi, Optimization by simulated annealing,Science, no. 220, pp. 671–680, 1983.Google Scholar
  6. [6]
    J. R. Koza.Genetic programming: on the programming of computers by means of natural selection, Bradford Book, MIT Press, Cambridge, MA, USA, 1992.Google Scholar
  7. [7]
    S. Bengio, Y. Bengio, J. Cloutier, J. Gecsei. On the optimization of a synaptic learning rule, inConference on Optimality in Biological and Artificial Networks, Dallas, USA, 1992.Google Scholar

Copyright information

© Kluwer Academic Publishers 1995

Authors and Affiliations

  • Samy Bengio
    • 1
  • Yoshua Bengio
    • 2
  • Jocelyn Cloutier
    • 2
  1. 1.LAB/RIO/TNT, France Télécom CNETLannion CedexFrance
  2. 2.Département IROUniversité de MontréalMontréalCanada

Personalised recommendations