Computational Models of Neurons and Neural Nets

From Natural to Artificial Neural Computation

Volume 930 of the series Lecture Notes in Computer Science pp 195-201

Date:

The influence of the sigmoid function parameters on the speed of backpropagation learning

  • Jun HanAffiliated withResearch Group Computational Intelligence Dept. of Computer Science, University of Dortmund
  • , Claudio MoragaAffiliated withResearch Group Computational Intelligence Dept. of Computer Science, University of Dortmund

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Sigmoid function is the most commonly known function used in feed forward neural networks because of its nonlinearity and the computational simplicity of its derivative. In this paper we discuss a variant sigmoid function with three parameters that denote the dynamic range, symmetry and slope of the function respectively. We illustrate how these parameters influence the speed of backpropagation learning and introduce a hybrid sigmoidal network with different parameter configuration in different layers. By regulating and modifying the sigmoid function parameter configuration in different layers the error signal problem, oscillation problem and asymmetrical input problem can be reduced. To compare the learning capabilities and the learning rate of the hybrid sigmoidal networks with the conventional networks we have tested the two-spirals benchmark that is known to be a very difficult task for backpropagation and their relatives.