Neural Processing Letters

, Volume 38, Issue 1, pp 1-15

First online:

Selective Recurrent Neural Network

  • Branko ŠterAffiliated withFaculty of Computer and Information Science, University of Ljubljana Email author 

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


It is known that recurrent neural networks may have difficulties remembering data over long time lags. To overcome this problem, we propose an extended architecture of recurrent neural networks, which is able to deal with long time lags between relevant input signals. A register of latches at the input layer of the network is applied to bypass irrelevant input information and to propagate relevant inputs. The latches are implemented with differentiable multiplexers, thus enabling the derivatives to be propagated through the network. The relevance of input vectors is learned concurrently with the weights of the network using a gradient-based algorithm.


Recurrent neural networks Temporal processing Long-term dependencies Latch Multiplexer Finite state automata