Recurrent Neural Networks: Some Systems-Theoretic Aspects

  • Mirek Kárný
  • Kevin Warwick
  • Vera Kůrková
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)


Recurrent nets have been introduced in control, computation, signal processing, optimization, and associate memory applications. Given matrices A ∈ ℝ n ×n , B ∈ ℝ n ×m , C ∈ ℝ p ×n , as well as a fixed Lipschitz scalar function σ : ℝ → ℝ, the continuous time recurrent network Σ with activation function σ and weight matrices (A, B,C) is given by:
$$\frac{{dx}}{{dt}}(t) = {\overrightarrow \sigma ^{(n)}}\left( {Ax(t) + Bu(t)} \right) , y\left( t \right) = Cx\left( t \right)$$
where \({\overrightarrow \sigma ^{\left( n \right)}}\): ℝ n → ℝ n is the diagonal map
$${\overrightarrow \sigma ^{\left( n \right)}}:\left( {\begin{array}{*{20}{c}}{{x_1}} \\ \vdots \\ {{x_n}}\end{array}} \right) \mapsto \left({\begin{array}{*{20}{c}}{\sigma \left( {{x_1}} \right)} \\ \vdots \\ {\sigma \left( {{x_n}} \right)} \end{array}} \right)$$


Recurrent Neural Network Admissible Pair Recurrent Network Minimal Realization Independence Property 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Albertini, F., and P. Dai Pra, “Forward accessibility for recurrent neural networks,” IEEE Trans. Automat. Control 40 (1995): 1962–1968CrossRefMATHMathSciNetGoogle Scholar
  2. [2]
    Albertini, F., and E.D. Sontag, “For neural networks, function determines form,” Neural Networks 8 (1993): 975–990.CrossRefGoogle Scholar
  3. [3]
    Albertini, F., and E.D. Sontag, “Uniqueness of weights for recurrent nets,” Systems and Networks: Math Theory and Applics,Proc. MTNS ‘83, Vol. 2, Akademie Verlag, Regensburg, pp. 599–602. Extended version: http: //www.math.rutgers.edufsontag/FTP DIR/ Scholar
  4. [4]
    Albertini, F., and E.D. Sontag, “State observability in recurrent neural networks,” Systems ei Control Letters 22 (1994): 235–244.CrossRefMATHMathSciNetGoogle Scholar
  5. [5]
    Hautus, M., “A set of IP-functions,” unpublished manuscript, Eindhoven University, August 1993.Google Scholar
  6. [6]
    Leshno, M., V.Ya. Lin, A. Pinkus, and S. Schocken, “Multilayer feedforward networks with a non-polynomial activation function can approximate any function,” Neural Networks 8 (1993): 861–867.CrossRefGoogle Scholar
  7. [7]
    Siegelmann, H.T., and E.D. Sontag, “Analog computation, neural networks, and circuits,” Theor. Comp. Sci. 131 (1994): 331–360.CrossRefMATHMathSciNetGoogle Scholar
  8. [8]
    Siegelmann, H.T., and E.D. Sontag, “On the computational power of neural nets,” J. Comp. Syst. Sci. 50 (1995): 132–150.CrossRefMATHMathSciNetGoogle Scholar
  9. [9]
    Sontag, E.D., “Neural nets as systems models and controllers,” in Proc. Seventh Yale Workshop on Adaptive and Learning Systems, pp. 73–79, Yale University, 1992.Google Scholar
  10. [10]
    Sontag, E.D., “Neural networks for control,” in Essays on Control: Perspectives in the Theory and its Applications ( H.L. Trentelman and J.C. Willems, eds.), Birkhauser, Boston, 1993, pp. 339–380.Google Scholar
  11. [11]
    Sontag, E.D., Mathematical Control Theory: Deterministic Finite Dimensional Systems, Springer, New York, 1990.MATHGoogle Scholar
  12. [12]
    Sontag, E.D., and H.J. Sussmann, “Complete controllability of continuous-time recurrent neural networks,” Systems and Control Letters 30 (1997): 177–183.CrossRefMATHMathSciNetGoogle Scholar
  13. [13]
    Sussmann, H.J., “Existence and uniqueness of minimal realizations of nonlinear systems,” Math. Sys. Theory 10 (1977): 263–284.CrossRefMathSciNetGoogle Scholar
  14. [14]
    Sussmann, H.J., “Uniqueness of the weights for minimal feedforward nets with a given input-output map,” Neural Networks 5 (1992): 589–593.CrossRefGoogle Scholar
  15. [15]
    Zbikowski, R., “Lie algebra of recurrent neural networks and identifiability,” Proc. Amer. Auto. Control Conf., San Francisco, 1993, pp. 2900–2901.Google Scholar

Copyright information

© Springer-Verlag London Limited 1998

Authors and Affiliations

  • Mirek Kárný
    • 1
  • Kevin Warwick
    • 2
  • Vera Kůrková
    • 3
  1. 1.Institute of Information Theory & AutomationPrague 8Czech Republic
  2. 2.Department of CyberneticsUniversity of ReadingWhiteknights, ReadingUK
  3. 3.Institute of Computer ScienceAcademy of Sciences of the Czech RepublicPrague 8Czech Republic

Personalised recommendations