Recurrent Neural Networks: Some Systems-Theoretic Aspects

  • Mirek Kárný
  • Kevin Warwick
  • Vera Kůrková
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)

Abstract

Recurrent nets have been introduced in control, computation, signal processing, optimization, and associate memory applications. Given matrices A ∈ ℝ n ×n , B ∈ ℝ n ×m , C ∈ ℝ p ×n , as well as a fixed Lipschitz scalar function σ : ℝ → ℝ, the continuous time recurrent network Σ with activation function σ and weight matrices (A, B,C) is given by:
$$\frac{{dx}}{{dt}}(t) = {\overrightarrow \sigma ^{(n)}}\left( {Ax(t) + Bu(t)} \right) , y\left( t \right) = Cx\left( t \right)$$
(1)
where \({\overrightarrow \sigma ^{\left( n \right)}}\): ℝ n → ℝ n is the diagonal map
$${\overrightarrow \sigma ^{\left( n \right)}}:\left( {\begin{array}{*{20}{c}}{{x_1}} \\ \vdots \\ {{x_n}}\end{array}} \right) \mapsto \left({\begin{array}{*{20}{c}}{\sigma \left( {{x_1}} \right)} \\ \vdots \\ {\sigma \left( {{x_n}} \right)} \end{array}} \right)$$
(2)

Keywords

Manifold 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Albertini, F., and P. Dai Pra, “Forward accessibility for recurrent neural networks,” IEEE Trans. Automat. Control 40 (1995): 1962–1968CrossRefMATHMathSciNetGoogle Scholar
  2. [2]
    Albertini, F., and E.D. Sontag, “For neural networks, function determines form,” Neural Networks 8 (1993): 975–990.CrossRefGoogle Scholar
  3. [3]
    Albertini, F., and E.D. Sontag, “Uniqueness of weights for recurrent nets,” Systems and Networks: Math Theory and Applics,Proc. MTNS ‘83, Vol. 2, Akademie Verlag, Regensburg, pp. 599–602. Extended version: http: //www.math.rutgers.edufsontag/FTP DIR/93mtns-nn-extended.ps.gzGoogle Scholar
  4. [4]
    Albertini, F., and E.D. Sontag, “State observability in recurrent neural networks,” Systems ei Control Letters 22 (1994): 235–244.CrossRefMATHMathSciNetGoogle Scholar
  5. [5]
    Hautus, M., “A set of IP-functions,” unpublished manuscript, Eindhoven University, August 1993.Google Scholar
  6. [6]
    Leshno, M., V.Ya. Lin, A. Pinkus, and S. Schocken, “Multilayer feedforward networks with a non-polynomial activation function can approximate any function,” Neural Networks 8 (1993): 861–867.CrossRefGoogle Scholar
  7. [7]
    Siegelmann, H.T., and E.D. Sontag, “Analog computation, neural networks, and circuits,” Theor. Comp. Sci. 131 (1994): 331–360.CrossRefMATHMathSciNetGoogle Scholar
  8. [8]
    Siegelmann, H.T., and E.D. Sontag, “On the computational power of neural nets,” J. Comp. Syst. Sci. 50 (1995): 132–150.CrossRefMATHMathSciNetGoogle Scholar
  9. [9]
    Sontag, E.D., “Neural nets as systems models and controllers,” in Proc. Seventh Yale Workshop on Adaptive and Learning Systems, pp. 73–79, Yale University, 1992.Google Scholar
  10. [10]
    Sontag, E.D., “Neural networks for control,” in Essays on Control: Perspectives in the Theory and its Applications ( H.L. Trentelman and J.C. Willems, eds.), Birkhauser, Boston, 1993, pp. 339–380.Google Scholar
  11. [11]
    Sontag, E.D., Mathematical Control Theory: Deterministic Finite Dimensional Systems, Springer, New York, 1990.MATHGoogle Scholar
  12. [12]
    Sontag, E.D., and H.J. Sussmann, “Complete controllability of continuous-time recurrent neural networks,” Systems and Control Letters 30 (1997): 177–183.CrossRefMATHMathSciNetGoogle Scholar
  13. [13]
    Sussmann, H.J., “Existence and uniqueness of minimal realizations of nonlinear systems,” Math. Sys. Theory 10 (1977): 263–284.CrossRefMathSciNetGoogle Scholar
  14. [14]
    Sussmann, H.J., “Uniqueness of the weights for minimal feedforward nets with a given input-output map,” Neural Networks 5 (1992): 589–593.CrossRefGoogle Scholar
  15. [15]
    Zbikowski, R., “Lie algebra of recurrent neural networks and identifiability,” Proc. Amer. Auto. Control Conf., San Francisco, 1993, pp. 2900–2901.Google Scholar

Copyright information

© Springer-Verlag London Limited 1998

Authors and Affiliations

  • Mirek Kárný
    • 1
  • Kevin Warwick
    • 2
  • Vera Kůrková
    • 3
  1. 1.Institute of Information Theory & AutomationPrague 8Czech Republic
  2. 2.Department of CyberneticsUniversity of ReadingWhiteknights, ReadingUK
  3. 3.Institute of Computer ScienceAcademy of Sciences of the Czech RepublicPrague 8Czech Republic

Personalised recommendations