Skip to main content
SpringerLink
Log in

Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Bartels–Stewart algorithm is an effective and widely used method with an O(n 3) time complexity for solving a static Sylvester equation. When applied to time-varying Sylvester equation, the computation burden increases intensively with the decrease of sampling period and cannot satisfy continuous realtime calculation requirements. Gradient-based recurrent neural network are able to solve the time-varying Sylvester equation in real time but there always exists an estimation error. In contrast, the recently proposed Zhang neural network has been proven to converge to the solution of the Sylvester equation ideally when time goes to infinity. However, this neural network with the suggested activation functions never converges to the desired value in finite time, which may limit its applications in realtime processing. To tackle this problem, a sign-bi-power activation function is proposed in this paper to accelerate Zhang neural network to finite-time convergence. The global convergence and finite-time convergence property are proven in theory. The upper bound of the convergence time is derived analytically. Simulations are performed to evaluate the performance of the neural network with the proposed activation function. In addition, the proposed strategy is applied to online calculating the pseudo-inverse of a matrix and nonlinear control of an inverted pendulum system. Both theoretical analysis and numerical simulations validate the effectiveness of proposed activation function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bhatia R, Rosenthal P (1997) How and why to solve the operator equation AX-XB = Y. Bull Lond Math Soc 29(1): 1–21

    Article  MathSciNet  MATH  Google Scholar 

  2. Harker M, Leary PO (2011) Least squares surface reconstruction from gradients: direct algebraic methods with spectral, tikhonov, and constrained regularization. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), Colorado Springs, pp 2529–2536

  3. Syrmos VL (1994) Disturbance decoupling using constrained sylvester equations. IEEE Trans Auto Control 39(4): 797–803

    Article  MathSciNet  MATH  Google Scholar 

  4. Syrmos VL, Lewis FL (1993) Output feedback eigenstructure assignment using two sylvester equations. IEEE Trans Auto Control 38(3): 495–499

    Article  MathSciNet  MATH  Google Scholar 

  5. Lee HC, Choi JW (2004) Linear time-varying eigenstructure assignment with flight control application. IEEE Trans Aerosp Electron Syst 40(1): 145–157

    Article  Google Scholar 

  6. Bartels RH, Stewart GW (1972) Solution of the matrix equation AX + XB = C. Commun ACM 15: 820–826

    Article  Google Scholar 

  7. Stykel T (2002) Numerical solution and perturbation theory for generalized lyapunov equations. Linear Algebra Appl 349(1–3): 155–185

    Article  MathSciNet  MATH  Google Scholar 

  8. Monsalve M (00 2) 2008) Block linear method for large scale Sylvester equations. Comput Appl Math 27: 47–59

    Google Scholar 

  9. Skowronski MD, Harris JG (2007) Noise-robust automatic speech recognition using a predictive echo state network. IEEE Trans Audio Speech Lang Process 15(5): 1724–1730

    Article  Google Scholar 

  10. Burrows N, Niranjan M (1994) The use of recurrent neural networks for classification. In: IEEE workshop on neural networks for signal processing IV, Scotland, pp 117–125

  11. Husken M, Stagge P (2003) Recurrent neural networks for time series classification. Neurocomputing 50: 223–235

    Article  Google Scholar 

  12. Lin S, Li Y, Liu B (2012) Model-free control of lorenz chaos using an approximate optimal control strategy. Commun Nonlinear Sci Numer Simul 17: 12

    Article  Google Scholar 

  13. Smith KA (1999) Neural networks for combinatorial optimization: a review of more than a decade of research. INFORMS J Comput 11: 15–34

    Article  MathSciNet  MATH  Google Scholar 

  14. Li S, Lou Y, Liu B (2012) Bluetooth aided mobile phone localization: a nonlinear neural circuit approach. ACM Trans Embedded Comput Syst (accepted)

  15. Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc Natl Acad Sci 81(10): 3088–3092

    Article  Google Scholar 

  16. Ding F, Chen T (2005) Gradient based iterative algorithms for solving a class of matrix equations. IEEE Trans Auto Control 50(8): 1216–1221

    Article  MathSciNet  Google Scholar 

  17. Zhou B, Li ZY, Duan GR, Wang Y (2009) Least squares solutions to general coupled sylvester matrix equations. J Comput Appl Math 224: 759–776

    Article  MathSciNet  MATH  Google Scholar 

  18. Manherz R, Jordan B, Hakimi S (1968) Analog methods for computation of the generalized inverse. IEEE Trans Auto Control 13(5): 582–585

    Article  Google Scholar 

  19. Jang JS, Lee SY, Shin SY (1987) An optimization network for matrix inversion. In: Proceedings of NIPS’87, Graz, pp 397–401

  20. Zhang Y, Ge SS (2005) Design and analysis of a general recurrent neural network model for time-varying matrix inversion. IEEE Trans Neural Netw 16(6): 1477–1490

    Article  Google Scholar 

  21. Xiao L, Zhang Y (2011) Zhang neural network versus gradient neural network for solving time-varying linear inequalities. IEEE Trans Neural Netw 22(10): 1676–1684

    Article  MathSciNet  Google Scholar 

  22. Zhang Y, Ma W, Cai B (2009) From zhang neural network to newton iteration for matrix inversion. IEEE Trans Circuits Syst I 56(7): 1405–1415

    Article  MathSciNet  Google Scholar 

  23. Zhang Y, Yi C, Guo D, Zheng J (2011) Comparison on zhang neural dynamics and gradient-based neural dynamics for online solution of nonlinear time-varying equation. Neural Comput Appl 20: 1–7

    Article  Google Scholar 

  24. Haykin Simon S (1999) Neural networks: a comprehensive foundation, 2 edn. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  25. Bhat SP, Bernstein DS (2000) Finite-time stability of continuous autonomous systems. SIAM J Control Optim 38: 751–766

    Article  MathSciNet  MATH  Google Scholar 

  26. Zhang Y, Fan Z, Li Z (2007) Zhang neural network for online solution of time-varying sylvester equation. In: Proceedings of the 2nd international conference on advances in computation and intelligence, ISICA’07. Berlin, Springer, pp 276–285

  27. Zhang Y, Ge S (2003) A general recurrent neural network model for time-varying matrix inversion, vol 6. In: Proceedings of 42nd IEEE conference on decision and control, San Diego, pp 6169 – 6174

  28. Li Z, Zhang Y (2010) Improved Zhang neural network model and its solution of time-varying generalized linear matrix equations. Expert Syst Appl 37: 7213–7218

    Article  Google Scholar 

  29. Khalil HK (2001) Nonlinear Systems, 3rd edn. Prentice Hall, Englewood Cliffs

    Google Scholar 

  30. Bertsekas DP, Tsitsiklis J.N. (1996) Neuro-dynamic programming, optimization and neural computation series, 3. Athena Scientific, Belmont

  31. Kamruzzaman J, Aziz SM (2002) A note on activation function in multilayer feedforward learning. In: Proceedings of the 2002 international joint conference on neural networks 2002, IJCNN ’02, vol 1, Honolulu, pp 519–523

  32. Stephen LC, Carl DM (2009) Generalized inverses of linear transformations. Society for Industrial and Applied Mathematics, Philadephia

    MATH  Google Scholar 

  33. Banerjee KS (1980) Generalized inverse of matrices and its applications. Technometrics 15(1): 471–512

    Google Scholar 

  34. Myers RH (1994) Classical and modern regression with applications, 2nd edn. PWS-KENT Publishing Company, Boston

    Google Scholar 

  35. Buss SR (2004) Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods, Technical report. pp 1–19

  36. Steriti RJ, Fiddy MA (1993) Regularized image reconstruction using svd and a neural network method for matrix inversion. IEEE Trans Signal Process 41(10): 3074–3077

    Article  MATH  Google Scholar 

  37. Zhang Y (2005) Revisit the analog computer and gradient-based neural system for matrix inversion. In: Proceedings of the 2005 IEEE international symposium on, mediterrean conference on control and automation intelligent control 2005, Limassol, pp 1411–1416

  38. Wang J (1993) A recurrent neural network for real-time matrix inversion. Appl Math Comput 55(1): 89–100

    Article  MathSciNet  MATH  Google Scholar 

  39. Bhattacharyya SP, Souza ED (1982) Pole assignment via Sylvester’s equation. Syst Control Lett 1(4): 261–263

    Article  MATH  Google Scholar 

  40. Chen CT (1998) Linear system theory and design, 3rd edn. Oxford University Press, New York

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, 07030, USA

    Shuai Li

  2. Key Lab of Visual Media Processing and Transmission, Shenzhen Institute of Information Technology, Shenzhen, 518029, Guangdong, China

    Sanfeng Chen

  3. Department of Computer Science, University of Massachusetts, Amherst, MA, 01003, USA

    Bo Liu

Authors
  1. Shuai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanfeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Li.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, S., Chen, S. & Liu, B. Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function. Neural Process Lett 37, 189–205 (2013). https://doi.org/10.1007/s11063-012-9241-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9241-1

Keywords

Search

Navigation