A Primal Neural Network for Online Equality-Constrained Quadratic Programming

Article
  • 43 Downloads

Abstract

This paper aims at solving online equality-constrained quadratic programming problem, which is widely encountered in science and engineering, e.g., computer vision and pattern recognition, digital signal processing, and robotics. Recurrent neural networks such as conventional GradientNet and ZhangNet are considered as powerful solvers for such a problem in light of its high computational efficiency and capability of circuit realisation. In this paper, an improved primal recurrent neural network and its electronic implementation are proposed and analysed. Compared to the existing recurrent networks, i.e. GradientNet and ZhangNet, our network can theoretically guarantee superior global exponential convergence. Robustness performance of our such neural model is also analysed under a large model implementation error, with the upper bound of stead-state solution error estimated. Simulation results demonstrate theoretical analysis on the proposed model, which also verify the effectiveness of the proposed model for online equality-constrained quadratic programming.

Keywords

Recurrent neural networks Online equality-constrained quadratic programming Global exponential convergence Robustness analysis 

Notes

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 61773375, Grant 61375036, and Grant 61511130079, and in part by the Microsoft Collaborative Research Project, and by the Academy of Finland under No.298700.

Compliance with Ethical Standards

Conflict of interests

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. 1.
    Chen K, Jia K, Zhang Z, Kämäräinen JK. Spectral attribute learning for visual regression. Pattern Recogn. 2017; (66):74–81. in press.Google Scholar
  2. 2.
    Chen K, Loy CC, Gong S, Xiang T. Feature mining for localised crowd counting. British Machine Vision Conference; 2012. p. 21.1–21.11.Google Scholar
  3. 3.
    Chen K, Gong S, Xiang T, Loy CC. Cumulative attribute space for age and crowd density estimation. IEEE Conference on Computer Vision and Pattern Recognition; 2013. p. 2467–2474.Google Scholar
  4. 4.
    Chen K, Tuhtan JA, Fuentes-Pérez JF, Toming G, Musall M, Strokina N, Kämäräinen JK, Kruusmaa M. Estimation of flow turbulence metrics with a lateral line probe and regression. IEEE Trans Instrum Meas 2017;66(4):651–60.CrossRefGoogle Scholar
  5. 5.
    Leithead W, Zhang Y. O(N 2)-operation approximation of covariance matrix inverse in Gaussian process regression based on quasi-Newton BFGS method. Commun Stat-Simul Comput 2007;36(2):367–80.CrossRefGoogle Scholar
  6. 6.
    Chen K, Zhang L, Zhang Y. Cyclic motion generation of multi-link planar robot performing square end-effector trajectory analyzed via gradient-descent and Zhang et al’s neural-dynamic methods. International Symposium on Systems and Control in Aerospace and Astronautics; 2008. p. 1–6.Google Scholar
  7. 7.
    Wang J, Zhang Y. Recurrent neural networks for real-time computation of inverse kinematics of redundant manipulators. Machine Intelligence: Quo Vadis. 2004;299–319.Google Scholar
  8. 8.
    Zhang Y. A set of nonlinear equations and inequalities arising in robotics and its online solution via a primal neural network. Neurocomputing 2006;70(1):513–24.CrossRefGoogle Scholar
  9. 9.
    Zhang Y, Li K. Bi-criteria velocity minimization of robot manipulators using LVI-based primal-dual neural network and illustrated via PUMA560 robot arm. Robotica 2010;28(4):525–37.CrossRefGoogle Scholar
  10. 10.
    Zhang Y, Ma W, Li XD, Tan HZ, Chen K. Matlab simulink modeling and simulation of LVI-based primal–dual neural network for solving linear and quadratic programs. Neurocomputing 2009;72(7):1679–87.CrossRefGoogle Scholar
  11. 11.
    Suykens J, Vandewalle J. Least squares support vector machine classifiers. Neural Process Lett 1999;9(3): 293–300.CrossRefGoogle Scholar
  12. 12.
    Suykens J, Van Gestel T, De Brabanter J, De Moor B, Vandewalle J, Suykens J, Van Gestel T. 2002. Least squares support vector machines. vol 4 World Scientific.Google Scholar
  13. 13.
    Wang Z, Chen S. New least squares support vector machines based on matrix patterns. Neural Process Lett 2007;26(1):41–56.CrossRefGoogle Scholar
  14. 14.
    Chapelle O. Training a support vector machine in the primal. Neural Comput 2007;19(5):1155–78.CrossRefPubMedGoogle Scholar
  15. 15.
    Zhang Y, Leithead WE, Leith DJ. Time-series Gaussian process regression based on Toeplitz computation of O(N 2) operations and O(N)-level storage. IEEE Conference on Decision and Control; 2005. p. 3711–3716.Google Scholar
  16. 16.
    Hopfield JJ, Tank DW. Neural computation of decisions in optimization problems. Biol Cybern 1985;52(3): 141–52.PubMedGoogle Scholar
  17. 17.
    Wang J. Recurrent neural network for solving quadratic programming problems with equality constraints. Electron Lett 1992;28(14):1345–7.CrossRefGoogle Scholar
  18. 18.
    Zhang Y. Towards piecewise-linear primal neural networks for optimization and redundant robotics. IEEE International Conference on Networking, Sensing and Control; 2006. p. 374–379.Google Scholar
  19. 19.
    Zhang Y, Li Z. Zhang neural network for online solution of time-varying convex quadratic program subject to time-varying linear-equality constraints. Phys Lett A 2009;373(18):1639–43.CrossRefGoogle Scholar
  20. 20.
    Zhang Y, Yang Y, Ruan G. Performance analysis of gradient neural network exploited for online time-varying quadratic minimization and equality-constrained quadratic programming. Neurocomputing 2011;74(10): 1710–9.CrossRefGoogle Scholar
  21. 21.
    Chen K. Recurrent implicit dynamics for online matrix inversion. Appl Math Comput 2013;219(20):10218–24.Google Scholar
  22. 22.
    Chen K, Yi C. Robustness analysis of a hybrid of recursive neural dynamics for online matrix inversion. Appl Math Comput 2016;273:969–75.Google Scholar
  23. 23.
    Chen K. Improved neural dynamics for online sylvester equations solving. Inf Process Lett 2016;116(7):455–9.CrossRefGoogle Scholar
  24. 24.
    Chen K. Robustness analysis of Wang neural network for online linear equation solving. Electron Lett 2012;48 (22):1391– 2.CrossRefGoogle Scholar
  25. 25.
    Chen K. Implicit dynamic system for online simultaneous linear equations solving. Electron Lett 2013;49(2): 101–2.CrossRefGoogle Scholar
  26. 26.
    Zhang Y, Chen K, Tan HZ. Performance analysis of gradient neural network exploited for online time-varying matrix inversion. IEEE Trans Autom Control 2009;54(8):1940–5.CrossRefGoogle Scholar
  27. 27.
    Zhang Y, Li S, Zhang X. Simulink comparison of varying-parameter convergent-differential neural-network and gradient neural network for solving online linear time-varying equations. World Congress on Intelligent Control and Automation; 2016. p. 887–894.Google Scholar
  28. 28.
    Zhang Z, Chen S, Zheng L, Zhang J. Matlab Simulink of varying-parameter convergent-differential neural-network for solving online time-varying matrix inverse. International Symposium on Computational Intelligence and Design; 2016. p. 320–325.Google Scholar
  29. 29.
    Chen K, Guo D, Tan Z, Yang Z, Zhang Y. Cyclic motion planning of redundant robot arms: simple extension of performance index may not work. International Symposium on Intelligent Information Technology Application; 2008. p. 635– 639.Google Scholar
  30. 30.
    Mead C, Ismail M. 1989. Analog VLSI implementation of neural systems. Springer Science & Business Media.Google Scholar
  31. 31.
    Zhang Z, Li Z, Zhang Y, Luo Y, Li Y. Neural-dynamic-method-based dual-arm CMG scheme with time-varying constraints applied to humanoid robots. IEEE Trans Neural Netw Learn Syst 2015;26(12):3251–62.CrossRefPubMedGoogle Scholar
  32. 32.
    Zhang Z, Zhang Y. Equivalence of different-level schemes for repetitive motion planning of redundant robots. Acta Automatica Sinica 2013;39(1):88–91.CrossRefGoogle Scholar
  33. 33.
    Zhang Y, Yang Y, Ruan G. Performance analysis of gradient neural network exploited for online time-varying quadratic minimization and equality-constrained quadratic programming. Neurocomputing 2011;74(10): 1710–9.CrossRefGoogle Scholar
  34. 34.
    Zhang Y, Ge SS. Design and analysis of a general recurrent neural network model for time-varying matrix inversion. IEEE Trans Neural Netw 2005;16(6):1477–90.CrossRefPubMedGoogle Scholar
  35. 35.
    Chen K, Zhang Z. An Improved Recurrent Network for Online Equality-Constrained Quadratic Programming. Advances in Brain Inspired Cognitive Systems; 2016. p. 1–10.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Lab of Signal ProcessingTampere University of TechnologyTampereFinland
  2. 2.CAS Center for Excellence in Brain Science and Intelligence Technology (CEBSIT)BeijingChina
  3. 3.National Laboratory of Pattern RecognitionInstitute of Automation, Chinese Academy of Sciences (NLPR, CASIA)BeijingChina
  4. 4.University of Chinese Academy of Sciences (UCAS)BeijingChina

Personalised recommendations