Skip to main content
SpringerLink
Log in

Improved Recurrent Neural Networks for Text Classification and Dynamic Sylvester Equation Solving

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The solution of the text classification and time-varying problems are two basic practical problems frequently encountered in the fields of science and engineering, and most of the text classification and dynamic problems solving are realized by recurrent neural networks (RNN), therefore, the improvement on the convergence and robustness of the RNN models becomes increasingly important. Based on this fact, two novel activation functions (NAF) are proposed to improve the performance of each RNN formula for text classification, dynamic problems solving and dynamic matrix inversion in this work. Firstly, the first NAF (\(\textrm{NAF}_1\)) is applied to the two-layer simple RNN model, long short-term memory RNN model and gated recurrent unit RNN model for text classification. Comparing with the above three RNN models activated by reported activation functions (rectified linear unit (ReLU) function, leak relu (LReLU), exponential linear unit (ELU), scaled ELU], the \(\textrm{NAF}_1\)-activated RNN models achieve higher accuracy in text classification. In addition, based on the second NAF (\(\textrm{NAF}_2\)), an improved fixed-time convergent recurrent neural network (IFTCRNN) model for time-varying problems solving is constructed. The \(\textrm{NAF}_2\)-based IFTCRNN model achieves fixed-time convergence and strong robustness to noises in time-varying Sylvester matrix equation solving, dynamic matrix inversion and robot manipulator trajectory tracking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29

Similar content being viewed by others

References

  1. Sable CL, Hatzivassiloglou V (2000) Text-based approaches for non-topical image categorization. Int J Digit Libr 3(3):261–275

    Article  Google Scholar 

  2. Sanchez-Pi N, Mart L, Garcia ACB (2014) Text classification techniques in oil industry applications. Proceedings of international joint conference SOCO13-CISIS13-ICEUTE13. Springer, pp 211–220

  3. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2):135–168

    Article  MATH  Google Scholar 

  4. Chen M, Jin X, Shen D (2011) Short text classification improved by learning multi-granularity topics. In: Proceedings of the 22nd international joint conference on artificial intelligence. Citeseer, pp 1776–1781

  5. Onan A, Korukoglu S, Bulut H (2016) Ensemble of keyword extraction methods and classifiers in text classification. Expert Syst Appl 57:232–237

    Article  Google Scholar 

  6. Tang B, He H, Baggenstoss P et al (2016) A Bayesian classification approach using class-specific features for text categorization. IEEE Trans Knowl Data Eng 28:1602–1606

    Article  Google Scholar 

  7. Tam S, Setiono R (2002) A comparative study of centroidbased, neighbourhood-based and statistical approaches for effective document categorization. Proceedings of the 16th international conference on pattern recognition (ICPR’02) 4(4):235–238

  8. Prasad R, Kulkarni U, Prasad JR (2009) Machine learning in evolving connectionist text summarizer. In: 3rd International conference on anti-counterfeiting, security, and identification in communication, Hong Kong, pp 539–543

  9. Joachims T (1998) Text Categorization with Support Vector Machines: Learning with many relevant features. Lect Notes Comput Sci 1398:137–142

    Article  Google Scholar 

  10. Zhang T, Oles FJ (2001) Text categorization based on regularized linear classification methods. Inf Retr 4(1):5–31

    Article  MATH  Google Scholar 

  11. Wang P, Xu B, Xu J et al (2016) Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification. Neurocomputing 174:806–814

    Article  Google Scholar 

  12. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882

  13. Lai S, Xu L, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. Proc AAAI 333:2267–2273

    Google Scholar 

  14. Chen H, Sun M, Tu C, Lin Y, Liu Z (2016) Neural sentiment classification with user and product attention. In: Proceedings of the empirical methods in natural language processing,y EMNLP, pp 1650–1659

  15. Zablith F, Osman IH (2019) Review modus, text classification and sentiment prediction of unstructured reviews using a hybrid combination of machine learning and evaluation models. Appl Math Model 71:569–583

    Article  Google Scholar 

  16. Wu M, Yin B, Vosoughi A, Studer C, Cavallaro JR, Dick C (2013) Approximate matrix inversion for high-throughput data detection in the large-scale MIMO uplink. Proc IEEE Int Symp Circuits Syst 54:2155–2158

    Google Scholar 

  17. Zhang Z, Deng X, Kong L, Li S (2020) A circadian rhythms learning network for resisting cognitive periodic noises of time-varying dynamic system and applications to robots. IEEE Trans Cognit Dev Syst 12(3):575–587

    Article  Google Scholar 

  18. Zhang Z, Yang S, Chen S, Luo Y, Yang H, Liu Y (2020) A vector-based constrained obstacle avoidance scheme for wheeled mobile redundant robot manipulator. IEEE Trans Cognit Dev Syst 13:465–474

    Article  Google Scholar 

  19. Li Z, Yuan W, Zhao S, Yu Z, Kang Y, Chen CLP (2019) Brain-actuated control of dual-arm robot manipulation with relative motion. IEEE Trans Cognit Dev Syst 11(1):51–62

    Article  Google Scholar 

  20. Li J, Li Z, Li X, Feng Y, Hu Y, Xu B (2021) Skill learning strategy based on dynamic motion primitives for human–robot cooperative manipulation. IEEE Trans Cognit Dev Syst 13(1):105–117

    Article  Google Scholar 

  21. Jin J, Zhu J, Gong J, Chen W (2022) Novel activation functions-based ZNN models for fixed-time solving dynamic Sylvester equation. Neural Comput Appl 34:14297–14315

  22. Jin J, Zhu J, Zhao L, Chen L (2022) A fixed-time convergent and noise tolerant zeroing neural network for online solution of time-varying matrix inversion. Appl Soft Comput 130:109691

    Article  Google Scholar 

  23. Zhang Y, Ge SS (2005) Design and analysis of a general recurrent neural network model for time-varying matrix inversion. IEEE Trans Neural Networks 16:1477–1490

    Article  Google Scholar 

  24. Zhang Y, Ma W, Cai B (2008) From Zhang neural network to Newton iteration for matrix inversion. IEEE Trans Circuits Syst I Regul Pap 56:1405–1415

    Article  MathSciNet  MATH  Google Scholar 

  25. Zhang Y, Li Z, Li K (2011) Complex-valued Zhang neural network for online complex-valued time-varying matrix inversion. Appl Math Comput Simul Model Pract Theory 217:10066–10073

    MathSciNet  MATH  Google Scholar 

  26. Yang Y, Zhang Y (2013) Superior robustness of power-sum activation functions in Zhang neural networks for time-varying quadratic programs perturbed with large implementation errors. Neural Comput Appl 22:175–185

    Article  Google Scholar 

  27. Zhang Y, Jin L, Ke Z (2012) Superior performance of using hyperbolic sine activation functions in ZNN illustrated via time-varying matrix square roots finding. Comput Inf Syst 9:1603–1625

    Article  Google Scholar 

  28. Li S, Chen S, Liu B (2013) Accelerating a recurrent neural network to finite-time convergence for solving time-varying Sylvester equation by using a sign-bi-power activation function. Neural Process Lett 37:189–205

    Article  Google Scholar 

  29. Li S, Li Y (2014) Nonlinearly activated neural network for solving time-varying complex sylvester equation. IEEE Trans Cybern 44(8):1397–1407

    Article  Google Scholar 

  30. Wei Q, Dobigeon N, Tourneret JY (2015) Fast fusion of multi-band images based on solving a Sylvester equation. IEEE Trans Image Process 24(11):4109–4121

    Article  MathSciNet  MATH  Google Scholar 

  31. Yu S, He Z, Qi T, Wang X (2021) The equivalence canonical form of five quaternion matrices with applications to imaging and Sylvester-type equations. J Comput Appl Math 393:113494

    Article  MathSciNet  MATH  Google Scholar 

  32. Qi Y, Jin L, Li H, Li Y, Liu M (2020) Discrete computational neural dynamics models for solving time-dependent Sylvester equations with applications to robotics and MIMO systems. IEEE Trans Industr Inf 16(10):6231–6241

    Article  Google Scholar 

  33. Xiao L, Cao Y, Dai J, Jia L, Tan H (2021) Finite-time and predefined-time convergence design for zeroing neural network: theorem, method, and verification. IEEE Trans Ind Inf 17(7):4724–4732

    Article  Google Scholar 

  34. Xiao L, Zhang Y, Zuo Q, Dai J, Li J, Tang W (2020) A noise-tolerant zeroing neural network for time-dependent complex matrix inversion under various kinds of noises. IEEE Trans Ind Inf 16(6):3757–3766

    Article  Google Scholar 

  35. Xiao L, Dai J, Lu R, Li S, Li J, Wang S (2020) Design and Comprehensive Analysis of a Noise-Tolerant ZNN Model With Limited-Time Convergence for Time-Dependent Nonlinear Minimization. IEEE Trans Neural Networks Learn Syst 31(12):5339–5348

    Article  MathSciNet  Google Scholar 

  36. Jia L, Xiao L, Dai J, Qi Z, Zhang Z, Zhang Y (2021) Design and application of an adaptive fuzzy control strategy to zeroing neural network for solving time-variant QP problem. IEEE Trans Fuzzy Syst 29(6):1544–1555

    Article  Google Scholar 

  37. Xiao L, Tao J, Dai J, Wang Y, Jia L, Yongjun H (2021) A Parameter-changing and complex-valued zeroing neural-network for finding solution of time-varying complex linear matrix equations in finite time. IEEE Trans Ind Inf 17(10):6634–6643

    Article  Google Scholar 

  38. Yan X, Liu M, Jin L, Li S, Hu B, Zhang X, Huang Z (2019) New zeroing neural network models for solving nonstationary Sylvester equation with verifications on mobile manipulators. IEEE Trans Ind Inf 15(9):5011–5022

    Article  Google Scholar 

  39. Zhang Z, Zheng L, Weng J, Mao Y, Lu W, Xiao L (2018) A new varying-parameter recurrent neural-network for online solution of time-varying Sylvester equation. IEEE Trans Cybern 48(11):3135–3148

    Article  Google Scholar 

  40. Zhang Z et al (2018) A varying-parameter convergent-differential neural network for solving joint-angular-drift problems of redundant robot manipulators. IEEE/ASME Trans Mechatron 23(2):679–689

    Article  Google Scholar 

  41. Zhang Z, Zheng L, Qiu T, Deng F (2020) Varying-Parameter Convergent-Differential Neural Solution to Time-Varying Overdetermined System of Linear Equations. IEEE Trans Autom Control 65(2):874–881

    Article  MathSciNet  MATH  Google Scholar 

  42. Xiao L, Zhang Y (2014) From different Zhang functions to various ZNN models accelerated to finite-time convergence for time-varying linear matrix equation. Neural Process Lett 39:309–326

    Article  Google Scholar 

  43. Zhang Z, Chen T, Wang M, Zheng L (2020) An exponential-type anti-noise varying-gain network for solving disturbed time-varying inversion systems. IEEE Trans Neural Networks Learn Syst 31(9):3414–3427

    Article  MathSciNet  Google Scholar 

  44. Jin J, Gong J (2020) An interference-tolerant fast convergence zeroing neural network for dynamic matrix inversion and its application to mobile manipulator path tracking. Alex Eng J 60:659–669

    Article  Google Scholar 

  45. Jin J, Gong J (2021) A noise-tolerant fast convergence ZNN for dynamic matrix inversion. Int J Comput Math 8:1–19

  46. Chen C, Li L, Peng H, Yang Y, Zhao H (2020) A new fixed-time stability theorem and its application to the fixed-time synchronization of neural networks. Neural Netw 123:412–419

    Article  MATH  Google Scholar 

  47. Zhao L, Jin J, Gong J (2021) Robust zeroing neural network for fixed-time kinematic control of wheeled mobile robot in noise-polluted environment. Math Comput Simul 185:289–307

    Article  MathSciNet  MATH  Google Scholar 

  48. Jin J, Chen W, Zhao L, Chen L, Tang Z (2022) A nonlinear zeroing neural network and its applications on time-varying linear matrix equations solving, electronic circuit currents computing and robotic manipulator trajectory tracking. Comput Appl Math 41: 319

  49. Jin J, Chen W, Qiu L, Zhu J, Liu H (2023) A noise tolerant parameter-variable zeroing neural network and its applications. Math Comput Simul 207:482–498

  50. Jin J, Chen W, Chen C (2022) A predefined fixed-time convergence ZNN and its applications to time-varying quadratic programming solving and dual-arm manipulator cooperative trajectory tracking. IEEE Trans Ind Inf. https://doi.org/10.1109/TII.2022.3220873

  51. Jin J, Qiu L (2022) A robust fast convergence zeroing neural network and its applications to dynamic sylvester equation solving and robot trajectory tracking. J Franklin Inst 359:3183–3209

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant No. 62273141), Natural Science Foundation of Hunan Province (Grant No. 2020JJ4315), Scientific Research Fund of Hunan Provincial Education Department (Grant No. 20B216).

Author information

Authors and Affiliations

  1. School of Information and Electrical Engineering, Hunan University of Science and Technology, Xiangtan, 411201, China

    Weijie Chen, Jie Jin, Lixin Qiu & Jingcan Zhu

  2. Department of Information and Electronic Engineering, International Hellenic University, Thessaloniki, Greece

    Dimitrios Gerontitis

Authors
  1. Weijie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dimitrios Gerontitis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lixin Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jingcan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Weijie Chen or Jie Jin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, W., Jin, J., Gerontitis, D. et al. Improved Recurrent Neural Networks for Text Classification and Dynamic Sylvester Equation Solving. Neural Process Lett 55, 8755–8784 (2023). https://doi.org/10.1007/s11063-023-11176-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11176-6

Keywords

Search

Navigation