Research on motion pattern recognition of exoskeleton robot based on multimodal machine learning model

  • Yi Zheng
  • Qingjun SongEmail author
  • Jixin Liu
  • Qinghui Song
  • Qingchao Yue
Deep Learning & Neural Computing for Intelligent Sensing and Control


Exoskeleton as a real-time interaction with the wearer’s intelligent robot, in recent years, becomes a hot topic mouth class research in the field of robotics. Wearable exoskeleton outside the body, combined with the organic body, plays a role in the protection and support. By wearing an exoskeleton robot, it is possible to expand the wearer’s athletic ability, increase muscle endurance, and enable the wearer to complete tasks that he or she cannot perform under natural conditions. Based on the above advantages, the exoskeleton robot in military medical care and rehabilitation has broad application prospects. This paper describes the multimodal model of machine learning research status and research significance of the text on the exoskeleton robot applications, and on the basis of a detailed study of gait. It mainly involves: analysis and planning and obtaining motion information processing, pattern recognition and analysis of gait and the gait conversion process, and the EEG and joint position, foot pressure, such as different modes of data as input to machine learning models to improve the timeliness, accuracy and safety of gait planning. Since the common movement process involves the transformation process of gait, this paper studies the gait transformation process including squatting, walking on the ground and standing.


Exoskeleton robot Motion pattern recognition Multimodal Machine learning model 



This work was supported by the National Natural Science Foundation of China (No. 51674155), the Key Research and Development Plan of Shandong Province (No. 2018GGX106001), and Shandong Provincial College Science and Technology Planning project (J18KA009).

Compliance with ethical standards

Conflict of interest

The authors declared that they have no conflicts of interest to this work.


  1. 1.
    Shore L, Power V, de Eyto A, O’Sullivan L (2018) Technology acceptance and user-centred design of assistive exoskeletons for older adults: a commentary. Robotics 7(1):3CrossRefGoogle Scholar
  2. 2.
    Yang C, Huaiwei W, Li Z, He W, Wang N, Chun-Yi S (2018) Mind control of a robotic arm with visual fusion technology. IEEE Trans Ind Inform 14(9):3822–3830CrossRefGoogle Scholar
  3. 3.
    Wright FD, Conte TM (2018) Standards: roadmapping computer technology trends enlightens industry. Computer 51(6):100–103CrossRefGoogle Scholar
  4. 4.
    Faria C, Erlhagen W, Rito M et al (2015) Review of robotic technology for stereotactic neurosurgery. IEEE Rev Biomed Eng 8:125–137CrossRefGoogle Scholar
  5. 5.
    Wiederhold BK (2017) Robotic technology remains a necessary part of healthcare’s future editorial. Cyberpsychol Behav Soc Netw 20(9):511–512CrossRefGoogle Scholar
  6. 6.
    Suri RM, Dearani JA, Mihaljevic T et al (2016) Mitral valve repair using robotic technology: safe, effective, and durable. J Thorac Cardiovasc Surg 151(6):1450–1454CrossRefGoogle Scholar
  7. 7.
    Kim YS, Lee J, Lee S et al (2005) A force reflected exoskeleton-type masterarm for human–robot interaction. IEEE Trans Syst Man Cybern A Syst Hum 35(2):198–212CrossRefGoogle Scholar
  8. 8.
    Agarwal P, Deshpande AD (2019) A framework for adaptation of training task, assistance and feedback for optimizing motor (re)-learning with a robotic exoskeleton. IEEE Robot Autom Lett 4(2):808–815CrossRefGoogle Scholar
  9. 9.
    Qiu S, Li Z, Wei H et al (2017) BrainźMachine interface and visual compressive sensing-based teleoperation control of an exoskeleton robot. IEEE Trans Fuzzy Syst 25(1):58–69CrossRefGoogle Scholar
  10. 10.
    Huang B, Li Z, Wu X, Ajoudani A, Bicchi A, Liu J (2019) Coordination control of a dual-arm exoskeleton robot using human impedance transfer skills. IEEE Trans Syst Man Cybern: Syst 49(5):954–963CrossRefGoogle Scholar
  11. 11.
    Karelis AD, Carvalho LP, Castillo MJ et al (2017) Effect on body composition and bone mineral density of walking with a robotic exoskeleton in adults with chronic spinal cord injury. J Rehabil Med 49(1):84CrossRefGoogle Scholar
  12. 12.
    Gopura RARC, Kiguchi K (2008) Development of an exoskeleton robot for human wrist and forearm motion assist. In: International conference on industrial & information systems, pp 432–459Google Scholar
  13. 13.
    Gao B, Ma H, Guo S et al (2017) Design and evaluation of a 3-degree-of-freedom upper limb rehabilitation exoskeleton robot. In: IEEE international conference on mechatronics & automation, pp 345–374Google Scholar
  14. 14.
    Liu H, Ju Z, Ji X, et al. (2017) Human Motion Sensing and Recognition. A Fuzzy Qualitative Approach. Studies in Computational Intelligence. vol 675. Springer, Berlin, HeidelbergCrossRefGoogle Scholar
  15. 15.
    Xu C, He J, Zhang X et al (2018) Recurrent transformation of prior knowledge based model for human motion recognition. Comput Intell Neurosci 2018(1):1–12Google Scholar
  16. 16.
    Vu C, Kim J (2018) Human motion recognition using E-textile sensor and adaptive neuro-fuzzy inference system. Fibers Polym 19(12):2657–2666CrossRefGoogle Scholar
  17. 17.
    Lin CJ, Wu C, Chaovalitwongse WA (2017) Integrating Human behavior modeling and data mining techniques to predict human errors in numerical typing. IEEE Trans Hum-Mach Syst 45(1):39–50CrossRefGoogle Scholar
  18. 18.
    Sale P, Franceschini M, Waldner A et al (2012) Use of the robot assisted gait therapy in rehabilitation of patients with stroke and spinal cord injury. Eur J Phys Rehabil Med 48(1):111Google Scholar
  19. 19.
    Cao J, Xie SQ, Das R et al (2014) Control strategies for effective robot assisted gait rehabilitation: the state of art and future prospects. Med Eng Phys 36(12):1555–1566CrossRefGoogle Scholar
  20. 20.
    Jarrassé N, Proietti T, Crocher V et al (2014) Robotic exoskeletons: a perspective for the rehabilitation of arm coordination in stroke patients. Front Hum Neurosci 8(947):1845–1846Google Scholar
  21. 21.
    Taheri H, Rowe JB, Gardner D et al (2012) Robot-assisted Guitar Hero for finger rehabilitation after stroke. Conf Proc IEEE Eng Med Biol Soc 2012(4):3911–3917Google Scholar
  22. 22.
    Zerdoumi S, Sabri AQM, Kamsin A et al (2017) Image pattern recognition in big data: taxonomy and open challenges: survey. Multim Tools Appl 2:1–31Google Scholar
  23. 23.
    Nikonov DE, Csaba G, Porod W et al (2017) Coupled-oscillator associative memory array operation for pattern recognition. IEEE J Explor Solid-State Comput Devices Circuits 1:85–93CrossRefGoogle Scholar
  24. 24.
    Nabet BY, Qiu Y, Shabason JE et al (2017) Exosome RNA unshielding couples stromal activation to pattern recognition receptor signaling in cancer. Cell 170(2):352–366CrossRefGoogle Scholar
  25. 25.
    Lu Z, Chen X, Zhang X et al (2017) Real-Time control of an exoskeleton hand robot with myoelectric pattern recognition. Int J Neural Syst 27(5):1750009CrossRefGoogle Scholar
  26. 26.
    Lu Z, Tong RK, Zhang X et al (2018) Myoelectric pattern recognition for controlling a robotic hand: a feasibility study in stroke. IEEE Trans Bio-Med Eng 99:1Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Yi Zheng
    • 1
  • Qingjun Song
    • 2
    Email author
  • Jixin Liu
    • 1
  • Qinghui Song
    • 3
  • Qingchao Yue
    • 1
  1. 1.Institute of Intelligence and ManufactureQingdao Huanghai UniversityQingdaoChina
  2. 2.Department of Mechanical and Electronic EngineeringShandong University of Science and TechnologyTai’anChina
  3. 3.College of Mechanical and Electronic EngineeringShandong University of Science and TechnologyQingdaoChina

Personalised recommendations