Skip to main content
SpringerLink
Log in

Safety Protection Method of Rehabilitation Robot Based on fNIRS and RGB-D Information Fusion

  • Article
  • Published:
Journal of Shanghai Jiaotong University (Science) Aims and scope Submit manuscript

Abstract

In order to improve the safety protection performance of the rehabilitation robot, an active safety protection method is proposed in the rehabilitation scene. The oxyhemoglobin concentration information and RGB-D information are combined in this method, which aims to realize the comprehensive monitoring of the invasion target, the patient’s brain function movement state, and the joint angle in the rehabilitation scene. The main focus is to study the fusion method of the oxyhemoglobin concentration information and RGB-D information in the rehabilitation scene. Frequency analysis of brain functional connectivity coefficient was used to distinguish the basic motion states. The human skeleton recognition algorithm was used to realize the angle monitoring of the upper limb joint combined with the depth information. Compared with speed and separation monitoring, the protection method of multi-information fusion is safer and more comprehensive for stroke patients. By building the active safety protection platform of the upper limb rehabilitation robot, the performance of the system in different safety states is tested, and the safety protection performance of the method in the upper limb rehabilitation scene is verified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. WANG L D. Brief report on stroke prevention and treatment in China, 2019: Report on stroke prevention and treatment in China Writing Group [J]. Chinese Journal of Cerebrovascular Diseases, 2020, 17(5): 272–281 (in Chinese).

    Google Scholar 

  2. FU L, WU R, ZHAO J. The evolution and enlightenment of safety specification of cooperative robots: ISO/TS 15066 [J]. Robot, 2017, 39(4): 532–540 (in Chinese).

    Google Scholar 

  3. DE SANTIS A, SICILIANO B, DE LUCA A, et al. An atlas of physical human-robot interaction [J]. Mechanism and Machine Theory, 2008, 43(3): 253–270.

    Article  Google Scholar 

  4. KIKUUWE R, YASUKOUCHI S, FUJIMOTO H, et al. Proxy-based sliding mode control: A safer extension of PID position control [J]. IEEE Transactions on Robotics, 2010, 26(4): 670–683.

    Article  Google Scholar 

  5. YU W, ROSEN J. A novel linear PID controller for an upper limb exoskeleton [C]//49th IEEE Conference on Decision and Control (CDC). Atlanta, GA: IEEE, 2010: 3548–3553.

    Google Scholar 

  6. SUGAR T G, HE J P, KOENEMAN E J, et al. Design and control of RUPERT: A device for robotic upper extremity repetitive therapy [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2007, 15(3): 336–346.

    Article  Google Scholar 

  7. ZHANG Y B, WANG Z X, JI L H, et al. The clinical application of the upper extremity compound movements rehabilitation training robot [C]//9th International Conference on Rehabilitation Robotics, 2005. Chicago, IL: IEEE, 2005: 91–94.

    Google Scholar 

  8. DUVINAGE M, CASTERMANS T, PETIEAU M, et al. A subjective assessment of a P300 BCI system for lower-limb rehabilitation purposes [C]//2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. San Diego, CA: IEEE, 2012: 3845–3849.

    Google Scholar 

  9. TONG L N, HOU Z G, PENG L, et al. Multichannel sEMG time series analysis based human motion recognition method [J]. Acta Automatica Sinica, 2014, 40(5): 810–821 (in Chinese).

    Google Scholar 

  10. KHALID A, KIRISCI P, GHRAIRI Z, et al. Safety requirements in collaborative human-robot cyberphysical system [M]//Dynamics in logistics. Cham: Springer International Publishing, 2016: 41–51.

    Google Scholar 

  11. WANG Z P, ZHOU Y J, CHEN L, et al. BCI monitor enhances electroencephalographic and cerebral hemodynamic activations during motor training [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(4): 780–787.

    Article  Google Scholar 

  12. LIQ B, FENG J, GUO J, et al. Effects of themultisensory rehabilitation product for home-based hand training after stroke on cortical activation by using NIRS methods [J]. Neuroscience Letters, 2020, 717: 134682.

    Article  Google Scholar 

  13. WANG M J, HU Z S, LIU L, et al. Disrupted functional brain connectivity networks in children with attentiondeficit/hyperactivity disorder: Evidence from restingstate functional near-infrared spectroscopy [J]. Neurophotonics, 2020, 7(1): 015012.

    Article  Google Scholar 

  14. BU L G, WU Y, YAN Q, et al. Effects of physical training on brain functional connectivity of methamphetamine dependencies as assessed using functional near-infrared spectroscopy [J]. Neuroscience Letters, 2020, 715: 134605.

    Article  Google Scholar 

  15. HUO C C, XU G C, LI Z Y, et al. Limb linkage rehabilitation training-related changes in cortical activation and effective connectivity after stroke: A functional near-infrared spectroscopy study [J]. Scientific Reports, 2019, 9: 6226.

    Article  Google Scholar 

  16. ZHANG S, ZHENG Y C, WANG D F, et al. Application of a common spatial pattern-based algorithm for an fNIRS-based motor imagery brain-computer interface [J]. Neuroscience Letters, 2017, 655: 35–40.

    Article  Google Scholar 

  17. LU K, XU G C, LI W H, et al. Frequency-specific functional connectivity related to the rehabilitation task of stroke patients [J]. Medical Physics, 2019, 46(4): 1545–1560.

    Article  Google Scholar 

  18. LIU Z, ZHANG M, XU G, et al. Effective connectivity analysis of the brain network in drivers during actual driving using near-infrared spectroscopy [J]. Frontiers in Behavioral Neuroscience, 2017, 11: 211.

    Article  Google Scholar 

  19. XU G C, ZHANG M, WANG Y, et al. Functional connectivity analysis of distracted drivers based on the wavelet phase coherence of functional near-infrared spectroscopy signals [J]. PLoS One, 2017, 12(11): e0188329.

    Article  Google Scholar 

  20. XU L W, WANG B T, XU G C, et al. Functional connectivity analysis using fNIRS in healthy subjects during prolonged simulated driving [J]. Neuroscience Letters, 2017, 640: 21–28.

    Article  Google Scholar 

  21. HU X, ZHUANG C, WANG F, et al. fNIRS evidence for recognizably different positive emotions [J]. Frontiers in Human Neuroscience, 2019, 13: 120.

    Article  Google Scholar 

  22. LIU Q Y, WANG B T, LIU Y, et al. Frequency-specific effective connectivity in subjects with cerebral infarction as revealed by NIRS method [J]. Neuroscience, 2018, 373: 169–181.

    Article  Google Scholar 

  23. NIKOLAKIS N, MARATOS V, MAKRIS S. A cyber physical system (CPS) approach for safe human-robot collaboration in a shared workplace [J]. Robotics and Computer-Integrated Manufacturing, 2019, 56: 233–243.

    Article  Google Scholar 

  24. NI Z S, GU Y, LIU Q L, et al. Flexible calibration method for binocular stereo vision in large field of view [J]. Optics and Precision Engineering, 2017, 25(7): 1882–1889 (in Chinese).

    Google Scholar 

  25. CAO Z, HIDALGO G, SIMON T, et al. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(1): 172–186.

    Article  Google Scholar 

  26. SNAVELY K N. Scene reconstruction and visualization from Internet photo collections [D]. Seattle, WA: University of Washington, 2008.

    Google Scholar 

  27. STAUFFER C, GRIMSON W E L. Adaptive background mixture models for real-time tracking [C]//Proceedings of 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149). Fort Collins, CO: IEEE, 1999: 246–252.

    Google Scholar 

  28. REDMON J, FARHADI A. YOLOv3: An incremental improvement [EB/OL]. [2021-01-30]. https://arxiv.org/pdf/1804.02767.pdf.

  29. SU H L, HUO C C, WANG B T, et al. Alterations in the coupling functions between cerebral oxyhaemoglobin and arterial blood pressure signals in post-stroke subjects [J]. PLoS One, 2018, 13(4): e0195936.

    Article  Google Scholar 

  30. BU L G, QI L P, YAN W, et al. Acute kick-boxing exercise alters effective connectivity in the brain of females with methamphetamine dependencies [J]. Neuroscience Letters, 2020, 720: 134780.

    Article  Google Scholar 

  31. JIAN C Y, DENG L C, LIANG L K, et al. Neuromuscular control of the agonist-antagonist muscle coordination affected by visual dimension: An EMG-fNIRS study [J]. IEEE Access, 2020, 8: 100768–100777.

    Article  Google Scholar 

Download references

Author information

Author notes

  1. These authors contributed equally to this work.

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, Jiangsu Provincial Key Laboratory of Advanced Robotics, Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou, Jiangsu, 215123, China

    Dong Li  (李 栋), Yulin Fan  (樊钰琳), Guodong Chen  (陈国栋) & Wenzheng Chi  (迟文政)

  2. Department of Gerontology, Shanghai Jiao Tong University Affiliated Sixth People’s Hospital, Shanghai, 200233, China

    Zheng Wang  (王 正)

  3. Department of Instrument Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Na Lü  (吕 娜)

Authors
  1. Dong Li  (李 栋)
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yulin Fan  (樊钰琳)
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Na Lü  (吕 娜)
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guodong Chen  (陈国栋)
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zheng Wang  (王 正)
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wenzheng Chi  (迟文政)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Guodong Chen  (陈国栋) or Zheng Wang  (王 正).

Additional information

Foundation item: the Interdisciplinary Program of Shanghai Jiao Tong University (No. YG2019QNA25)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, D., Fan, Y., Lü, N. et al. Safety Protection Method of Rehabilitation Robot Based on fNIRS and RGB-D Information Fusion. J. Shanghai Jiaotong Univ. (Sci.) 27, 45–54 (2022). https://doi.org/10.1007/s12204-021-2365-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12204-021-2365-6

Key words

CLC number

Document code

Search

Navigation