Skip to main content

Setting Up an Anonymous Gesture Database as Well as Enhancing It with a Verbal Script Simulator for Rehabilitation Applications

  • 167 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13301)

Abstract

Physical therapy patients are rehabilitated by performing exercises at home that do not consider proper movement and can be detrimental to the healing process. Maintaining patient anonymity is an important aspect of collecting patient data. Using our method, we are able to collect information about limb movements in a completely anonymous manner by taking a picture of the patient in the clinic and immediately converting the picture into an anatomical skeleton. A human gesture database accompanied by a verbal script simulator and anonymous tagging was created with the intention of tagging, measuring, and inferring human gestures using neural networks. We have developed a system that utilizes neural network autoencoder architecture to classify the quality and accuracy of patients’ movements in videos. Since there is a lack of videos of tagged physiotherapy exercises, we simulate patients’ movements to enhance the database. The purpose of this paper is to describe a simulator that mimics the output of OpenPose software so that synthetic human skeletal movements can be computed without utilizing OpenPose. As inputs, these vectors are fed to the autoencoder which, after compressing them into low dimension vectors, classifies them according to their movement using the Dynamic Time Warping (DTW) distance algorithm. Validation of the research was performed on a dataset of 7 different physiotherapy exercises, and 91.8% accuracy was achieved.

Keywords

  • OpenPose
  • Anonymous Gestures
  • Simulation
  • Siamese network
  • Physiotherapy exercises
  • Metaverse

This work was supported a grant from the Ministry of Science & Technology, Israel & the Ministry of Education, Youth and Sports of the Czech Republic.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-07689-3_13
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-031-07689-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

References

  1. Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7291–7299 (2017)

    Google Scholar 

  2. Lugaresi, C., et al.: MediaPipe: a framework for perceiving and processing reality. In: Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  3. Liao, Y., Vakanski, A., Xian, M.: A deep learning framework for assessing physical rehabilitation exercises. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 468–477 (2020)

    CrossRef  Google Scholar 

  4. Koch, G.: Siamese Neural Networks for One-Shot Image Recognition. Graduate Department of Computer Science University of Toronto, Toronto (2015)

    Google Scholar 

  5. Cai, X., Xu, T., Yi, J., Huang, J., Rajasekaran, S.: DTWNet: a dynamic TimeWarping network. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  6. Tanaka, R., Oshima, C., Nakayama, K.: Intention inference from 2D poses of preliminary action. In: GECCO’19: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1697–1700, July 2019

    Google Scholar 

  7. Cao, Z., Hidalgo, G., Simon, T., Wei, S., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using Part Affnity Fields. arXiv preprint arXiv:1812.08008 (2018)

  8. Segal, Y., et al.: Camera setup and OpenPose software without GPU for calibration and recording in telerehabilitation. In: IEEE E-Health and Bioengineering, Lasi, Romania (2021)

    Google Scholar 

  9. Segal, Y., Hadar, O.: Interpolation of missing frames of human body movements via video motion vectors - OpenPose accelerator. In: IEEE Conference on IoT for Rural Health Care, IEEE, CIRH-2021, Guntur, India (2021)

    Google Scholar 

  10. Cheung, K.H.: Optitrack - Estimation of Opti-track motion capture system data. Department of Electrical Engineering (2020)

    Google Scholar 

  11. Qualisys | motion capture systems, March 2020. https://bit.ly/YS-Qualisys

  12. Motion capture system | BTS bioengineering, SMART-DX, March 2020. https://www.btsbioengineering.com/

  13. Chen, Y., Shen, C., Wei, X., Liu, L., Yang, J.: Adversarial PoseNet: a structureaware convolutional network for human pose estimation. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1212–1221, October 2017

    Google Scholar 

  14. Wrnch - teaching cameras to read human body language, March 2020. https://wrnch.ai/

  15. Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1145–1153 (2017)

    Google Scholar 

  16. Wei, S., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4724–4732 (2016)

    Google Scholar 

  17. Zhang, Z.: Microsoft Kinect sensor and its effect. IEEE Multimedia 19(2), 4–10 (2012)

    CrossRef  Google Scholar 

  18. Home - Xsens 3D motion tracking, March 2020. www.xsens.com

  19. Rokoko - motion capture system - Smartsuit Pro, March 2020. https://www.rokoko.com/en/

  20. Shadow motion capture system, March 2020. https://www.motionshadow.com/

  21. Ahmadi, A., Destelle, F., Monaghan, D., O’Connor, N.: A framework for comprehensive analysis of a swing in sports using low-cost inertial sensors. In: SENSORS, 2014 IEEE, pp. 2211–2214, November 2014

    Google Scholar 

  22. Lucas, M., Hoyet, L., Le Clerc, F., Schnitzler, F., Hellier, P.: A survey on deep learning for skeleton-based human animation. In: COMPUTER GRAPHICS Forum (CGF) (2021)

    Google Scholar 

Download references

Acknowledgment

This work was supported a grant from the Ministry of Science & Technology, Israel & The Ministry of Education, Youth and Sports of the Czech Republic.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Yoram Segal or Ofer Hadar .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Segal, Y., Hadar, O. (2022). Setting Up an Anonymous Gesture Database as Well as Enhancing It with a Verbal Script Simulator for Rehabilitation Applications. In: Dolev, S., Katz, J., Meisels, A. (eds) Cyber Security, Cryptology, and Machine Learning. CSCML 2022. Lecture Notes in Computer Science, vol 13301. Springer, Cham. https://doi.org/10.1007/978-3-031-07689-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-07689-3_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-07688-6

  • Online ISBN: 978-3-031-07689-3

  • eBook Packages: Computer ScienceComputer Science (R0)