Skip to main content
SpringerLink
Log in

Towards Stroke Patients’ Upper-Limb Automatic Motor Assessment Using Smartwatches

  • Conference paper
  • First Online:
Pattern Recognition. ICPR International Workshops and Challenges (ICPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12661))

Included in the following conference series:

Abstract

Assessing the physical condition in rehabilitation scenarios is a challenging problem, since it involves Human Activity Recognition (HAR) and kinematic analysis methods. In addition, the difficulties increase in unconstrained rehabilitation scenarios, which are much closer to the real use cases. In particular, our aim is to design an upper-limb assessment pipeline for stroke patients using smartwatches. We focus on the HAR task, as it is the first part of the assessing pipeline. Our main target is to automatically detect and recognize four key movements inspired by the Fugl-Meyer assessment scale, which are performed in both constrained and unconstrained scenarios. In addition to the application protocol and dataset, we propose two detection and classification baseline methods. We believe that the proposed framework, dataset and baseline results will serve to foster this research field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://dag.cvc.uab.es/patientmonitoring/.

  2. 2.

    https://x-io.co.uk/open-source-imu-and-ahrs-algorithms/.

  3. 3.

    Dataset available at http://dag.cvc.uab.es/patientmonitoring/.

References

  1. Lee, K.B., et al.: Six-month functional recovery of stroke patients: a multi-time-point study. Int. J. Rehabil. Res. 38(2), 173 (2015)

    Article  Google Scholar 

  2. Zunino, A., et al.: Video gesture analysis for autism spectrum disorder detection. In: 24th International Conference on Pattern Recognition (ICPR), pp. 3421–3426. IEEE (2018)

    Google Scholar 

  3. Rueda, F.M., Fink, G.A.: Learning attribute representation for human activity recognition. In 24th International Conference on Pattern Recognition (ICPR), pp. 523–528. IEEE (2018)

    Google Scholar 

  4. Zinnen, A., van Laerhoven, K., Schiele, B.: Toward recognition of short and non-repetitive activities from wearable sensors. In: Schiele, B., et al. (eds.) AmI 2007. LNCS, vol. 4794, pp. 142–158. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76652-0_9

    Chapter  Google Scholar 

  5. Kim, J., Parnell, C., Wichmann, T., DeWeerth, S.P.: Longitudinal wearable tremor measurement system with activity recognition algorithms for upper limb tremor. In: 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6166–6169 (2016)

    Google Scholar 

  6. Morris, D., Saponas, T.S., Guillory, A., Kelner, I.: Recofit: using a wearable sensor to find, recognize, and count repetitive exercises. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3225–3234 (2014)

    Google Scholar 

  7. Oña, E.D., Jardón, A., Monge, E., Molina, F., Cano, R., Balaguer, C.: Towards automated assessment of upper limbs motor function based on fugl-meyer test and virtual environment. In: Masia, L., Micera, S., Akay, M., Pons, J.L. (eds.) ICNR 2018. BB, vol. 21, pp. 297–301. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-01845-0_60

    Chapter  Google Scholar 

  8. Schindler, R., Bouillon, M., Plamondon, R., Fischer, A.: Extending the sigma-lognormal model of the kinematic theory to three dimensions. In: Proceedings of the International Conference on Pattern Recognition and Artificial Intelligence (2018)

    Google Scholar 

  9. Akila, K., Chitrakala, S.: Highly refined human action recognition model to handle intraclass variability & interclass similarity. Multimedia Tools Appl. 78(15), 20877–20894 (2019). https://doi.org/10.1007/s11042-019-7392-z

    Article  Google Scholar 

  10. Bulling, A., Blanke, U., Schiele, B.: A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 46(3), 1–33 (2013)

    Article  Google Scholar 

  11. Clarkson, B.P.: Life patterns: structure from wearable sensors. PhD thesis, Massachusetts Institute of Technology (2002)

    Google Scholar 

  12. Bao, L., Intille, S.S.: Activity recognition from user-annotated acceleration data. In: Ferscha, A., Mattern, F. (eds.) Pervasive 2004. LNCS, vol. 3001, pp. 1–17. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24646-6_1

    Chapter  Google Scholar 

  13. He, Z., Jin, L.: Activity recognition from acceleration data based on discrete consine transform and SVM. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 5041–5044. IEEE (2009)

    Google Scholar 

  14. Bhat, G., Deb, R., Chaurasia, V.V., Shill, H., Ogras, U.Y.: Online human activity recognition using low-power wearable devices. In: IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1–8. IEEE (2018)

    Google Scholar 

  15. Dehghani, A., Sarbishei, O., Glatard, T., Shihab, E.: A quantitative comparison of overlapping and non-overlapping sliding windows for human activity recognition using inertial sensors. Sensors 19(22), 5026 (2019)

    Article  Google Scholar 

  16. Zhang, Y., Zhang, Y., Zhang, Z., Bao, J., Song, Y.: Human activity recognition based on time series analysis using U-net. arXiv preprint arXiv:1809.08113 (2018)

  17. Rastegari, E., Ali, H.: A bag-of-words feature engineering approach for assessing health conditions using accelerometer data. Smart Health 16, 100116 (2020)

    Article  Google Scholar 

  18. Yao, R., Lin, G., Shi, Q., Ranasinghe, D.C.: Efficient dense labeling of human activity sequences from wearables using fully convolutional networks. Pattern Recogn. 78, 252–266 (2017)

    Article  Google Scholar 

  19. Ahmed, N., Rafiq, J.I., Islam, M.R.: Enhanced human activity recognition based on smartphone sensor data using hybrid feature selection model. Sensors 20(1), 317 (2020)

    Article  Google Scholar 

  20. Garcia-Gonzalez, D., Rivero, D., Fernandez-Blanco, E., Luaces, M.R.: A public domain dataset for real-life human activity recognition using smartphone sensors. Sensors 20(8), 2200 (2020)

    Article  Google Scholar 

  21. Souza, W., Kavitha, R.: Human activity recognition using accelerometer and gyroscope sensors. Int. J. Eng. Technol. pp. 1171–1179 (2017)

    Google Scholar 

  22. Ayumi, V., Fanany, M.I.: A comparison of SVM and RVM for human action recognition. Internetworking Indonesia J. 8(1), 29–33 (2016)

    Google Scholar 

  23. Zhang, M., Sawchuk, A.A.: Human daily activity recognition with sparse representation using wearable sensors. IEEE J. Biomed. Health Inform. 3, 553–560 (2013)

    Article  Google Scholar 

  24. Xu, W., Zhang, M., Sawchuk, A.A., Sarrafzadeh, M.: Robust human activity and sensor location corecognition via sparse signal representation. IEEE Trans. Biomed. Eng. 11, 3169–3176 (2012)

    Google Scholar 

  25. Mehrang, S., Pietilä, J., Korhonen, I.: An activity recognition framework deploying the random forest classifier and a single optical heart rate monitoring and triaxial accelerometer wrist-band. Sensors 18(2), 613 (2018)

    Article  Google Scholar 

  26. Xu, L., Yang, W., Cao, Y., Li, Q.: Human activity recognition based on random forests. In: Liu, Y., Zhao, L., Cai, G., Xiao, G., Li, K., Wang, L., eds. 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2017, Guilin, China, July 29–31, 2017, pp. 548–553. IEEE (2017)

    Google Scholar 

  27. Ha, S., Choi, S.: Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. In: International Joint Conference on Neural Networks (IJCNN), pp. 381–388 (2016)

    Google Scholar 

  28. Ordóñez, F., Roggen, D.: Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)

    Article  Google Scholar 

  29. Tian, Y., Zhang, J., Chen, L., Geng, Y., Wang, X.: Selective ensemble based on extreme learning machine for sensor-based human activity recognition. Sensors 19(16), 3468 (2019)

    Article  Google Scholar 

  30. Ryanne, J.M., et al.: Recognizing complex upper extremity activities using body worn sensors. PLoS One 10(3), e0118642 (2015)

    Article  Google Scholar 

  31. Butt, A.H., et al.: Assessment of purposeful movements for post-stroke patients in activites of daily living with wearable sensor device. In: IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pp. 1–8 (2019)

    Google Scholar 

  32. Rosati, S., Balestra, G., Knaflitz, M.: Comparison of different sets of features for human activity recognition by wearable sensors. Sensors 18(122), 4189 (2018)

    Article  Google Scholar 

  33. Pires, I.M., et al.: Pattern recognition techniques for the identification of activities of daily living using mobile device accelerometer. Electronics 9(3), 509 (2017)

    Article  Google Scholar 

  34. Cervantes, J., Garcia-Lamont, F., Rodríguez, L., Lopez, A.: A comprehensive survey on support vector machine classification: applications, challenges and trends. Neurocomputing 408, 189–215 (2020)

    Article  Google Scholar 

  35. Banegas-Luna, A.J., et al.: When will the mist clear? on the interpretability of machine learning for medical applications: a survey. arXiv preprint arXiv:2010.00353 (2020)

  36. Lawhern, V.J., Solon, A.J., Waytowich, N.R., Gordon, S.M., Hung, C.P., Lance, B.J.: EEGNet: a compact convolutional network for EEG-based brain-computer interfaces. J. Neural Eng. 15(5), 056013 (2016)

    Article  Google Scholar 

Download references

Acknowledgment

This work has been partially supported by the H2020 ATTRACT EU project (Grant Agreement 777222, TPPA 773, RPM3D), the Spanish project RTI2018-095645-B-C21, the FI fellowship AGAUR 2020 FI-SDUR 00497 (with the support of the Secretaria d’Universitats i Recerca of the Generalitat de Catalunya and the Fons Social Europeu), the Ramon y Cajal Fellowship RYC-2014-16831 and the CERCA Program/ Generalitat de Catalunya. C. Carmona-Duarte was supported by a Viera y Clavijo contract from the Universidad de Las Palmas de Gran Canaria. The authors would like to thank Oriol Ramos and Ángel Sánchez for fruitful discussions.

Author information

Authors and Affiliations

  1. Computer Vision Center, Computer Science Department, Universitat Autònoma de Barcelona, Barcelona, Spain

    Asma Bensalah, Jialuo Chen, Alicia Fornés & Josep Lladós

  2. Instituto Universitario para el Desarrollo Tecnológico y la Innovación en Comunicaciones, Universidad de Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain

    Cristina Carmona-Duarte & Miguel Ángel Ferrer

Authors
  1. Asma Bensalah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialuo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alicia Fornés
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cristina Carmona-Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Josep Lladós
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Miguel Ángel Ferrer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Asma Bensalah , Jialuo Chen , Alicia Fornés , Cristina Carmona-Duarte , Josep Lladós or Miguel Ángel Ferrer .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bensalah, A., Chen, J., Fornés, A., Carmona-Duarte, C., Lladós, J., Ferrer, M.Á. (2021). Towards Stroke Patients’ Upper-Limb Automatic Motor Assessment Using Smartwatches. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12661. Springer, Cham. https://doi.org/10.1007/978-3-030-68763-2_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68763-2_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68762-5

  • Online ISBN: 978-3-030-68763-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation