User Modeling and User-Adapted Interaction

, Volume 27, Issue 2, pp 215–265 | Cite as

A therapy-driven gamification framework for hand rehabilitation

  • Imad AfyouniEmail author
  • Faizan Ur Rehman
  • Ahmad Muaz Qamar
  • Sohaib Ghani
  • Syed Osama Hussain
  • Bilal Sadiq
  • Mohamed Abdur Rahman
  • Abdullah Murad
  • Saleh Basalamah


Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation centers or hospitals, leading to slower recovery times for corresponding patients. Therefore, there is a high demand for the development of technology-based personalized solutions to guide and encourage patients towards performing online rehabilitation program that can help them live independently at home. This paper introduces an innovative e-health framework that develops adaptive serious games for people with hand disabilities. The aim of this work is to provide a patient-adaptive environment for the gamification of hand therapies in order to facilitate and encourage rehabilitation issues. Theoretical foundations (i.e., therapy and patient models) and algorithms to match therapy-based hand gestures to navigational movements in 3D space within the serious game environment have been developed. A novel game generation module is introduced, which translates those movements into a 3D therapy-driven route on a real-world map and with different levels of difficulty based on the patient profile and capabilities. In order to enrich the user navigation experience, a 3D spatio-temporal validation region is also generated, which tracks and adjusts the patient movements throughout the session. The gaming environment also creates and adds semantics to different types of attractive and repellent objects in space depending on the difficulty level of the game. Relevant benchmarks to assess the patient interaction with the environment along with a usability and performance testing of our framework are introduced to ensure quantitative as well as qualitative improvements. Trial tests in one disability center were conducted with a total number of five subjects, having hand motor controls problems, who used our gamified physiotherapy solution to help us in measuring the usability and users’ satisfaction levels. The obtained results and feedback from therapists and patients are very encouraging.


Difficulty Level Patient Performance Wrist Flexion Ulnar Deviation Game Engine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This project was supported by the KACST Research program (M-S-37-60) in the Kingdom of Saudi Arabia. We would like to thank all the therapists and doctors that gave us useful feedback of the system and medical specific knowledge in the Disability Children Association in Makkah. Particularly, we thank Dr. Shehnaz Hassan of physiotherapy department of Umm Al-Qura University for her time and efforts. We would also like to thank all the patients who agree to use our framework and participated in the user study and also gave valuable feedback. We kindly acknowledge the useful suggestions from Dr. Mohamed F. Mokbel of University of Minnesota, Dr. Sohaib Khan and Akhlaq Ahmad from Umm Al-Qura University, and Dr. Ahmed Lbath of University of Grenoble Alpes, France. We would like to thank Wadi Makkah Technology Innovation Center at Umm Al-Qura University, Saudi Arabia for providing the necessary resources.


  1. Afyouni, I., Rehman, F.U., Qamar, A., Ahmad, A., Abdur, M., Basalamah, S.: A GIS-based serious game recommender for online physical therapy. In: Proceedings of the Third International ACM SIGSPATIAL Workshop on HealthGIS (HealthGIS’14), ACM (2014)Google Scholar
  2. Ayala, D., Wolfson, O., Xu, B., Dasgupta, B., Lin, J.: Parking slot assignment games. In: Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, pp. 299–308 (2011)Google Scholar
  3. Bartolomé, N.A., Zorrilla, A.M., Zapirain, B.G.: Can game-based therapies be trusted? Is game-based education effective? A systematic review of the serious games for health and education. In: Proceedings of 2011 16th International Conference on Computer Games (CGAMES), IEEE, pp. 275–282 (2011)Google Scholar
  4. Boian, R., Sharma, A., Han, C., Merians, A., Burdea, G., Adamovich, S., Recce, M., Tremaine, M., Poizner, H., et al.: Virtual Reality-Based Post-Stroke Hand Rehabilitation. Studies in Health Technology and Informatics, pp. 64–70. IOS press, Newport beach, CA (2002)Google Scholar
  5. Boulanger, C., Boulanger, A., de Greef, L., Kearney, A., Sobel, K., Transue, R., Sweedyk, Z., Dietz, P.H., Bathiche, S.: Stroke rehabilitation with a sensing surface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp. 1243–1246 (2013)Google Scholar
  6. Cameirão, M.S., i Badia, S.B., Oller, E.D., Verschure, P.F.: Neurorehabilitation using the virtual reality based rehabilitation gaming system: methodology, design, psychometrics, usability and validation. J. Neuroeng. Rehabil. 7(1), 1 (2010)CrossRefGoogle Scholar
  7. Chang, Y.J., Chen, S.F., Huang, J.D.: A kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res. Dev. Disabil. 32(6), 2566–2570 (2011)CrossRefGoogle Scholar
  8. Chen, Y., Xu, W., Sundaram, H., Rikakis, T., Liu, S.M.: A dynamic decision network framework for online media adaptation in stroke rehabilitation. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 5(1), 4 (2008)Google Scholar
  9. Chittaro, L., Carchietti, E., De Marco, L., Zampa, A.: Personalized emergency medical assistance for disabled people. User Model. User-adap. Interact. 21(4–5), 407–440 (2011)CrossRefGoogle Scholar
  10. Crawford, S., Fuhrmann, S., Li, B., MacEachren, A., Gahegan, M., Downs, R.: Mapstats for kids: developing and assessing web-based tools to foster geographic and statistical literacy in children. In: Proceedings of the Annual National Conference on Digital Government Research, Digital Government Society of North America (2003)Google Scholar
  11. Eurostat: healthcare personnel statistics—dentists, pharmacists and physiotherapists.,_pharmacists_and_physiotherapists (2017)
  12. Gajdosik, R.L., Bohannon, R.W.: Clinical measurement of range of motion. Phys. Ther. 67(12), 1867–1872 (1987)CrossRefGoogle Scholar
  13. Gouaïch, A., Hocine, N., Van Dokkum, L., Mottet, D.: Digital-pheromone based difficulty adaptation in post-stroke therapeutic games. In: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, ACM, pp. 5–12 (2012)Google Scholar
  14. Grant, J.A., Mohtadi, N.G., Maitland, M.E., Zernicke, R.F.: Comparison of home versus physical therapy-supervised rehabilitation programs after anterior cruciate ligament reconstruction a randomized clinical trial. Am. J. Sports Med. 33(9), 1288–1297 (2005)CrossRefGoogle Scholar
  15. Hashemian, Y., Wang, D.: Touchpoints: an exertion game with strategy. In: Proceedings of the first ACM SIGCHI Annual Symposium on Computer–Human Interaction in Play, ACM, pp. 419–420 (2014)Google Scholar
  16. Heuser, A., Kourtev, H., Hentz, V., Forducey, P.: Tele-rehabilitation using the rutgers master II glove following carpal tunnel release surgery. In: 2006 International Workshop on Virtual Rehabilitation, IEEE, pp. 88–93 (2006)Google Scholar
  17. Hocine, N., Gouaïch, A., Cerri, S.A., Mottet, D., Froger, J., Laffont, I.: Adaptation in serious games for upper-limb rehabilitation: an approach to improve training outcomes. User Model. User-adap. Interact. 25(1), 65–98 (2015)CrossRefGoogle Scholar
  18. Jacob, R., Mooney, P., Corcoran, P., Winstanley, A.C.: Haptic-GIS: exploring the possibilities. SIGSPATIAL Special 2(3), 13–18 (2010)CrossRefGoogle Scholar
  19. Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., Cramer, S.C.: Free-hand interaction with leap motion controller for stroke rehabilitation. In: Proceedings of the CHI’14 Extended Abstracts on Human Factors in Computing Systems, ACM, pp. 1663–1668 (2014)Google Scholar
  20. Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)CrossRefGoogle Scholar
  21. Kim, D., Hilliges, O., Izadi, S., Butler, A.D., Chen, J., Oikonomidis, I., Olivier, P.: Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, ACM, pp. 167–176 (2012)Google Scholar
  22. Komatireddy, R., Hutchins, S., Shah, M.: Systems, apparatus and methods for non-invasive motion tracking to augment patient administered physical rehabilitation. US Patent App. 13/524,708 (2013)Google Scholar
  23. Ma, M., Bechkoum, K.: Serious games for movement therapy after stroke. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, IEEE, pp. 1872–1877 (2008)Google Scholar
  24. Ma, M., McNeill, M., Charles, D., McDonough, S., Crosbie, J., Oliver, L., McGoldrick, C.: Adaptive virtual reality games for rehabilitation of motor disorders. In: International Conference on Universal Access in Human–Computer Interaction, Springer, pp. 681–690 (2007)Google Scholar
  25. Mayo, M.J.: Games for science and engineering education. Commun. ACM 50(7), 30–35 (2007)CrossRefGoogle Scholar
  26. Michael, D.R., Chen, S.L.: Serious Games: Games that Educate, Train, and Inform. Muska & Lipman, Cincinnati (2005)Google Scholar
  27. Mihelj, M., Novak, D., Milavec, M., Ziherl, J., Olenšek, A., Munih, M.: Virtual rehabilitation environment using principles of intrinsic motivation and game design. Presence Teleoperators Virtual Environ. 21(1), 1–15 (2012)CrossRefGoogle Scholar
  28. Norkin, C.C., White, D.J.: Measurement of Joint Motion A Guide to Goniometry, chap 6, 4th edn. F. A. Davis, Duxbury (2009)Google Scholar
  29. Numrich, S.K.: Culture, models, and games: incorporating warfare’s human dimension. IEEE Intell. Syst. 4(23), 58–61 (2008)CrossRefGoogle Scholar
  30. Omelina, L., Jansen, B., Bonnechere, B., Van Sint Jan, S., Cornelis, J.: Serious games for physical rehabilitation: designing highly configurable and adaptable games. In: Proceedings of the 9th International Conference on Disability, Virutal Reality and Associated Technologies, pp. 195–201 (2012)Google Scholar
  31. Peng, S.T., Hsu, S.Y., Hsieh, K.: An interactive immersive serious game application for Kunyu Quantu world map. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2(5), 221 (2015)CrossRefGoogle Scholar
  32. Pirovano, M., Mainetti, R., Baud-Bovy, G., Lanzi, P.L., Borghese, N.A.: Self-adaptive games for rehabilitation at home. In: 2012 IEEE Conference on Computational Intelligence and Games (CIG), IEEE, pp. 179–186 (2012)Google Scholar
  33. Qamar, A., Afyouni, I., Hossain, D., Ur Rehman, F., Toonsi, A., Abdur Rahman, M., Basalamah, S.: A multimedia e-health framework towards an interactive and non-invasive therapy monitoring environment. In: Proceedings of the ACM International Conference on Multimedia, ACM, pp. 745–746 (2014a)Google Scholar
  34. Qamar, A., Afyouni, I., Rahman, M.A., Rehman, F.U., Hussain, D., Basalamah, S., Lbath, A.: A “gis-based” serious game interface for therapy monitoring. In: Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Dallas/Fort Worth, TX, USA, November 4–7, 2014, ACM, pp. 589–592 (2014b)Google Scholar
  35. Rabin, B., Burdea, G., Hundal, J., Roll, D., Damiani, F.: Integrative motor, emotive and cognitive therapy for elderly patients chronic post-stroke: a feasibility study of the brightarm rehabilitation system. In: 2011 International Conference on Virtual Rehabilitation, IEEE, pp. 1–8 (2011)Google Scholar
  36. Rego, P., Moreira, P.M., Reis, L.P.: Serious games for rehabilitation: a survey and a classification towards a taxonomy. In: Proceedings of the 2010 5th Iberian Conference on Information Systems and Technologies (CISTI), IEEE, pp. 1–6 (2010)Google Scholar
  37. Saini, S., Rambli, D.R.A., Sulaiman, S., Zakaria, M.N., Shukri, S.R.M.: A low-cost game framework for a home-based stroke rehabilitation system. In: Proceedings of the International Conference on Computer and Information Science (ICCIS), IEEE, vol. 1, pp. 55–60 (2012)Google Scholar
  38. Sawyer, B.: From cells to cell processors: the integration of health and video games. IEEE Comput. Graph. Appl. 28(6), 83–85 (2008)CrossRefGoogle Scholar
  39. Stone, E.E., Skubic, M.: Passive in-home measurement of stride-to-stride gait variability comparing vision and kinect sensing. In: Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, pp. 6491–6494 (2011)Google Scholar
  40. Stone, E.E., Skubic, M.: Mapping kinect-based in-home gait speed to tug time: a methodology to facilitate clinical interpretation. In: Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, IEEE, pp. 57–64 (2013)Google Scholar
  41. Wang, C., Phua, K.S., Ang, K.K., Guan, C., Zhang, H., Lin, R., Chua, K.S.G., Ang, B.T., Kuah, C.W.K.: A feasibility study of non-invasive motor-imagery bci-based robotic rehabilitation for stroke patients. In: Proceedings of the 4th International IEEE/EMBS Conference on Neural Engineering, IEEE, pp. 271–274 (2009)Google Scholar
  42. Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380–6393 (2013)CrossRefGoogle Scholar
  43. Zhao, W., Chai, J., Xu, Y.Q.: Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, pp. 33–42 (2012)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  • Imad Afyouni
    • 1
    Email author
  • Faizan Ur Rehman
    • 2
  • Ahmad Muaz Qamar
    • 3
  • Sohaib Ghani
    • 2
  • Syed Osama Hussain
    • 1
  • Bilal Sadiq
    • 1
  • Mohamed Abdur Rahman
    • 3
  • Abdullah Murad
    • 1
  • Saleh Basalamah
    • 3
  1. 1.Wadi Makkah Technology Innovation CenterUQUMeccaKingdom of Saudi Arabia
  2. 2.Science and Technology UnitUQUMeccaKingdom of Saudi Arabia
  3. 3.College of Computer and Information SystemsUQUMeccaKingdom of Saudi Arabia

Personalised recommendations