We present an overview of IMPACT, a system that uses virtual and augmented reality to enhance the mirror therapy for the treatment of phantom limb pain and stroke rehabilitation. The system tracks users’ movements and the head position using a combination of depth cameras and inertial sensors. This information is then fused together in a 3D full-body human model able to adapt to the patient’s characteristics (e.g., skin color, limb size) and avoid unnatural movements due to possible bad tracking (e.g., a joint bended in an unnatural way). IMPACT includes three serious games that can be played in first-person point of view and that allow different levels of immersion according to the needs of the treatment.
KeywordsAugmented reality Virtual reality Mirror therapy Phantom limb pain
This research project was supported by the Hasler Stiftung and by the University of Applied Sciences Western Switzerland. In addition, the authors would like to thank Elisa Nannini, Vincent Brodard, Quentin Seydoux, Luc Francey, and Charlotte Junod for their contribution to develop the system and games.
- 2.Tzionas, D., Vrenas, K., Eleftheriadis, S., Georgoulis, S., Petrantonakis, P., Hadjileontiadis, L.: Phantom limb pain management using facial expression analysis, biofeedback and augmented reality interfacing. In: Proceedings of the 3rd International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2010). UTAD & SAE Institute, Oxford, United Kingdom (2010)Google Scholar
- 9.Shen, Y., Ong, S., Nee, A.: An augmented reality system for hand movement rehabilitation. In: Proceedings of the 2nd International Convention on Rehabilitation Engineering & Assistive Technology, pp. 189–192. Singapore Therapeutic, Assistive & Rehabilitative Technologies (START) Centre (2008)Google Scholar
- 12.Ortiz-Catalan, M., Guðmundsdóttir, R.A., Kristoffersen, M.B., Zepeda-Echavarria, A., Caine-Winterberger, K., Kulbacka-Ortiz, K., Widehammar, C., Eriksson, K., Stockselius, A., Ragnö, C., Pihlar, Z., Burger, H., Hermansson, L.: Phantom motor execution facilitated by machine learning and augmented reality as treatment for phantom limb pain: a single group, clinical trial in patients with chronic intractable phantom limb pain. Lancet 388, 2885–2894 (2016)CrossRefGoogle Scholar
- 13.Gregory-Dean, A.: Amputations: statistics and trends. Ann. R. Coll. Surg. Engl. 73, 137–142 (1991)Google Scholar
- 14.Carrino, F., Rizzotti, D., Gheorghe, C., Kabasu Bakajika, P., Francescotti-Paquier, F., Mugellini, E.: Augmented reality treatment for phantom limb pain. In: Shumaker, R., Lackey, S. (eds.) VAMR 2014, Part II. LNCS, vol. 8526, pp. 248–257. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07464-1_23CrossRefGoogle Scholar