Skip to main content
SpringerLink
Log in

i-Therapy: a non-invasive multimedia authoring framework for context-aware therapy design

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present an e-Therapy framework, named i-Therapy, which collects live therapeutic context by analyzing multi-sensory body joint data in a non-invasive way. Using our proposed framework, a therapist can model complex gestures by mapping them to a set of primitive gesture sequences and generate high-level therapies. As a proof of concept, we have developed scenarios to express a Hemiplegic patient’s behavior into a set of track-able primitive gestures. The initial feedback from the therapists who have tested our developed framework is encouraging. Finally, we share the implementation details and analysis of test results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.leapmotion.com/

  2. http://www.microsoft.com/en-us/kinectforwindows/

  3. http://www.naturalpoint.com/optitrack/

  4. http://openni.org/

  5. http://www.primesense.com/

  6. http://threejs.org/

  7. http://jpgraph.net/

  8. http://www.merckmanuals.com/professional/special_subjects/rehabilitation/physical_therapy_pt.html

References

  1. Alamri A, and Kim HN, and Cha J, and El Saddik A (2012) Serious Games for Rehabilitation of Stroke Patients with Vibrotactile Feedback. Int. J. Comp. Sci. Sport. 9, 2

  2. Albrecht I, Haber J, and Seidel HP (2003) Construction and animation of anatomically based human hand models. In Proceedings of SCA. 98–109

  3. An J, and Pack S. and An S, and Kim M, and Jeon Y (2009) A novel service architecture for personalized context aware services. In Proceedings of 11th Int. Conf. Advanced Communication Technology. 1, 554–559

  4. Anderson SA, and Blandine CG (1993) Anatomy of Movement. Chicago: Eastland Press. ISBN 0-939616-17-3

  5. Aziz O et al (2007) A pervasive body sensor network for measuring postoperative recovery at home. J Surg Innov 14:83–90, 2

    Article  Google Scholar 

  6. Baran M, Lehrer N, Siwiak D, ChenY, Duff M. and Ingalls T. and Rikakis T (2011) Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors. In Engineering in Medicine and Biology Society, EMBC, 7602–7605

  7. Bardram JE (2005) Activity-Based Computing: Support for Mobility and Collaboration in Ubiquitous Computing. Pers Ubiquit Comput Springer-Verlag 9:312–322, 5

    Article  Google Scholar 

  8. Bhardwaj S, Lee D-S, Mukhopadhyay SC, Chung W-Y (2008) Ubiquitous Healthcare Data Analysis and Monitoring Using Multiple Wireless Sensors for Elderly Person. Sensors Transducers J 90:87–99

    Google Scholar 

  9. Bottazzi D, Corradi A, and Montanari R (2006) Context-Aware Middleware Solutions for Anytime and Anywhere Emergency Assistance to Elderly People. IEEE Communications Magazine. 82–90

  10. Chang Y, Chen S, Huang J (2011) A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res Dev Disabil 32(6):2566–2570

    Article  Google Scholar 

  11. Chang C, Lange B, Zhang M, Koenig S, Requejo P, Somboon N, Sawchuk AA, Rizzo AA (2012) Towards Pervasive Physical Rehabilitation Using Microsoft Kinect. In Proceedings of 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). San Diego, California USA, 159–162

  12. Clarka RA, Puab Y-H, Fortina K, Ritchiea C, Websterc KE, Denehya L, Bryant AL (2012) Validity of the Microsoft Kinect for assessment of postural control. Gait Posture 36(3):372–377

    Article  Google Scholar 

  13. Dey AK (2001) Understanding and Using Context. Pers Ubiquit Comput Springer-Verlag 5(1):4–7

    Article  Google Scholar 

  14. Erdemir A, McLean S, Herzog W, van den Bogert AJ (2007) Model-based estimation of muscle forces exerted during movements. J Clin Biomech Elsevier 22(2):131–54

    Article  Google Scholar 

  15. Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: A review. Comput Vis Image Underst 108(1–2):52–57

    Article  Google Scholar 

  16. e-Therapy (2009) Considerations for the Provision of E-Therapy. http://store.samhsa.gov/shin/content/SMA09-4450/SMA09-4450.pdf

  17. Gabel M, Renshaw E, Schuster A, and Gilad-Bachrach R (2012) Full Body Gait Analysis with Kinect. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBC). 1964–1967

  18. Gama A, Chaves T, Figueiredo L, and Teichrieb V (2012) Poster: Improving Motor Rehabilitation Process through a Natural Interaction Based System Using Kinect Sensor. In Proceedings of IEEE Symposium on 3D User Interfaces. Orange County, CA USA, 145–146

  19. Gips J, Betke M, DiMattia PA (2007) Early experiences using visual tracking for computer access by people with profound physical disabilities. Disabil Rehabil: Assist Technol 2:189–199, 4

    Google Scholar 

  20. Gorini A, Gaggioli A, Vigna C, Riva G (2008) A Second Life for eHealth: Prospects for the Use of 3-D Virtual Worlds in Clinical Psychology. J Med Internet Res 10(3):e21

    Article  Google Scholar 

  21. Green CS, Bavelier D (2007) Action-video-game experience alters the spatial resolution of vision. Psychol Sci 18:88–94, 1

    Article  Google Scholar 

  22. Harley L. et al. (2011) The Design of an Interactive Stroke Rehabilitation Gaming System. In Proceedings of the 14th Intl. Conf. Human-computer interaction: users and applications. Orlando FL, 167–173

  23. Holden MK (2005) Virtual Environments for Motor Rehabilitation: Review. Cyber Psychol Behavior 8:187–211, 3

    Article  Google Scholar 

  24. Hovet L, Ryall K, Mcdonnell R, and O’Sullivan C (2012) Sleight of hand: perception of finger motion from reduced marker sets. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 2, 79–86

  25. Huang Jun-Da (2011) Kinerehab: a kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. In Proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '11). ACM, New York, NY, USA, 319–320

  26. Huber M, Rabin B, Docan C, Burdea G, Nwosu ME, AbdelBaky M, and Golomb MR (2008) PlayStation 3-based tele-rehabilitation for children with hemiplegia. Virtual Rehabilitation. 105–112

  27. Hudes K (2011) Conservative management of a case of medial epicondylosis in a recreational squash player. J Can Chiropr Assoc 55(1):26–31

    Google Scholar 

  28. Khoshelham K, Elberink SO (2012) Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors 12:1437–1454

    Article  Google Scholar 

  29. Kim D, and Hilliges O, and Iazddi S, and Butler A, and Chen J. and Oikonomidis I, and Olivier P (2012) Digits: Free 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor Dithered color quantization. In Proceedings of ACM UIST, 167–176

  30. Koolwaaij J, and Wibbels M, and Böhm S, and Luther M (2009) ContextKing: Virtual Kingdoms in Real Life. In Proceedings of 1st IEEE Int. Conf. Games and Virtual Worlds. 61–67

  31. Krco S (2004) Health care sensor networks-architecture and protocols. Ad Hoc Sensor Wirel Networks 1:1–25

    Google Scholar 

  32. LaBelle K (2011) Evaluation of Kinect Joint Tracking for Clinical and In-Home Stroke Rehabilitation Tools. Undergraduate thesis, Notre Dame

    Google Scholar 

  33. Lange B. et al. (2012) Interactive game-based rehabilitation using the Microsoft Kinect. In Proceedings of IEEE Virtual Reality. 171–172

  34. Liu CK (2009) Dextrous manipulation from a grasping pose. ACM Trans Graph 28(3):e59

    Google Scholar 

  35. Livingston MA. et al. (2012) Performance Measurements for the Microsoft Kinect Skeleton. In Proceedings of IEEE Virtual Reality. Orange County, CA, USA, 119–120

  36. Ma M, and Bechkoum K (2008) Serious Games for Movement Therapy after Stroke. Systems. In Proceedings of IEEE International Conference on Man and Cybernetics. Singapore, 1872–1877

  37. Ma M, and McNeill M. and Charles D, and McDonough S, and Crosbie J, and Oliver L, and McGoldrick C (2007) Adaptive virtual reality games for rehabilitation of motor disorders. In Proceedings of the 4th international conference on Universal access in human-computer interaction: ambient interaction (UAHCI'07). Constantine Stephanidis (Ed.). Springer-Verlag, Berlin, Heidelberg, 681–690

  38. Mehm F, Hardy S, Göbel S, and Steinmetz R (2011) Collaborative Authoring of Serious Games for Health. In Proceedings of the 19th ACM international conference on Multimedia. ACM New York, NY, USA, 807–808

  39. Norris J (2009) The Growth and Direction of Healthcare Support Groups in Virtual Worlds. 3D Virtual Worlds for Health and Healthcare. 2, 2

  40. Phung D, and Adams B, and Tran K, and Venkatesh S, and Kumar M (2009) High accuracy context recovery using clustering mechanisms. In Proceedings of IEEE Int. Conf. Pervasive Computing and Communications. 1–9

  41. Pollard NS, and Zordan VB (2005) Physically based grasping control from example. In Proceedings of SCA. 311–318

  42. Rahman MdA, and Basalamah S, and Toonsi AH, and El Saddik A (2013b) A Framework Toward Detecting and Visualizing Kinematic Data for Children with Hemiplegia. In Proceedings of the 15th IEEE International Conference on e-Health Networking, Application & Services (IEEE Healthcom, 2013). Lisbon, Portugal, October 9 – 12

  43. Rahman MdA, and El Saddik A (2013c) Modeling e-Health Framework towards Personalized and Context-Aware Multimedia Services. Handbook of Innovative Medical Technologies, Springer, eds. Borko Furht and Ankur Agarwal, ISBN 978-1-4614-8494-3

  44. Rahman MdA, and El Saddik A, and Gueaieb W (2011) Augmenting Context Awareness by Combining Body Sensor Networks and Social Networks. IEEE Transactions on Instrumentation and Measurement. 60, 2, 345–353

  45. Rahman MdA, and Hamdan S, and El. Saddik A, and Gueaieb W (2010) Context-Aware Social Networks Mashup: A Personalized Web Perspective. In Proceedings of IEEE Int. Instrumentation and Measurement Technology Conference (I2MTC 2010). Austin, Tx, USA, 1228–1233

  46. Rahman MdA, and Qamar AM, and Ahmed MA, and Rahman MA, and Basalamah S (2013a) Multimedia Interactive Therapy Environment for Children Having Physical Disabilities. In Proceedings of ACM ICMR. Dallas, USA, 313–314

  47. Roslyn B, and Chuanchu W, and Kok Soon P, and Kai Keng A. et al. (2012) A feasibility study of non-invasive motor-imagery BCI-based robotic rehabilitation for Stroke patients. In Proceedings of 4th International IEEE/EMBS Conference on Neural Engineering. 271–274

  48. Ryu H, and Park I, and Hyun SJ, and Lee D (2007) A Task Decomposition Scheme for Context Aggregation in Personal Smart Space. Software Technologies for Embedded and Ubiquitous Systems, Lecture Notes in Computer Science. Springer Berlin/Heidelberg, 20–29

  49. Saini S, and Sulaiman S, and Zakaria M, and Shukri S (2012) A Low-cost Game Framework for a Home-based Stroke Rehabilitation System. In Proceedings of International Conference on Computer and Information Science (ICCIS). 1, 55–60

  50. Sakzewski L, Ziviani J, Boyd R (2009) Systematic Review and Meta-analysis of Therapeutic Management of Upper-Limb Dysfunction in Children with Congenital Hemiplegia. Pediatrics 123(6):1111–1122

    Article  Google Scholar 

  51. Sarikaya S, and Alim MA, and Rezaei S (2006) Integrating wireless EEGs into medical sensor networks. In Proceedings of ACM IWCMC. Vancouver, British Columbia, Canada, 1369–1373

  52. Schönauer C, and Pintaric T, and Kaufmann H, and Jansen-Kosterink S, and Vollenbroek-Hutten M (2011) Chronic Pain Rehabilitation with a Serious Game using Multimodal Input. In Proceedings of International Conference on Virtual Rehabilitation. Zurich Switzerland

  53. Stone EE. and Skubic M (2011) Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBC). Boston, Massachusetts, USA, 6491–6494.

  54. Sueda S, Kaufman A, PAI DK (2008) Musculotendon simulation for hand animation. ACM Trans Graph 27(3):e83

    Article  Google Scholar 

  55. Suma E, and Lange B, and Rizzo A, and Krum D, and Bolas M (2011) FAAST: The Flexible Action and Articulated Skeleton Toolkit. In Proceedings of IEEE Virtual Reality. 247–248

  56. Taylor RM, and Hudson TC, and Seeger A, and Weber H, and Juliano J, and Helser AT (2001) VRPN: a device-independent, network-transparent VR peripheral system. In ACM Virtual Reality Software & Technology. 55–61

  57. Tong X, and Xu P, and Yan X (2012) Research on skeleton animation motion data based on Kinect. In Proceedings of 5th IEEE International Symposium on Computational Intelligence and Design. Hangzhou, China, 347–350

  58. Tsang W, and Singh K, and Eugene F (2005) Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. In Proceedings of SCA, pp. 319–328

  59. Wu WH (2008) MEDIC: Medical embedded device for individualized care. Journal of Artificial Intelligence in Medicine. Elsevier, 42, 2, 137–152

  60. Yang BH, Rhee S (2000) Development of the ring sensor for healthcare automation. Robot Auton Syst 30:273–281

    Article  Google Scholar 

  61. Zhao W, and Chai J, and XU Y (2012) Combining Marker-based Mocap and RGB-D Camera for Acquiring High-fidelity Hand Motion Data. ACM SIGGRAPH, 33–42

  62. Zhou H, Hu H (2008) Human motion tracking for rehabilitation—A survey. Biomed Sig Process Control 3:1–18

    Article  Google Scholar 

Download references

Acknowledgments

This project was supported by the NSTIP strategic technologies program (11-INF1703-10) in the Kingdom of Saudi Arabia. The authors would like to thank the therapists who have given us feedback and domain specific knowledge. The authors would also like to thank Dr. Farooque Alwari, Ahmad Qamar and Delwar Hossain of Advanced Media Laboratory of Umm Al-Qura University for helping in demo and usability testing.

Author information

Authors and Affiliations

  1. Advanced Media Laboratory, College of Computer and Information Systems, Umm Al-Qura University, Makkah, Saudi Arabia

    Md. Abdur Rahman

Authors
  1. Md. Abdur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md. Abdur Rahman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahman, M.A. i-Therapy: a non-invasive multimedia authoring framework for context-aware therapy design. Multimed Tools Appl 75, 1843–1867 (2016). https://doi.org/10.1007/s11042-014-2376-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2376-5

Keywords

Search

Navigation