The Role of Serious Games in Robot Exoskeleton-Assisted Rehabilitation of Stroke Patients

  • David J. Cornforth
  • Alexander Koenig
  • Robert Riener
  • Katherine August
  • Ahsan H. Khandoker
  • Chandan Karmakar
  • Marimuthu Palaniswami
  • Herbert F. Jelinek
Part of the Advances in Game-Based Learning book series (AGBL)


This chapter describes how serious games can be used to improve the rehabilitation of stroke patients. Determining ideal training conditions for rehabilitation is difficult, as no objective measures exist and the psychological state of patients during therapy is often neglected. What is missing is a way to vary the difficulty of the tasks during a therapy session in response to the patient needs, in order to adapt the training specifically to the individual. In this chapter, we describe such a method. A serious game is used to present challenges to the patient, including motor and cognitive tasks. The psychological state of the patient is inferred from measures computed from heart rate variability (HRV) as well as breathing frequency, skin conductance response, and skin temperature. Once the psychological state of the patient can be determined from these measures, it is possible to vary the tasks in real time by adjusting parameters of the game. The serious game aspect of the training allows the virtual environment to become adaptive in real time, leading to improved matching of the activity to the needs of the patient. This is likely to lead to improved training outcomes and has the potential to lead to faster and more complete recovery, as it enables training that is challenging yet does not overstress the patient.


Robot exoskeleton Stroke rehabilitation Physiological measurements Closed loop difficulty 


  1. Adamovich, S. V., Fluet, G. G., Tunik, E., & Merians, A. S. (2009). Sensorimotor training in virtual reality: A review. NeuroRehabilitation, 25(1), 29–44.Google Scholar
  2. Aisen, M. L., Krebs, H. I., Hogan, N., McDowell, F., & Volpe, B. T. (1997). The effect of robot-assisted therapy and rehabilitative training on motor recovery following stroke. Archives of Neurology, 54, 443–446.CrossRefGoogle Scholar
  3. Amano, M., Oida, E., & Moritani, T. (2005). Age-associated alteration of sympatho-vagal balance in a female population assessed through the tone-entropy analysis. European Journal of Applied Physiology, 94, 602–610.CrossRefGoogle Scholar
  4. Andreassi, J. L. (2007). Psychophysiology: Human behavior and physiological response 5th ed. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  5. Arazpour, M., Mehrpour, S. R., Bani, M. A., Hutchins, S. W., Bahramizadeh, M., & Rahgozar, M. (2014). Comparison of gait between healthy participants and persons with spinal cord injury when using a powered gait orthosis—A pilot study. Spinal Cord, 52(1), 44–48.CrossRefGoogle Scholar
  6. Banz, R., Bolliger, M., Colombo, G., Dietz, V., & Lunenburger, L. (2008). Computerized visual feedback: An adjunct to robotic-assisted gait training. Physical Therapy, 88, 1135–1145.CrossRefGoogle Scholar
  7. Berntson, G. G., Bigger, J. T., Jr., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., et al. (1997). Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology, 34(6), 623–648.CrossRefGoogle Scholar
  8. Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59.CrossRefGoogle Scholar
  9. Brennan, M., Palaniswami, M., & Kamen, P. (2001). New insights into the relationship between Poincare plot geometry and linear measures of heart rate variability. In Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Vol. 1, pp. 526–529).Google Scholar
  10. Brown, R., & Macefield, V. G. (2014). Skin sympathetic nerve activity in humans during exposure to emotionally-charged images: Sex differences. Frontiers in Physiology, 5, 111.CrossRefGoogle Scholar
  11. Burke, J. W., McNeill, M. D. J., Charles, D. K., Morrow, P. J., Crosbie, J. H., & McDonough, S. M. (2009). Optimising engagement for stroke rehabilitation using serious games. The Visual Computer, 25(12), 1085–1099.Google Scholar
  12. Carroll, D., Turner, J. R., & Prasad, R. (1986). The effects of level of difficulty of mental arithmetic challenge on heart rate and oxygen-consumption. International Journal of Psychophysiology, 4, 167–173.CrossRefGoogle Scholar
  13. Christov, I. (2004). Real time electrocardiogram QRS detection using combined adaptive threshold. Biomedical Engineering Online, 3, 8.CrossRefGoogle Scholar
  14. Colombo, G., Joerg, M., Schreier, R., & Dietz, V. (2000). Treadmill training of paraplegic patients using a robotic orthosis. Journal of Rehabilitation Research and Development, 37, 693–700.Google Scholar
  15. Dawson, M. E., Schell, A. M., & Filion, D. L. (2008). Handbook of psychophysiology (3rd ed.). New York: Cambridge University Press.Google Scholar
  16. de la Cruz Torres, B., Lopez, C. L., & Orellana, J. N. (2008). Analysis of heart rate variability at rest and during aerobic exercise: A study in healthy people and cardiac patients. British Journal of Sports Medicine, 42(9), 715–720.CrossRefGoogle Scholar
  17. Delaney, J. P. A., & Brodie, D. A. (2000). Effects of short-term psychological stress on the time and frequency domains of heart-rate variability. Perceptual and Motor Skills, 91, 515–524.CrossRefGoogle Scholar
  18. Dietterich, T. G., & Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2, 263–286.Google Scholar
  19. Dietz, V., & Duysens, J. (2000). Significance of load receptor input during locomotion: A review. Gait & Posture, 11(2), 102–110.CrossRefGoogle Scholar
  20. Doberenz, S., Roth, W. T., Wollburg, E., Maslowski, N. I., & Kim, S. (2011). Methodological considerations in ambulatory skin conductance monitoring. International Journal of Psychophysiology, 80(2), 87–95.CrossRefGoogle Scholar
  21. Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern classification and scene analysis. London: Wiley.Google Scholar
  22. Guadagnoli, M. A., & Lee, T. D. (2004). Challenge point: A framework for conceptualizing the effects of various practice conditions in motor learning. Journal of Motor Behavior, 36, 212–224.CrossRefGoogle Scholar
  23. Gunther, A., Witte, O. W., & Hoyer, D. (2010). Autonomic dysfunction and risk stratification assessed from heart rate pattern. Open Neurology Journal, 4, 39–49.Google Scholar
  24. Hamilton, P. S., & Tompkins, W. J. (1986). Quantitative investigation of QRS detection rules using the MIT/BIH arrhythmia database. IEEE Transactions on Biomedical Engineering, BME-33, 1157–1165.CrossRefGoogle Scholar
  25. Harro, C. C., Shoemaker, M. J., Frey, O., Gamble, A. C., Harring, K. B., Karl, K. L., et al. (2014). The effects of speed-dependent treadmill training and rhythmic auditory-cued overground walking on balance function, fall incidence, and quality of life in individuals with idiopathic Parkinson's disease: A randomized controlled trial. NeuroRehabilitation, 34(3), 541–556.Google Scholar
  26. Hirshfield, L. M., Bobko, P., Barelka, A., Hirshfield, S. H., Farrington, M. T, Gulbronson, S., et al. (2014). Using noninvasive brain measurement to explore the psychological effects of computer malfunctions on users during human–computer interactions. Advances in Human-Computer Interaction. doi:  10.1155/2014/101038.
  27. Holden, M. K. (2005). Virtual environments for motor rehabilitation: Review. CyberPsychology & Behavior, 8, 187–211.CrossRefGoogle Scholar
  28. Houtsma, J. A., & Van Houten, F. J. (2006). Virtual reality and a haptic master-slave set-up in post-stroke upper-limb rehabilitation. Proceedings of the Institute of Mechanical Engineers H, 220, 715–718.CrossRefGoogle Scholar
  29. Jelinek, H. F., August, K. G., Imam, Md. H., Khandoker, A. H., Khalaf, K., Koenig, A., et al. (2014). Influence of stroke location on heart rate variability in robot-assistive neurorehabilitation. In Proceedings of the 2nd Middle East Conference on Biomedical Engineering (pp. 253–256). ISBN: 978-1-4799-4799-7/14.Google Scholar
  30. Jelinek, H. F., August, K., Khandoker, A., Issam, H. M., Koenig, A., & Riener, R. (2011). Heart rate asymmetry and emotional response to robot-assist task challenges in post-stroke patients. In Proceedings of the Computers in Cardiology Conference (Vol. 38, pp. 521–524).Google Scholar
  31. Karmakar, C. K., Khandoker, A. H., Gubbi, J., & Palaniswami, M. (2011). Defining asymmetry in heart rate variability signals using a Poincaré plot. Physiological Measurement, 30, 1227–1240.CrossRefGoogle Scholar
  32. Karmakar, C. K., Khandoker, A. H., Jelinek, H. F., & Palaniswami, M. (2013). Risk stratification of cardiac autonomic neuropathy based on multi-scale tone-entropy. Medical and Biological Engineering and Computing, 51(5), 537–546. doi: 10.1007/s11517-012-1022-5.CrossRefGoogle Scholar
  33. Karmakar, C. K., Khandoker, A. H., Voss, A., & Palaniswami, M. (2011). Sensitivity of temporal heart rate variability in Poincaré plot to changes in parasympathetic nervous system activity. Biomedical Engineering Online, 10, 17.CrossRefGoogle Scholar
  34. Khandoker, A. H., Jelinek, H. F., Moritani, T., & Palaniswami, M. (2010). Association of cardiac autonomic neuropathy with alteration of sympatho-vagal balance through heart rate variability analysis. Medical Engineering and Physics, 32, 61–67.CrossRefGoogle Scholar
  35. Koenig, A., Omlin, X., Zimmerli, L., Sapa, M., Krewer, C., Bolliger, M., et al. (2011). Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation. Journal of Rehabilitation Research and Development, 48(4), 367–385.CrossRefGoogle Scholar
  36. Lakusic, N., Mahovic, D., Babic, T., & Sporis, D. (2003). Changes in autonomic control of heart rate after ischemic cerebral stroke. Acta Medica Croatica, 57(4), 269–273.Google Scholar
  37. Lotze, M., Braun, C., Birbaumer, N., Anders, S., & Cohen, L. G. (2003). Motor learning elicited by voluntary drive. Brain, 126, 866–872.CrossRefGoogle Scholar
  38. Mackersie, C. L., & Cones, H. (2011). Subjective and psychophysiological indexes of listening effort in a competing-talker task. Journal of the American Academy of Audiology, 22(2), 113–122.CrossRefGoogle Scholar
  39. Maclean, N., Pound, P., Wolfe, C., & Rudd, A. (2002). The concept of patient motivation. Stroke, 33(2), 444–448.CrossRefGoogle Scholar
  40. Malik, M., & Camm, A. J. (Eds.). (1995). Heart rate variability. Armonk, NY: Futura.Google Scholar
  41. Mancuso, D. L, Knight, K. L. (1992). Effects of prior physical activity on skin surface temperature response of the ankle during and after a 30-minute ice pack application. Journal of Athletic Training, 27(3), 242, 244, 246, 248-249.Google Scholar
  42. McAuley, E., Duncan, T., & Tammen, V. V. (1989). Psychometric properties of the intrinsic motivation inventory in a competitive sports setting—A confirmatory factor analysis. Research Quarterly for Exercise and Sport, 60, 48–58.CrossRefGoogle Scholar
  43. Meyer, T., Peters, J., Zander, T., Scholkopf, B., & Grosse-Wentrup, M. (2014). Predicting motor learning performance from electroencephalographic data. Journal of NeuroEngineering and Rehabilitation, 11(1), 24.CrossRefGoogle Scholar
  44. Moré, J. J. (1977). The Levenberg-Marquardt algorithm: implementation and theory. In G. A. Watson (Ed.), Numerical analysis (Lecture notes in mathematics, Vol. 630, pp. 105–116). New York: Springer.CrossRefGoogle Scholar
  45. Morris, J. D. (1995). Observations: SAM—The Self Assessment Mannequin: An efficient cross-cultural measurement of emotional response. Journal of Advertising Research, 35(6), 63–70.Google Scholar
  46. Nef, T., Guidali, M., & Riener, R. (2009). ARMin III—Arm therapy exoskeleton with an ergonomic shoulder actuation. Applied Bionics and Biomechanics, 6, 16.CrossRefGoogle Scholar
  47. Nef, T., Mihelj, M., & Riener, R. (2007). ARMin: A robot for patient-cooperative arm therapy. Medical and Biological Engineering and Computing, 45, 887–900.CrossRefGoogle Scholar
  48. Nguyen, D., & Widrow, B. (1990). Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In Proceedings of the IJCNN International Joint Conference on Neural Networks (Vol. 3, pp. 21–26).Google Scholar
  49. Ohsuga, M., Shimono, F., & Genno, H. (2001). Assessment of phasic work stress using autonomic indices. International Journal of Psychophysiology, 40, 211–220.CrossRefGoogle Scholar
  50. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine Series 6, 2(11), 559–572.CrossRefGoogle Scholar
  51. Porta, A., Casali, K. R., Casali, A. G., Gnecchi-Ruscone, T., Tovaldini, E., Montano, N., et al. (2008). Temporal asymmetries of short-term heart period variability are linked to autonomic regulation. American Journal of Physiology—Regulatory, Integrative and Comparative Physiology, 295, R550557.CrossRefGoogle Scholar
  52. Rego, P., Moreira, P. M., & Reis, L. P. (2011). Serious games for rehabilitation: A survey and a classification towards a taxonomy. In: 2010 5th Iberian Conference on Information Systems and Technologies (CISTI) (pp. 1–6).Google Scholar
  53. Rosenblueth, A., & Simeone, A. (1984). The interrelations of vagal and accelerator effects on the cardiac rate. American Journal of Physiology, 110, 42–55.Google Scholar
  54. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178.CrossRefGoogle Scholar
  55. Saposnik, G., Teasell, R., Mamdani, M., Hall, J., McIlroy, W., Cheung, D., et al. (2010). Effectiveness of virtual reality using Wii gaming technology in stroke rehabilitation: A pilot randomized clinical trial and proof of principle. Stroke, 41(7), 1477–1484.CrossRefGoogle Scholar
  56. Stauffer, Y., Allemand, Y., Bouri, M., Fournier, J., Clavel, R., Metrailler, P., et al. (2009). The WalkTrainer—A new generation of walking reeducation device combining orthoses and muscle stimulation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 17, 38–45.CrossRefGoogle Scholar
  57. Suess, W. M., Alexander, A. B., Smith, D. D., Sweeney, H. W., & Marion, R. J. (1980). The effects of psychological stress on respiration—A preliminary-study of anxiety and hyperventilation. Psychophysiology, 17, 535–540.CrossRefGoogle Scholar
  58. Thayer, J. F., Hansen, A. L., Saus-Rose, E., & Johnsen, B. H. (2009). Heart rate variability, prefrontal neural function, and cognitive performance: The neurovisceral integration perspective on self-regulation, adaptation and health. Annals of Behavioral Medicine, 37, 141–153.Google Scholar
  59. Veneman, J. F., Kruidhof, R., Hekman, E. E., Ekkelenkamp, R., Van Asseldonk, E. H., & van der Kooij, H. (2007). Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 15, 379–386.CrossRefGoogle Scholar
  60. Vichitvanichphong, S., Talaei-Khoei, A., Kerr, D., & Ghapanchi, A. H. (2014). Adoption of assistive technologies for aged care: A realist review of recent studies. In: 2014 47th Hawaii International Conference on System Sciences (HICSS) (pp. 2706–2715). doi: 10.1109/HICSS.2014.341.
  61. Winchester, P., & Querry, R. (2006). Robotic orthoses for body weight-supported treadmill training. Physical Medicine and Rehabilitation Clinics of North America, 17, 159–172.CrossRefGoogle Scholar
  62. Wiemeyer, J., & Kliem, A. (2012). Serious games in prevention and rehabilitation—A new panacea for elderly people? European Review of Aging and Physical Activity, 9(1), 41–50. doi: 10.1007/s11556-011-0093-x.
  63. Zimmerli, L., Duschau-Wicke, A., Mayr, A., Riener, R., & Lunenburger, L. (2009). Virtual reality and gait rehabilitation: Augmented feedback for the Lokomat. In Proceedings of the IEEE Virtual Rehabilitation International Conference (pp. 150–153).Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • David J. Cornforth
    • 1
  • Alexander Koenig
    • 2
  • Robert Riener
    • 2
  • Katherine August
    • 2
  • Ahsan H. Khandoker
    • 3
  • Chandan Karmakar
    • 3
  • Marimuthu Palaniswami
    • 4
  • Herbert F. Jelinek
    • 5
  1. 1.University of Newcastle, AustraliaCallaghanAustralia
  2. 2.ETH ZurichZurichSwitzerland
  3. 3.The University of MelbourneParkvilleAustralia
  4. 4.The University of MelbourneParkvilleAustralia
  5. 5.Charles Sturt UniversityAlburyAustralia

Personalised recommendations