AMIE: Automatic Monitoring of Indoor Exercises

  • Tom DecroosEmail author
  • Kurt Schütte
  • Tim Op De Beéck
  • Benedicte Vanwanseele
  • Jesse Davis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11053)


Patients with sports-related injuries need to learn to perform rehabilitative exercises with correct movement patterns. Unfortunately, the feedback a physiotherapist can provide is limited by the number of physical therapy appointments. We study the feasibility of a system that automatically provides feedback on correct movement patterns to patients using a Microsoft Kinect camera and Machine Learning techniques. We discuss several challenges related to the Kinect’s proprietary software, the Kinect data’s heterogeneity, and the Kinect data’s temporal component. We introduce AMIE, a machine learning pipeline that detects the exercise being performed, the exercise’s correctness, and if applicable, the mistake that was made. To evaluate AMIE, ten participants were instructed to perform three types of typical rehabilitation exercises (squats, forward lunges and side lunges) demonstrating both correct movement patterns and frequent types of mistakes, while being recorded with a Kinect. AMIE detects the type of exercise almost perfectly with 99% accuracy and the type of mistake with 73% accuracy. Code related to this paper is available at:



Tom Decroos is supported by the Research Foundation-Flanders (FWO-Vlaanderen). Kurt Schütte and Benedicte Vanwanseele are partially supported by the KU Leuven Research Fund (C22/15/015) and imec.icon research funding. Tim Op De Beéck and Jesse Davis are partially supported by the KU Leuven Research Fund (C22/15/015, C32/17/036).


  1. 1.
    Antón, D., Goñi, A., Illarramendi, A., Torres-Unda, J.J., Seco, J.: KiRes: a kinect-based telerehabilitation system. In: 2013 IEEE 15th International Conference on e-Health Networking, Applications & Services (Healthcom), pp. 444–448 (2013)Google Scholar
  2. 2.
    Bagnall, A., Lines, J., Bostrom, A., Large, J., Keogh, E.: The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 31(3), 606–660 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Borgers, J., Vos, S., Scheerder, J.: Belgium (Flanders). In: Running Across Europe, pp. 28–58. Palgrave Macmillan, London (2015)Google Scholar
  4. 4.
    Campbell, R., Evans, M., Tucker, M., Quilty, B., Dieppe, P., Donovan, J.: Why don’t patients do their exercises? Understanding non-compliance with physiotherapy in patients with osteoarthritis of the knee. J. Epidemiol. Commun. Health 55(2), 132–138 (2001)CrossRefGoogle Scholar
  5. 5.
    Chan, D.K., Lonsdale, C., Ho, P.Y., Yung, P.S., Chan, K.M.: Patient motivation and adherence to postsurgery rehabilitation exercise recommendations: the influence of physiotherapists’ autonomy-supportive behaviors. Arch. Phys. Med. Rehabil. 90(12), 1977–1982 (2009)CrossRefGoogle Scholar
  6. 6.
    Chang, C.Y., et al.: Towards pervasive physical rehabilitation using Microsoft Kinect. In: 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), pp. 159–162. IEEE (2012)Google Scholar
  7. 7.
    Chang, Y.J., Chen, S.F., Huang, J.D.: A kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res. Dev. Disabil. 32(6), 2566–2570 (2011)CrossRefGoogle Scholar
  8. 8.
    Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. ACM (2016)Google Scholar
  9. 9.
    Council, Physical Activity: 2018 participation report: the Physical Activity Council’s annual study tracking sports, fitness, and recreation participation in the US (2017)Google Scholar
  10. 10.
    Fernández-Baena, A., Susín, A., Lligadas, X.: Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments. In: 2012 4th International Conference on Intelligent Networking and Collaborative Systems (INCoS), pp. 656–661. IEEE (2012)Google Scholar
  11. 11.
    Foulds, J., Frank, E.: A review of multi-instance learning assumptions. Knowl. Eng. Rev. 25(1), 1–25 (2010)CrossRefGoogle Scholar
  12. 12.
    Stanford Children’s Health: Sports Injury Statistics (2010).
  13. 13.
    Hootman, J.M., Dick, R., Agel, J.: Epidemiology of collegiate injuries for 15 sports: summary and recommendations for injury prevention initiatives. J. Athl. Train. 42(2), 311–319 (2007)Google Scholar
  14. 14.
    Knight, E., Werstine, R.J., Rasmussen-Pennington, D.M., Fitzsimmons, D., Petrella, R.J.: Physical therapy 2.0: leveraging social media to engage patients in rehabilitation and health promotion. Phys. Ther. 95(3), 389–396 (2015)CrossRefGoogle Scholar
  15. 15.
    Komatireddy, R., Chokshi, A., Basnett, J., Casale, M., Goble, D., Shubert, T.: Quality and quantity of rehabilitation exercises delivered by a 3D motion controlled camera. Int. J. Phys. Med. Rehabil. 2(4) (2014)Google Scholar
  16. 16.
    Lange, B., Chang, C.Y., Suma, E., Newman, B., Rizzo, A.S., Bolas, M.: Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, pp. 1831–1834. IEEE (2011)Google Scholar
  17. 17.
    Levac, D.E., Miller, P.A.: Integrating virtual reality video games into practice: clinicians’ experiences. Physiother. Theory Pract. 29(7), 504–512 (2013)CrossRefGoogle Scholar
  18. 18.
    de Loes, M., Dahlstedt, L., Thomee, R.: A 7-year study on risks and costs of knee injuries in male and female youth participants in 12 sports. Scand. J. Med. Sci. Sports 10(2), 90–97 (2000)CrossRefGoogle Scholar
  19. 19.
    Palazzo, C., et al.: Barriers to home-based exercise program adherence with chronic low back pain: patient expectations regarding new technologies. Ann. Phys. Rehabil. Med. 59(2), 107–113 (2016)CrossRefGoogle Scholar
  20. 20.
    Pedregosa, F., et al.: Scikit-learn: machine learning in python. Journal of machine learning research 12(Oct)Google Scholar
  21. 21.
    Pfister, A., West, A.M., Bronner, S., Noah, J.A.: Comparative abilities of Microsoft Kinect and Vicon 3D motion capture for gait analysis. J. Med. Eng. Technol. 38(5), 274–280 (2014)CrossRefGoogle Scholar
  22. 22.
    Pisters, M.F., et al.: Long-term effectiveness of exercise therapy in patients with osteoarthritis of the hip or knee: a systematic review. Arthritis Care Res. 57(7), 1245–1253 (2007)CrossRefGoogle Scholar
  23. 23.
    Ray, S., Scott, S., Blockeel, H.: Multi-instance learning. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 701–710. Springer, Boston (2011). Scholar
  24. 24.
    Sheu, Y., Chen, L.H., Hedegaard, H.: Sports-and recreation-related injury episodes in the united states, 2011–2014. Natl. Health Stat. Rep. (99), 1–12 (2016)Google Scholar
  25. 25.
    Su, C.J., Chiang, C.Y., Huang, J.Y.: Kinect-enabled home-based rehabilitation system using dynamic time warping and fuzzy logic. Appl. Soft Comput. 22, 652–666 (2014)CrossRefGoogle Scholar
  26. 26.
    Tang, R., Yang, X.D., Bateman, S., Jorge, J., Tang, A.: Physio@home: exploring visual guidance and feedback techniques for physiotherapy exercises. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 4123–4132. ACM (2015)Google Scholar
  27. 27.
    Taylor, M.J., McCormick, D., Shawis, T., Impson, R., Griffin, M.: Activity-promoting gaming systems in exercise and rehabilitation. J. Rehabil. Res. Dev. 48(10), 1171–1186 (2011)CrossRefGoogle Scholar
  28. 28.
    Wojcicki, J.M., Heyman, M.B.: Let’s move - childhood obesity prevention from pregnancy and infancy onward. N. Engl. J. Med. 362(16), 1457–1459 (2010)CrossRefGoogle Scholar
  29. 29.
    Zhao, W.: On automatic assessment of rehabilitation exercises with realtime feedback. In: 2016 IEEE International Conference on Electro Information Technology (EIT), pp. 0376–0381. IEEE (2016)Google Scholar
  30. 30.
    Zhao, W., Feng, H., Lun, R., Espy, D.D., Reinthal, M.A.: A kinect-based rehabilitation exercise monitoring and guidance system. In: 2014 5th IEEE International Conference on Software Engineering and Service Science (ICSESS), pp. 762–765. IEEE (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Tom Decroos
    • 1
    Email author
  • Kurt Schütte
    • 2
  • Tim Op De Beéck
    • 1
  • Benedicte Vanwanseele
    • 2
  • Jesse Davis
    • 1
  1. 1.Department of Computer ScienceKU LeuvenLeuvenBelgium
  2. 2.Department of Movement SciencesKU LeuvenLeuvenBelgium

Personalised recommendations