Patient MoCap: Human Pose Estimation Under Blanket Occlusion for Hospital Monitoring Applications

  • Felix AchillesEmail author
  • Alexandru-Eugen Ichim
  • Huseyin Coskun
  • Federico Tombari
  • Soheyl Noachtar
  • Nassir Navab
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9900)


Motion analysis is typically used for a range of diagnostic procedures in the hospital. While automatic pose estimation from RGB-D input has entered the hospital in the domain of rehabilitation medicine and gait analysis, no such method is available for bed-ridden patients. However, patient pose estimation in the bed is required in several fields such as sleep laboratories, epilepsy monitoring and intensive care units. In this work, we propose a learning-based method that allows to automatically infer 3D patient pose from depth images. To this end we rely on a combination of convolutional neural network and recurrent neural network, which we train on a large database that covers a range of motions in the hospital bed. We compare to a state of the art pose estimation method which is trained on the same data and show the superior result of our method. Furthermore, we show that our method can estimate the joint positions under a simulated occluding blanket with an average joint error of 7.56 cm.


Pose estimation Motion capture Occlusion CNN RNN Random forest 



The authors would like to thank Leslie Casas and David Tan from TUM and Marc Lazarovici from the Human Simulation Center Munich for their support. This work has been funded by the German Research Foundation (DFG) through grants NA 620/23-1 and NO 419/2-1.


  1. 1.
    Stone, E.E., Skubic, M.: Unobtrusive, continuous, in-home gait measurement using the microsoft kinect. IEEE Trans. Biomed. Eng. 60(10), 2925–2932 (2013)CrossRefGoogle Scholar
  2. 2.
    Kontschieder, P., Dorn, J.F., Morrison, C., Corish, R., Zikic, D., Sellen, A., D’Souza, M., Kamm, C.P., Burggraaff, J., Tewarie, P., Vogel, T., Azzarito, M., Glocker, B., Chin, P., Dahlke, F., Polman, C., Kappos, L., Uitdehaag, B., Criminisi, A.: Quantifying progression of multiple sclerosis via classification of depth videos. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part II. LNCS, vol. 8674, pp. 429–437. Springer, Heidelberg (2014)Google Scholar
  3. 3.
    Cunha, J., Choupina, H., Rocha, A., Fernandes, J., Achilles, F., Loesch, A., Vollmar, C., Hartl, E., Noachtar, S.: NeuroKinect: a novel low-cost 3Dvideo-EEG system for epileptic seizure motion quantification. PLOS ONE 11(1), e0145669 (2015)CrossRefGoogle Scholar
  4. 4.
    Benbadis, S.R., LaFrance, W., Papandonatos, G., Korabathina, K., Lin, K., Kraemer, H., et al.: Interrater reliability of eeg-video monitoring. Neurology 73(11), 843–846 (2009)CrossRefGoogle Scholar
  5. 5.
    Li, Y., Berkowitz, L., Noskin, G., Mehrotra, S.: Detection of patient’s bed statuses in 3D using a microsoft kinect. In: EMBC. IEEE (2014)Google Scholar
  6. 6.
    Yu, M.-C., Wu, H., Liou, J.-L., Lee, M.-S., Hung, Y.-P.: Multiparameter sleep monitoring using a depth camera. In: Schier, J., Huffel, S., Conchon, E., Correia, C., Fred, A., Gamboa, H., Gabriel, J. (eds.) BIOSTEC 2012. CCIS, vol. 357, pp. 311–325. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56(1), 116–124 (2013)CrossRefGoogle Scholar
  8. 8.
    Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A.: Efficient regression of general-activity human poses from depth images. In: ICCV. IEEE (2011)Google Scholar
  9. 9.
    Belagiannis, V., Rupprecht, C., Carneiro, G., Navab, N.: Robust optimization for deep regression. In: ICCV. IEEE (2015)Google Scholar
  10. 10.
    Fragkiadaki, K., Levine, S., Felsen, P., Malik, J.: Recurrent network models for human dynamics. In: ICCV. IEEE (2015)Google Scholar
  11. 11.
    Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
  12. 12.
    Museth, K., Lait, J., Johanson, J., Budsberg, J., Henderson, R., Alden, M., Cucka, P., Hill, D., Pearce, A.: OpenVDB: an open-source data structure and toolkit for high-resolution volumes. In: ACM SIGGRAPH 2013 Courses. ACM (2013)Google Scholar
  13. 13.
    Bouaziz, S., Martin, S., Liu, T., Kavan, L., Pauly, M.: Projective dynamics: fusing constraint projections for fast simulation. ACM Trans. Graph. (TOG) 33(4), 154 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Felix Achilles
    • 1
    • 2
    Email author
  • Alexandru-Eugen Ichim
    • 3
  • Huseyin Coskun
    • 1
  • Federico Tombari
    • 1
    • 4
  • Soheyl Noachtar
    • 2
  • Nassir Navab
    • 1
    • 5
  1. 1.Computer Aided Medical ProceduresTechnische Universität MünchenMunichGermany
  2. 2.Department of NeurologyLudwig-Maximilians-University of MunichMunichGermany
  3. 3.Graphics and Geometry LaboratoryEPFLLausanneSwitzerland
  4. 4.DISIUniversity of BolognaBolognaItaly
  5. 5.Computer Aided Medical ProceduresJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations