Advertisement

Learning an Infant Body Model from RGB-D Data for Accurate Full Body Motion Analysis

  • Nikolas HesseEmail author
  • Sergi Pujades
  • Javier Romero
  • Michael J. Black
  • Christoph Bodensteiner
  • Michael Arens
  • Ulrich G. Hofmann
  • Uta Tacke
  • Mijna Hadders-Algra
  • Raphael Weinberger
  • Wolfgang Müller-Felber
  • A. Sebastian Schroeder
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11070)

Abstract

Infant motion analysis enables early detection of neurodevelopmental disorders like cerebral palsy (CP). Diagnosis, however, is challenging, requiring expert human judgement. An automated solution would be beneficial but requires the accurate capture of 3D full-body movements. To that end, we develop a non-intrusive, low-cost, lightweight acquisition system that captures the shape and motion of infants. Going beyond work on modeling adult body shape, we learn a 3D Skinned Multi-Infant Linear body model (SMIL) from noisy, low-quality, and incomplete RGB-D data. SMIL is publicly available for research purposes at http://s.fhg.de/smil. We demonstrate the capture of shape and motion with 37 infants in a clinical environment. Quantitative experiments show that SMIL faithfully represents the data and properly factorizes the shape and pose of the infants. With a case study based on general movement assessment (GMA), we demonstrate that SMIL captures enough information to allow medical assessment. SMIL provides a new tool and a step towards a fully automatic system for GMA.

Keywords

Body models Data-driven Cerebral palsy Motion analysis Pose tracking General movement assessment 

References

  1. 1.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005)CrossRefGoogle Scholar
  2. 2.
    Bogo, F., Black, M.J., Loper, M., Romero, J.: Detailed full-body reconstructions of moving people from monocular RGB-D sequences. In: IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  3. 3.
    Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., Black, M.J.: Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016, Part V. LNCS, vol. 9909, pp. 561–578. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_34CrossRefGoogle Scholar
  4. 4.
    Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  5. 5.
    Hadders-Algra, M.: General movements: a window for early identification of children at high risk for developmental disorders. J. Pediatr. 145(2), S12–S18 (2004)CrossRefGoogle Scholar
  6. 6.
    Heinze, F., Hesels, K., Breitbach-Faller, N., Schmitz-Rode, T., Disselhorst-Klug, C.: Movement analysis by accelerometry of newborns and infants for the early detection of movement disorders due to infantile cerebral palsy. Med. Biol. Eng. Comput. 48(8), 765–772 (2010)CrossRefGoogle Scholar
  7. 7.
    Hesse, N., Stachowiak, G., Breuer, T., Arens, M.: Estimating body pose of infants in depth images using random ferns. In: IEEE International Conference on Computer Vision Workshops (ICCVW) (2015)Google Scholar
  8. 8.
    Karch, D., Kim, K.S., Wochner, K., Pietz, J., Dickhaus, H., Philippi, H.: Quantification of the segmental kinematics of spontaneous infant movements. J. Biomech. 41(13), 2860–2867 (2008)CrossRefGoogle Scholar
  9. 9.
    Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: a skinned multi-person linear model. ACM Trans. Graph. 34(6), 248 (2015)CrossRefGoogle Scholar
  10. 10.
    MakeHuman: Open source tool for making 3D characters. www.makehuman.org
  11. 11.
    Marcroft, C., Khan, A., Embleton, N.D., Trenell, M., Plötz, T.: Movement recognition technology as a method of assessing spontaneous general movements in high risk infants. Front. Neurol. 5, 284 (2014)Google Scholar
  12. 12.
    Meinecke, L., Breitbach-Faller, N., Bartz, C., Damen, R., Rau, G., Disselhorst-Klug, C.: Movement analysis in the early detection of newborns at risk for developing spasticity due to infantile cerebral palsy. Hum. Mov. Sci. 25(2), 125–144 (2006)CrossRefGoogle Scholar
  13. 13.
    Olsen, M.D., Herskind, A., Nielsen, J.B., Paulsen, R.R.: Model-based motion tracking of infants. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014, Part III. LNCS, vol. 8927, pp. 673–685. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16199-0_47CrossRefGoogle Scholar
  14. 14.
    Pons-Moll, G., Pujades, S., Hu, S., Black, M.J.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. 36(4), 73 (2017)CrossRefGoogle Scholar
  15. 15.
    Prechtl, H.: Qualitative changes of spontaneous movements in fetus and preterm infant are a marker of neurological dysfunction. Early Hum. Dev. 23(3), 151–158 (1990)CrossRefGoogle Scholar
  16. 16.
    Rahmati, H., Dragon, R., Aamo, O.M., Adde, L., Stavdahl, Ø., Van Gool, L.: Weakly supervised motion segmentation with particle matching. Comput. Vis. Image Underst. 140, 30–42 (2015)CrossRefGoogle Scholar
  17. 17.
    Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  18. 18.
    Spittle, A., Orton, J., Anderson, P.J., Boyd, R., Doyle, L.W.: Early developmental intervention programmes provided post hospital discharge to prevent motor and cognitive impairment in preterm infants. The Cochrane Library (2015)Google Scholar
  19. 19.
    Stahl, A., Schellewald, C., Stavdahl, Ø., Aamo, O.M., Adde, L., Kirkerød, H.: An optical flow-based method to predict infantile cerebral palsy. IEEE Trans. Neural Syst. Rehabil. Eng. 20(4), 605–614 (2012)CrossRefGoogle Scholar
  20. 20.
    Wei, S.E., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  21. 21.
    Zuffi, S., Kanazawa, A., Jacobs, D., Black, M.J.: 3D menagerie: modeling the 3D shape and pose of animals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Nikolas Hesse
    • 1
    Email author
  • Sergi Pujades
    • 2
  • Javier Romero
    • 3
  • Michael J. Black
    • 2
  • Christoph Bodensteiner
    • 1
  • Michael Arens
    • 1
  • Ulrich G. Hofmann
    • 4
  • Uta Tacke
    • 5
  • Mijna Hadders-Algra
    • 6
  • Raphael Weinberger
    • 7
  • Wolfgang Müller-Felber
    • 7
  • A. Sebastian Schroeder
    • 7
  1. 1.Fraunhofer Institute of Optronics, System Technologies and Image ExploitationEttlingenGermany
  2. 2.Max Planck Institute for Intelligent SystemsTübingenGermany
  3. 3.AmazonBarcelonaSpain
  4. 4.University Medical Center Freiburg, Faculty of MedicineUniversity of FreiburgFreiburg im BreisgauGermany
  5. 5.University Children’s Hospital BaselBaselSwitzerland
  6. 6.University Medical Center GroningenUniversity of GroningenGroningenNetherlands
  7. 7.Hauner Children’s HospitalLudwig Maximilian UniversityMunichGermany

Personalised recommendations