DARWIN: Deformable Patient Avatar Representation With Deep Image Network

  • Vivek SinghEmail author
  • Kai Ma
  • Birgi Tamersoy
  • Yao-Jen Chang
  • Andreas Wimmer
  • Thomas O’Donnell
  • Terrence Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)


In this paper, we present a technical approach to robustly estimate the detailed patient body surface mesh under clothing cover from a single snapshot of a range sensor. Existing methods either lack level of detail of the estimated patient body model, fail to estimate the body model robustly under clothing cover, or lack sufficient evaluation over real patient datasets. In this work, we overcome these limitations by learning deep convolutional networks over real clinical dataset with large variation and augmentation. Our approach is validated with experiments conducted over 1063 human subjects from 3 different hospitals and surface errors are measured against groundtruth from CT data.


  1. 1.
  2. 2.
    Achilles, F., Ichim, A.-E., Coskun, H., Tombari, F., Noachtar, S., Navab, N.: Patient MoCap: human pose estimation under blanket occlusion for hospital monitoring applications. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 491–499. Springer, Cham (2016). doi: 10.1007/978-3-319-46720-7_57CrossRefGoogle Scholar
  3. 3.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: SCAPE: shape completion and animation of people. ACM Trans. Graph. 24, 408–416 (2005)CrossRefGoogle Scholar
  4. 4.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation (2015). arXiv:1511.00561
  5. 5.
    Bauer, S., Wasza, J., Haase, S., Marosi, N., Hornegger, J.: Multi-modal surface registration for markerless initial patient setup in radiation therapy using microsoft’s Kinect sensor. In: ICCV Workshops (2011)Google Scholar
  6. 6.
    Bauer, S., et al.: Real-time range imaging in health care: a survey. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. LNCS, vol. 8200, pp. 228–254. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-44964-2_11CrossRefGoogle Scholar
  7. 7.
    Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., Black, M.J.: Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 561–578. Springer, Cham (2016). doi: 10.1007/978-3-319-46454-1_34CrossRefGoogle Scholar
  8. 8.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Ghesu, F., Georgescu, B., Grbic, S., Maier, A., Hornegger, J., Comaniciu, D.: Robust multiscale anatomical landmark detection in incomplete 3D-CT data. In: MICCAI (2017)Google Scholar
  10. 10.
    Grimm, T., Martinez, M., Benz, A., Stiefelhagen, R.: Sleep position classification from a depth camera using bed aligned maps. In: ICPR (2016)Google Scholar
  11. 11.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)Google Scholar
  12. 12.
    Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., Burnsides, D.: Civilian American and European surface anthropometry resource (CAESAR) final report. AFRL-HE-WP-TR-2002-0169 (2002)Google Scholar
  13. 13.
    Sathyanarayana, S., Satzoda, R.K., Sathyanarayana, S., Thambipillai, S.: Vision-based patient monitoring: a comprehensive review of algorithms and technologies. J. Ambient Intell. Humanized Comput. 1–27 (2015).
  14. 14.
    Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. T-PAMI 35(12), 2821–2840 (2013)CrossRefGoogle Scholar
  15. 15.
    Singh, V., Chang, Y., Ma, K., Wels, M., Soza, G., Chen, T.: Estimating a patient surface model for optimizing the medical scanning workflow. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 472–479. Springer, Cham (2014). doi: 10.1007/978-3-319-10404-1_59CrossRefGoogle Scholar
  16. 16.
    Tompson, J., Goroshin, R., Jain, A., LeCun, Y., Bregler, C.: Efficient object localization using convolutional networks. In: CVPR (2015)Google Scholar
  17. 17.
    Weiss, A., Hirshberg, D., Black, M.J.: Home 3D body scans from noisy image and range data. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds.) Consumer Depth Cameras for Computer Vision: Research Topics and Applications, pp. 99–118. Springer, London (2012). doi: 10.1007/978-1-4471-4640-7_6CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Vivek Singh
    • 1
    Email author
  • Kai Ma
    • 1
  • Birgi Tamersoy
    • 2
  • Yao-Jen Chang
    • 1
  • Andreas Wimmer
    • 2
  • Thomas O’Donnell
    • 1
  • Terrence Chen
    • 1
  1. 1.Medical Imaging TechnologiesSiemens Medical Solutions USA Inc.PrincetonUSA
  2. 2.Siemens Healthcare GmbHForchheimGermany

Personalised recommendations