Home 3D Body Scans from a Single Kinect

  • Alexander Weiss
  • David Hirshberg
  • Michael J. Black
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)

Abstract

The 3D shape of the human body is useful for applications in fitness, games, and apparel. Accurate body scanners, however, are expensive, limiting the availability of 3D body models. Although there has been a great deal of interest recently in the use of active depth sensing cameras, such as the Microsoft Kinect, for human pose tracking, little has been said about the related problem of human shape estimation. We present a method for human shape reconstruction from noisy monocular image and range data using a single inexpensive commodity sensor. The approach combines low-resolution image silhouettes with coarse range data to estimate a parametric model of the body. Accurate 3D shape estimates are obtained by combining multiple monocular views of a person moving in front of the sensor. To cope with varying body pose, we use a SCAPE body model which factors 3D body shape and pose variations. This enables the estimation of a single consistent shape, while allowing pose to vary. Additionally, we describe a novel method to minimize the distance between the projected 3D body contour and the image silhouette that uses analytic derivatives of the objective function. We use a simple method to estimate standard body measurements from the recovered SCAPE model and show that the accuracy of our method is competitive with commercial body scanning systems costing orders of magnitude more.

Keywords

Expense Balan 

Notes

Acknowledgements

We thank Loretta Reiss for her measurement expertise and Lisa Wang for mathematical discussions. This work was supported in part by NIH EUREKA award 1R01NS066311–01 and NSF award IIS–0812364.

References

  1. 1.
    Allen, B., Curless, B., Popovic, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003) CrossRefGoogle Scholar
  2. 2.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: SCAPE: shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005) CrossRefGoogle Scholar
  3. 3.
    Balan, A.: Detailed human shape and pose from images. Ph.D. thesis, Brown University (2010) Google Scholar
  4. 4.
    Balan, A., Sigal, L., Black, M., Davis, J., Haussecker, H.: Detailed human shape and pose from images. In: IEEE Conference on Computer Vision and Pattern Recognition (2007) Google Scholar
  5. 5.
    Balan, A., Black, M.: The naked truth: estimating body shape under clothing. In: European Conference on Computer Vision (2007) Google Scholar
  6. 6.
    Bouguet, J.: Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc (2007)
  7. 7.
    Charpiat, G., Faugeras, O., Keriven, R.: Approximations of shape metrics and application to shape warping and empirical shape statistics. Found. Comput. Math. 5(1), 1–58 (2005) MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Delamarre, Q., Faugeras, O.: 3D articulated models and multi-view tracking with silhouettes. In: International Conference on Computer Vision (1999) Google Scholar
  9. 9.
    Flanders, H.: Differentiation under the integral sign. Am. Math. Mon. 80(6), 615–627 (1973) MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Ganapathi, V., Plagemann, C., Koller, D., Thrun, S.: Real time motion capture using a single time-of-flight camera. In: IEEE Conference on Computer Vision and Pattern Recognition (2010) Google Scholar
  11. 11.
    Geman, S., McClure, D.: Statistical methods for tomographic image reconstruction. Bull. Int. Stat. Inst. LII(4), 5–21 (1987) MathSciNetGoogle Scholar
  12. 12.
    Grest, D., Herzog, D., Koch, R.: Human model fitting from monocular posture images. In: Vision, Modeling, and Visualization (2005) Google Scholar
  13. 13.
    Guan, P., Weiss, A., Balan, A., Black, M.: Estimating human shape and pose from a single image. In: International Conference on Computer Vision (2009) Google Scholar
  14. 14.
    Hasler, N., Rosenhahn, B., Thormählen, T., Wand, M., Gall, J., Seidel, H.P.: Markerless motion capture with unsynchronized moving cameras. In: IEEE Conference on Computer Vision and Pattern Recognition (2009) Google Scholar
  15. 15.
    Hasler, N., Stoll, C., Rosenhahn, B., Thormählen, T., Seidel, H.P.: Estimating body shape of dressed humans. Comput. Graph. 33(3), 211–216 (2009) CrossRefGoogle Scholar
  16. 16.
    Hasler, N., Ackermann, H., Rosenhahn, B., Thormählen, T., Seidel, H.P.: Multilinear pose and body shape estimation of dressed subjects from image sets. In: IEEE Conference on Computer Vision and Pattern Recognition (2010) Google Scholar
  17. 17.
    Jain, A., Thormählen, T., Seidel, H.P., Theobalt, C.: Moviereshape: tracking and reshaping of humans in videos. ACM Trans. Graph. 29(6), 148 (2010) CrossRefGoogle Scholar
  18. 18.
    Knossow, D., Ronfard, R., Horaud, R.: Human motion tracking with a kinematic parameterization of extremal contours. Int. J. Comput. Vis. 79(3), 247–269 (2008) CrossRefGoogle Scholar
  19. 19.
    Konolige, K., Mihelich, P.: ROS.org Wiki: Kinect_calibration/technical. http://www.ros.org/wiki/kinect_calibration/technical (2011)
  20. 20.
    de La Gorce, M., Paragios, N., Fleet, D.: Model-based hand tracking with texture, shading and self-occlusions. In: IEEE Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  21. 21.
    Microsoft: Kinect for X-BOX 360. http://www.xbox.com/kinect (2010)
  22. 22.
    Newcombe, R., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: International Symposium on Mixed and Augmented Reality (2011) Google Scholar
  23. 23.
    OpenKinect project. http://www.openkinect.org (2011)
  24. 24.
    Plänkers, R., Fua, P.: Model-based silhouette extraction for accurate people tracking. In: European Conference on Computer Vision (2002) Google Scholar
  25. 25.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: IEEE Conference on Computer Vision and Pattern Recognition (2011) Google Scholar
  26. 26.
    Sminchisescu, C., Telea, A.: Human pose estimation from silhouettes. a consistent approach using distance level sets. In: WSCG International Conference on Computer Graphics, Visualization and Computer Vision (2002) Google Scholar
  27. 27.
    Sumner, R., Popovic, J.: Deformation transfer for triangle meshes. ACM Trans. Graph. 23(3), 399–405 (2004) CrossRefGoogle Scholar
  28. 28.
    Tong, J., Zhou, J., Liu, L., Pan, Z., Yan, H.: Scanning 3D full human bodies using Kinects. IEEE Trans. Vis. Comput. Graph. 18(4), 643–650 (2012) CrossRefGoogle Scholar
  29. 29.
    Zhou, S., Fu, H., Liu, L., Cohen-Or, D., Han, X.: Parametric reshaping of human bodies in images. ACM Trans. Graph. 29(4), 126 (2010) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Alexander Weiss
    • 1
  • David Hirshberg
    • 2
  • Michael J. Black
    • 2
  1. 1.Brown UniversityProvidenceUSA
  2. 2.Max Planck Inst. for Intelligent SystemsTübingenGermany

Personalised recommendations