Advertisement

Analyzing Clothing Layer Deformation Statistics of 3D Human Motions

  • Jinlong YangEmail author
  • Jean-Sébastien Franco
  • Franck Hétroy-Wheeler
  • Stefanie Wuhrer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11211)

Abstract

Recent capture technologies and methods allow not only to retrieve 3D model sequence of moving people in clothing, but also to separate and extract the underlying body geometry, motion component and the clothing as a geometric layer. So far this clothing layer has only been used as raw offsets for individual applications such as retargeting a different body capture sequence with the clothing layer of another sequence, with limited scope, e.g. using identical or similar motions. The structured, semantics and motion-correlated nature of the information contained in this layer has yet to be fully understood and exploited. To this purpose we propose a comprehensive analysis of the statistics of this layer with a simple two-component model, based on PCA subspace reduction of the layer information on one hand, and a generic parameter regression model using neural networks on the other hand, designed to regress from any semantic parameter whose variation is observed in a training set, to the layer parameterization space. We show that this model not only allows to reproduce previous retargeting works, but generalizes the data generation capabilities to other semantic parameters such as clothing variation and size, or physical material parameters with synthetically generated training sequence, paving the way for many kinds of capture data-driven creation and augmentation applications.

Keywords

Clothing motion analysis 3D garment capture Capture data augmentation and retargeting 

Notes

Acknowledgement

Funded by France National Research grant ANR-14-CE24-0030 ACHMOV. We thank G. Pons-Moll for providing ClothCap [25] data, Kinovis platform(https://kinovis.inria.fr/) for acquiring the extended Inria dataset, and F. Bertails-Descoubes and A. H. Rasheed for providing the cloth simulator and generating synthetic data.

References

  1. 1.
    Opennn library. http://www.opennn.net/
  2. 2.
    de Aguiar, E., Sigal, L., Treuille, A., Hodgins, J.K.: Stable spaces for real-time clothing. ACM Trans. Graph. 29(4), 1–9 (2010)CrossRefGoogle Scholar
  3. 3.
    de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Trans. Graph. 27(3), 98 (2008)CrossRefGoogle Scholar
  4. 4.
    Allain, B., Franco, J.S., Boyer, E.: An efficient volumetric framework for shape tracking. In: Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  5. 5.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: SCAPE: shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005)CrossRefGoogle Scholar
  6. 6.
    Bălan, A., Black, M.: The naked truth: estimating body shape under clothing. In: European Conference on Computer Vision (2008)Google Scholar
  7. 7.
    Baraff, D., Witkin, A.: Large steps in cloth simulation. In: Conference on Computer Graphics and Interactive Techniques (1998)Google Scholar
  8. 8.
    Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., Black, M.J.: Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: European Conference on Computer Vision (2016)Google Scholar
  9. 9.
    Bridson, R., Marino, S., Fedkiw, R.: Simulation of clothing with folds and wrinkles. In: Symposium on Computer Animation (2003)Google Scholar
  10. 10.
    Collet, A., et al.: High-quality streamable free-viewpoint video. ACM Trans. Graph. 34(4), 69 (2015)CrossRefGoogle Scholar
  11. 11.
    Dibra, E., Jain, H., Öztireli, A.C., Ziegler, R., Gross, M.H.: HS-Nets: estimating human body shape from silhouettes with convolutional neural networks. In: International Conference on 3D Vision (2016)Google Scholar
  12. 12.
    Dibra, E., Jain, H., Öztireli, A.C., Ziegler, R., Gross, M.H.: Human shape from silhouettes using generative HKS descriptors and cross-modal neural networks. In: Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  13. 13.
    Dibra, E., Öztireli, A.C., Ziegler, R., Gross, M.H.: Shape from selfies: human body shape estimation using CCA regression forests. In: European Conference of Computer Vision (2016)Google Scholar
  14. 14.
    Gall, J., Stoll, C., De Aguiar, E., Theobalt, C., Rosenhahn, B., Seidel, H.P.: Motion capture using joint skeleton tracking and surface estimation. In: Conference on Computer Vision and Pattern Recognition (2009)Google Scholar
  15. 15.
    Guan, P., Reiss, L., Hirshberg, D.A., Weiss, A., Black, M.J.: Drape: Dressing any person. ACM Trans. Graph. 31(4), 35-1 (2012)CrossRefGoogle Scholar
  16. 16.
    Hasler, N., Stoll, C., Rosenhahn, B., Thormählen, T., Seidel, H.P.: Estimating body shape of dressed humans. Comput. Graph. 33(3), 211–216 (2009)CrossRefGoogle Scholar
  17. 17.
    Kaldor, J.M., James, D.L., Marschner, S.: Efficient yarn-based cloth with adaptive contact linearization. ACM Trans. Graph. 29(4), 105 (2010)CrossRefGoogle Scholar
  18. 18.
    Li, H., Adams, B., Guibas, L., Pauly, M.: Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28(5), 175 (2009)CrossRefGoogle Scholar
  19. 19.
    Li, J., et al.: An implicit frictional contact solver for adaptive cloth simulation. ACM Trans. Graph. 37(4), 52 (2018)Google Scholar
  20. 20.
    Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.: SMPL: a skinned multi-person linear model. ACM Trans. Graph. 34(6), 248 (2015)CrossRefGoogle Scholar
  21. 21.
    Neophytou, A., Hilton, A.: Shape and pose space deformation for subject specific animation. In: International Conference on 3D Vision (2013)Google Scholar
  22. 22.
    Neophytou, A., Hilton, A.: A layered model of human body and garment deformation. In: International Conference on 3D Vision (2014)Google Scholar
  23. 23.
    Newcombe, R.A., Fox, D., Seitz, S.M.: DynamicFusion: reconstruction and tracking of non-rigid scenes in real-time. In: Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  24. 24.
    Pishchulin, L., Wuhrer, S., Helten, T., Theobalt, C., Schiele, B.: Building statistical shape spaces for 3D human modeling. Pattern Recogn. 67, 276–286 (2017)CrossRefGoogle Scholar
  25. 25.
    Pons-Moll, G., Pujades, S., Hu, S., Black, M.: ClothCap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. 36(4), 73 (2017)CrossRefGoogle Scholar
  26. 26.
    Rhodin, H., Robertini, N., Casas, D., Richardt, C., Seidel, H.P., Theobalt, C.: General automatic human shape and motion capture using volumetric contour cues. In: European Conference on Computer Vision (2016)Google Scholar
  27. 27.
    Shehu, A., Yang, J., Franco, J.S., Hétroy-Wheeler, F., Wuhrer, S.: Computing temporal alignments of human motion sequences in wide clothing using geodesic patches. In: International Conference on 3D Vision (2016)Google Scholar
  28. 28.
    Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3d human figures using 2d image motion. In: European Conference on Computer Vision (2000)Google Scholar
  29. 29.
    Sigal, L., Balan, A.O., Black, M.J.: HumanEva: synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87(4), 4 (2010)CrossRefGoogle Scholar
  30. 30.
    Sigal, L.: A perceptual control space for garment simulation. ACM Trans. Graph. 34(4), 117 (2015)CrossRefGoogle Scholar
  31. 31.
    Sminchisescu, C., Triggs, B.: Estimating articulated human motion with covariance scaled sampling. Int. J. Robot. Res. 22(6), 371–391 (2003)CrossRefGoogle Scholar
  32. 32.
    Stoll, C., Gall, J., De Aguiar, E., Thrun, S., Theobalt, C.: Video-based reconstruction of animatable human characters. ACM Trans. Graph. 29(6), 139 (2010)CrossRefGoogle Scholar
  33. 33.
    Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27(3), 97 (2008)CrossRefGoogle Scholar
  34. 34.
    Volino, P., Magnenat-Thalmann, N., Faure, F.: A simple approach to nonlinear tensile stiffness for accurate cloth simulation. ACM Trans. Graph. 28(4) (2009)CrossRefGoogle Scholar
  35. 35.
    Wang, H., O’Brien, J.F., Ramamoorthi, R.: Data-driven elastic models for cloth: modeling and measurement. ACM Trans. Graph. 30(4), 71 (2011)Google Scholar
  36. 36.
    Wuhrer, S., Pishchulin, L., Brunton, A., Shu, C., Lang, J.: Estimation of human body shape and posture under clothing. Comput. Vis. Image Underst. 127, 31–42 (2014)CrossRefGoogle Scholar
  37. 37.
    Xu, W., Umentani, N., Chao, Q., Mao, J., Jin, X., Tong, X.: Sensitivity-optimized rigging for example-based real-time clothing synthesis. ACM Trans. Graph. 33(4), 107 (2014)zbMATHGoogle Scholar
  38. 38.
    Yang, J., Franco, J.S., Hétroy-Wheeler, F., Wuhrer, S.: Estimation of human body shape in motion with wide clothing. In: European Conference on Computer Vision (2016)Google Scholar
  39. 39.
    Zaharescu, A., Boyer, E., Varanasi, K., Horaud, R.: Surface feature detection and description with applications to mesh matching. In: Conference on Computer Vision and Pattern Recognition (2009)Google Scholar
  40. 40.
    Zhang, C., Pujades, S., Black, M., Pons-Moll, G.: Detailed, accurate, human shape estimation from clothed 3d scan sequences. In: Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  41. 41.
    Zuffi, S., Black, M.J.: The stitched puppet: a graphical model of 3D human shape and pose. In: Conference on Computer Vision and Pattern Recognition (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Jinlong Yang
    • 1
    Email author
  • Jean-Sébastien Franco
    • 1
  • Franck Hétroy-Wheeler
    • 2
  • Stefanie Wuhrer
    • 1
  1. 1.Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP(Institute of Engineering Univ. Grenoble Alpes), LJKGrenobleFrance
  2. 2.ICube, University of StrasbourgStrasbourgFrance

Personalised recommendations