Skip to main content
SpringerLink
Log in

BCNet: Learning Body and Cloth Shape from a Single Image

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12365))

Included in the following conference series:

Abstract

In this paper, we consider the problem to automatically reconstruct garment and body shapes from a single near-front view RGB image. To this end, we propose a layered garment representation on top of SMPL and novelly make the skinning weight of garment independent of the body mesh, which significantly improves the expression ability of our garment model. Compared with existing methods, our method can support more garment categories and recover more accurate geometry. To train our model, we construct two large scale datasets with ground truth body and garment geometries as well as paired color images. Compared with single mesh or non-parametric representation, our method can achieve more flexible control with separate meshes, makes applications like re-pose, garment transfer, and garment texture mapping possible.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We did not test  [7, 38] on this dataset as most of the samples are not A-pose.

References

  1. Alldieck, T., Magnor, M., Bhatnagar, B.L., Theobalt, C., Pons-Moll, G.: Learning to reconstruct people in clothing from a single RGB camera. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1175–1186 (2019)

    Google Scholar 

  2. Alldieck, T., Magnor, M., Xu, W., Theobalt, C., Pons-Moll, G.: Detailed human avatars from monocular video. In: International Conference on 3D Vision (3DV), pp. 98–109. IEEE (2018)

    Google Scholar 

  3. Alldieck, T., Magnor, M., Xu, W., Theobalt, C., Pons-Moll, G.: Video based reconstruction of 3d people models. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8387–8397 (2018)

    Google Scholar 

  4. Alldieck, T., Pons-Moll, G., Theobalt, C., Magnor, M.: Tex2shape: detailed full human body geometry from a single image. In: IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  5. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. In: ACM Transactions on Graphics (TOG), vol. 24, pp. 408–416. ACM (2005)

    Google Scholar 

  6. axyz (2019). https://secure.axyz-design.com/

  7. Bhatnagar, B.L., Tiwari, G., Theobalt, C., Pons-Moll, G.: Multi-garment net: learning to dress 3D people from images. In: IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  8. Blender (2019). https://www.blender.org/

  9. Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., Black, M.J.: Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 561–578. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_34

    Chapter  Google Scholar 

  10. Bouritsas, G., Bokhnyak, S., Ploumpis, S., Bronstein, M., Zafeiriou, S.: Neural 3D morphable models: spiral convolutional networks for 3D shape representation learning and generation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7213–7222 (2019)

    Google Scholar 

  11. Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing textures in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3606–3613 (2014)

    Google Scholar 

  12. Daněřek, R., Dibra, E., Öztireli, C., Ziegler, R., Gross, M.: Deepgarment: 3D garment shape estimation from a single image. In: Computer Graphics Forum, vol. 36, pp. 269–280. Wiley Online Library (2017)

    Google Scholar 

  13. De Aguiar, E., Sigal, L., Treuille, A., Hodgins, J.K.: Stable spaces for real-time clothing. ACM Trans. Graph. (TOG) 29(4), 1–9 (2010)

    Article  Google Scholar 

  14. Dibra, E., Jain, H., Oztireli, C., Ziegler, R., Gross, M.: Human shape from silhouettes using generative HKS descriptors and cross-modal neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4826–4836 (2017)

    Google Scholar 

  15. Flex, N.: 2019. https://developer.nvidia.com/flex/

  16. Gong, K., Liang, X., Li, Y., Chen, Y., Yang, M., Lin, L.: Instance-level human parsing via part grouping network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 770–785 (2018)

    Google Scholar 

  17. Guan, P., Reiss, L., Hirshberg, D.A., Weiss, A., Black, M.J.: Drape: dressing any person. ACM Trans. Graph. (TOG) 31(4), 1–35 (2012)

    Article  Google Scholar 

  18. Gundogdu, E., Constantin, V., Seifoddini, A., Dang, M., Salzmann, M., Fua, P.: Garnet: A two-stream network for fast and accurate 3D cloth draping. In: IEEE International Conference on Computer Vision (ICCV), pp. 8739–8748 (2019)

    Google Scholar 

  19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

    Google Scholar 

  20. Jackson, A.S., Manafas, C., Tzimiropoulos, G.: 3D human body reconstruction from a single image via volumetric regression. In: European Conference on Computer Vision (ECCV), (2018)

    Google Scholar 

  21. Jiang, B., Zhang, J., Cai, J., Zheng, J.: Learning 3D human body embedding. arXiv preprint arXiv:1905.05622 (2019)

  22. Kampouris, C., Zafeiriou, S., Ghosh, A., Malassiotis, S.: Fine-grained material classification using micro-geometry and reflectance. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 778–792. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_47

    Chapter  Google Scholar 

  23. Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: End-to-end recovery of human shape and pose. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7122–7131 (2018)

    Google Scholar 

  24. Kanazawa, A., Zhang, J.Y., Felsen, P., Malik, J.: Learning 3D human dynamics from video. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5614–5623 (2019)

    Google Scholar 

  25. Kato, H., Ushiku, Y., Harada, T.: Neural 3d mesh renderer. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3907–3916 (2018)

    Google Scholar 

  26. Lahner, Z., Cremers, D., Tung, T.: Deepwrinkles: accurate and realistic clothing modeling. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 667–684 (2018)

    Google Scholar 

  27. Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M.J., Gehler, P.V.: Unite the people: Closing the loop between 3D and 2D human representations. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6050–6059 (2017)

    Google Scholar 

  28. Liang, J., Lin, M.C.: Shape-aware human pose and shape reconstruction using multi-view images. In: The IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  29. Liu, J., Akhtar, N., Mian, A.: Temporally coherent full 3D mesh human pose recovery from monocular video. arXiv preprint arXiv:1906.00161 (2019)

  30. Liu, L., Zheng, Y., Tang, D., Yuan, Y., Fan, C., Zhou, K.: Neuroskinning: automatic skin binding for production characters with deep graph networks. ACM Trans. Graph. (TOG) 38(4), 114 (2019)

    Google Scholar 

  31. Loper, M., Mahmood, N., Black, M.J.: Mosh: motion and shape capture from sparse markers. ACM Trans. Graph. (TOG) 33(6), 220 (2014)

    Article  Google Scholar 

  32. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: Smpl: a skinned multi-person linear model. ACM Trans. Graph. (TOG) 34(6), 248 (2015)

    Article  Google Scholar 

  33. Mixamo (2019). https://www.mixamo.com/

  34. Mocap, C.: 2019. http://mocap.cs.cmu.edu/

  35. Natsume, R., Saito, S., Huang, Z., Chen, W., Ma, C., Li, H., Morishima, S.: Siclope: silhouette-based clothed people. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4480–4490 (2019)

    Google Scholar 

  36. Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., Schiele, B.: Neural body fitting: unifying deep learning and model based human pose and shape estimation. In: International Conference on 3D Vision (3DV), pp. 484–494. IEEE (2018)

    Google Scholar 

  37. Pavlakos, G., et al.: Expressive body capture: 3D hands, face, and body from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10975–10985 (2019)

    Google Scholar 

  38. Pavlakos, G., Zhu, L., Zhou, X., Daniilidis, K.: Learning to estimate 3D human pose and shape from a single color image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 459–468 (2018)

    Google Scholar 

  39. Pons-Moll, G., Pujades, S., Hu, S., Black, M.J.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. (TOG) 36(4), 1–15 (2017)

    Article  Google Scholar 

  40. Pumarola, A., Sanchez, J., Choi, G., Sanfeliu, A., Moreno-Noguer, F.: 3D people: modeling the geometry of dressed humans. In: IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  41. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 652–660 (2017)

    Google Scholar 

  42. Renderpeople (2019). https://renderpeople.com/3d-people

  43. Saito, S., Huang, Z., Natsume, R., Morishima, S., Kanazawa, A., Li, H.: Pifu: pixel-aligned implicit function for high-resolution clothed human digitization. In: IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  44. Santesteban, I., Otaduy, M.A., Casas, D.: Learning-based animation of clothing for virtual try-on. In: Computer Graphics Forum, vol. 38, pp. 355–366. Wiley Online Library (2019)

    Google Scholar 

  45. Sorkine, O.: Laplacian mesh processing. In: Eurographics - State of the Art Reports, pp. 53–70 (2005)

    Google Scholar 

  46. Varol, G., et al.: Bodynet: volumetric inference of 3D human body shapes. In: European Conference on Computer Vision (ECCV), pp. 20–36 (2018)

    Google Scholar 

  47. Varol, G., et al.: Learning from synthetic humans. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 109–117 (2017)

    Google Scholar 

  48. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations, ICLR (2018)

    Google Scholar 

  49. Wang, T.Y., Ceylan, D., Popovic, J., Mitra, N.J.: Learning a shared shape space for multimodal garment design. ACM Trans. Graph. (TOG) 37(6), 203:1–203:13 (2018)

    Google Scholar 

  50. Xu, W., Umentani, N., Chao, Q., Mao, J., Jin, X., Tong, X.: Sensitivity-optimized rigging for example-based real-time clothing synthesis. ACM Trans. Graph. (TOG) 33(4), 107 (2014)

    Google Scholar 

  51. Yang, J., Franco, J.S., Hétroy-Wheeler, F., Wuhrer, S.: Analyzing clothing layer deformation statistics of 3D human motions. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 237–253 (2018)

    Google Scholar 

  52. Yang, S., et al.: Physics-inspired garment recovery from a single-view image. ACM Trans. Graph. (TOG) 37(5), 170 (2018)

    Article  Google Scholar 

  53. Yang, Y., Yu, Y., Zhou, Y., Du, S., Davis, J., Yang, R.: Semantic parametric reshaping of human body models. In: International Conference on 3D Vision (3DV), vol. 2, pp. 41–48. IEEE (2014)

    Google Scholar 

  54. Yao, P., Fang, Z., Wu, F., Feng, Y., Li, J.: Densebody: directly regressing dense 3D human pose and shape from a single color image. arXiv preprint arXiv:1903.10153 (2019)

  55. Yu, T., et al.: Simulcap: Single-view human performance capture with cloth simulation. arXiv preprint arXiv:1903.06323 (2019)

  56. Zhang, C., Pujades, S., Black, M.J., Pons-Moll, G.: Detailed, accurate, human shape estimation from clothed 3D scan sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4191–4200 (2017)

    Google Scholar 

  57. Zheng, Z., Yu, T., Wei, Y., Dai, Q., Liu, Y.: Deephuman: 3D human reconstruction from a single image. In: IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  58. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2017)

    Article  Google Scholar 

  59. Zhu, H., Zuo, X., Wang, S., Cao, X., Yang, R.: Detailed human shape estimation from a single image by hierarchical mesh deformation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4491–4500 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Science and Technology of China, Hefei, China

    Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo & Ligang Liu

  2. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China

    Hujun Bao

Authors
  1. Boyi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juyong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinhao Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ligang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hujun Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juyong Zhang .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 2 (mp4 9115 KB)

Supplementary material 1 (pdf 23793 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jiang, B., Zhang, J., Hong, Y., Luo, J., Liu, L., Bao, H. (2020). BCNet: Learning Body and Cloth Shape from a Single Image. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12365. Springer, Cham. https://doi.org/10.1007/978-3-030-58565-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58565-5_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58564-8

  • Online ISBN: 978-3-030-58565-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation