GAN-Based Garment Generation Using Sewing Pattern Images

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12363)


The generation of realistic apparel model has become increasingly popular as a result of the rapid pace of change in fashion trends and the growing need for garment models in various applications such as virtual try-on. For such application requirements, it is important to have a general cloth model that can represent a diverse set of garments. Previous studies often make certain assumptions about the garment, such as the topology or suited body shape. We propose a unified method using the generative network. Our model is applicable to different garment topologies with different sewing patterns and fabric materials. We also develop a novel image representation of garment models, and a reliable mapping algorithm between the general garment model and the image representation that can regularize the data representation of the cloth. Using this special intermediate image representation, the generated garment model can be easily retargeted to another body, enabling garment customization. In addition, a large garment appearance dataset is provided for use in garment reconstruction, garment capturing, and other applications. We demonstrate that our generative model has high reconstruction accuracy and can provide rich variations of virtual garments.



This work is supported in part by Elizabeth Stevinson Iribe Professorship and National Science Foundation.

Supplementary material


  1. 1.
    Alldieck, T., Pons-Moll, G., Theobalt, C., Magnor, M.: Tex2shape: detailed full human body geometry from a single image, pp. 2293–2303 (2019)Google Scholar
  2. 2.
    Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid ICP algorithms for surface registration. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)Google Scholar
  3. 3.
    Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., Taubin, G.: The ball-pivoting algorithm for surface reconstruction. IEEE Trans. Vis. Comput. Graph. 5(4), 349–359 (1999)CrossRefGoogle Scholar
  4. 4.
    Bhatnagar, B.L., Tiwari, G., Theobalt, C., Pons-Moll, G.: Multi-garment net: learning to dress 3D people from images. In: IEEE International Conference on Computer Vision (ICCV). IEEE, October 2019Google Scholar
  5. 5.
    Bradley, D., Popa, T., Sheffer, A., Heidrich, W., Boubekeur, T.: Markerless garment capture. ACM Trans. Graph. 27(3), 99 (2008).
  6. 6.
    Brouet, R., Sheffer, A., Boissieux, L., Cani, M.: Design preserving garment transfer. ACM Trans. Graph. 31(4), 36:1–36:11 (2012).
  7. 7.
    Chen, X., Zhou, B., Lu, F., Wang, L., Bi, L., Tan, P.: Garment modeling with a depth camera. ACM Trans. Graph. 34(6), 203:1–203:12 (2015).
  8. 8.
    Danerek, R., Dibra, E., Öztireli, A.C., Ziegler, R., Gross, M.H.: Deepgarment: 3D garment shape estimation from a single image. Comput. Graph. Forum 36(2), 269–280 (2017).
  9. 9.
    Decaudin, P., Julius, D., Wither, J., Boissieux, L., Sheffer, A., Cani, M.: Virtual garments: a fully geometric approach for clothing design. Comput. Graph. Forum 25(3), 625–634 (2006).
  10. 10.
    Doersch, C.: Tutorial on variational autoencoders. CoRR abs/1606.05908 (2016).
  11. 11.
    Gabeur, V., Franco, J.S., Martin, X., Schmid, C., Rogez, G.: Moulding humans: non-parametric 3D human shape estimation from single images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2232–2241 (2019)Google Scholar
  12. 12.
    Goodfellow, I.J., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8–13 2014, Montreal, Quebec, Canada, pp. 2672–2680 (2014).
  13. 13.
    Guan, P., Reiss, L., Hirshberg, D.A., Weiss, A., Black, M.J.: DRAPE: dressing any person. ACM Trans. Graph. 31(4), 35:1–35:10 (2012).
  14. 14.
    Gundogdu, E., Constantin, V., Seifoddini, A., Dang, M., Salzmann, M., Fua, P.: Garnet: a two-stream network for fast and accurate 3d cloth draping. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8739–8748 (2019)Google Scholar
  15. 15.
    Huang, P., Yao, J., Zhao, H.: Automatic realistic 3D garment generation based on two images. In: 2016 International Conference on Virtual Reality and Visualization (ICVRV) (2016)Google Scholar
  16. 16.
    Jeong, M., Han, D., Ko, H.: Garment capture from a photograph. J. Vis. Comput. Animation 26(3–4), 291–300 (2015).
  17. 17.
    Jung, A., Hahmann, S., Rohmer, D., Bégault, A., Boissieux, L., Cani, M.: Sketching folds: developable surfaces from non-planar silhouettes. ACM Trans. Graph. 34(5), 155:1–155:12 (2015).
  18. 18.
    Lähner, Z., Cremers, D., Tung, T.: DeepWrinkles: accurate and realistic clothing modeling. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 698–715. Springer, Cham (2018). Scholar
  19. 19.
    Liang, J., Lin, M.C.: Shape-aware human pose and shape reconstruction using multi-view images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4352–4362 (2019)Google Scholar
  20. 20.
    Lin, M.C.: Efficient collision detection for animation and robotics. Ph.D. thesis, Department of Electrical Engineering and Computer Science (1993)Google Scholar
  21. 21.
    Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: a skinned multi-person linear model. ACM Trans. Graph. 34(6), 248:1–248:16 (2015).
  22. 22.
    Loper, M.M., Mahmood, N., Black, M.J.: MoSh: motion and shape capture from sparse markers. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 33(6), 220:1–220:13 (2014).
  23. 23.
    Narain, R., Samii, A., O’Brien, J.F.: Adaptive anisotropic remeshing for cloth simulation. ACM Trans. Graph. 31(6), 152:1–152:10 (2012).
  24. 24.
    Patel, C., Liao, Z., Pons-Moll, G.: Tailornet: predicting clothing in 3D as a function of human pose, shape and garment style. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7365–7375 (2020)Google Scholar
  25. 25.
    Razavi, A., van den Oord, A., Vinyals, O.: Generating diverse high-fidelity images with VQ-VAE-2. In: Advances in Neural Information Processing Systems, pp. 14866–14876 (2019)Google Scholar
  26. 26.
    Robson, C., Maharik, R., Sheffer, A., Carr, N.: Context-aware garment modeling from sketches. Comput. Graph. 35(3), 604–613 (2011).
  27. 27.
    Saito, S., Huang, Z., Natsume, R., Morishima, S., Kanazawa, A., Li, H.: Pifu: pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2304–2314 (2019)Google Scholar
  28. 28.
    Turquin, E., Cani, M., Hughes, J.F.: Sketching garments for virtual characters. In: 34. International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2007, San Diego, California, USA, August 5–9, 2007, Courses, p. 28 (2007).
  29. 29.
    Wang, T., Liu, M., Zhu, J., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18–22, 2018, pp. 8798–8807 (2018).
  30. 30.
    Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  31. 31.
    Wang, T.Y., Ceylan, D., Popovic, J., Mitra, N.J.: Learning a shared shape space for multimodal garment design. CoRR abs/1806.11335 (2018).
  32. 32.
    Yang, S., et al.: Physics-inspired garment recovery from a single-view image. ACM Trans. Graph. (TOG) 37(5), 170 (2018)CrossRefGoogle Scholar
  33. 33.
    Zheng, Z., Yu, T., Wei, Y., Dai, Q., Liu, Y.: Deephuman: 3D human reconstruction from a single image. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7739–7749 (2019)Google Scholar
  34. 34.
    Zhou, B., Chen, X., Fu, Q., Guo, K., Tan, P.: Garment modeling from a single image. Comput. Graph. Forum 32(7), 85–91 (2013).

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of MarylandCollege ParkUSA

Personalised recommendations