Skip to main content
SpringerLink
Log in

An improved generative adversarial network for translating clothes from the human body to tiled image

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Nowadays, purchasing the products similar to the styles of stars’ products has become a new trend in e-commerce platforms. Clothing transaction constitutes to the major part of these kinds of online purchasing. Traditional methods firstly segment clothes from human body and then input the segmented clothing image patch into a retrieval system as a query in a way that similar clothing items could be retrieved. However, the segmented clothing images usually contain complex backgrounds, and these clothing items appear to be twisted, as they are segmented from human body straightforwardly. In order to assist this cross-scenario clothing retrieval, this paper introduces a new triple-supervised GAN (TripleGAN) model by translating the clothes on human body into tiled clothes. Our model was trained on a large-scale dataset including over 30,000 clothing pairs constructed by ourselves. Extensive experimental results exhibit that our model consistently can generate tiled clothing images with more delicate details and higher quality compared with other models. Our model also shows promising performance in terms of cross-domain clothing retrieval in real-life applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://www.tmall.com/.

  2. https://www.net-a-porter.com/gb/en/.

  3. https://www.mytheresa.com/en-gb/.

References

  1. Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789–8797

  2. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In Advances in neural information processing systems, pp. 2672–2680

  3. Huang J, Feris RS, Chen Q, Yan S (2015) Cross-domain image retrieval with a dual attribute-aware ranking network. In Proceedings of the IEEE international conference on computer vision, pp. 1062–1070

  4. Huang X, Liu MY, Belongie S, Kautz J (2018) Multimodal unsupervised image-to-image translation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 172–189

  5. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134

  6. Ji W, Wang D, Hoi SCH, Wu P, Li J (2014) Deep learning for content-based image retrieval: A comprehensive study. In Proceedings of the ACM International Conference on Multimedia

  7. Ji Y, Zhang H, Wu QJ (2018) Saliency detection via conditional adversarial image-to-image network. Neurocomputing 316:357–368

    Article  Google Scholar 

  8. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105

  9. Lin K, Yang HF, Hsiao JH, Chen CS (2015) Deep learning of binary hash codes for fast image retrieval. In 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

  10. Liu L, Zhang H, Ji Y, Wu QJ (2019) Toward ai fashion design: an attribute-gan model for clothing match. Neurocomputing 341:156–167

    Article  Google Scholar 

  11. Liu L, Zhang H, Xu X, Zhang Z, Yan S (2019) Collocating clothes with generative adversarial networks cosupervised by categories and attributes: a multidiscriminator framework. IEEE transactions on neural networks and learning systems 99:1–5

    Article  Google Scholar 

  12. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In European conference on computer vision, pp. 21–37. Springer

  13. Liu Z, Luo P, Qiu S, Wang X, Tang X (2016) Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1096–1104

  14. Lu Y, Wu S, Tai YW, Tang CK (2018) Image generation from sketch constraint using contextual gan. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 205–220

  15. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  16. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  17. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263–7271

  18. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  19. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Springer

  20. Veit A, Kovacs B, Bell S, McAuley J, Bala K, Belongie S (2015) Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4642–4650

  21. Wang TC, Liu MY, Zhu JY, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8798–8807

  22. Wang X, Shi Y, Kitani KM (2016) Deep supervised hashing with triplet labels. In Asian conference on computer vision, pp. 70–84. Springer

  23. Yoo D, Kim N, Park S, Paek AS, Kweon IS (2016) Pixel-level domain transfer. In European Conference on Computer Vision, pp. 517–532. Springer

  24. Zhang H, Sun Y, Liu L, Wang X, Li L, Liu W (2018) Clothingout: a category-supervised gan model for clothing segmentation and retrieval. Neural computing and applications pp. 1–12

  25. Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X, Metaxas DN (2017) Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 5907–5915

  26. Zhang Y (2018) Xogan: One-to-many unsupervised image-to-image translation. arXiv preprint arXiv:1805.07277

  27. Zhu S, Urtasun R, Fidler S, Lin D, Change Loy C (2017) Be your own prada: Fashion synthesis with structural coherence. In Proceedings of the IEEE international conference on computer vision, pp. 1680–1688

Download references

Acknowledgements

This work was supported in part by the National Key R&D Program of China under Grant no. 2018YFB1003800, 2018YFB1003805, the National Natural Science Foundation of China under Grant no. 61972112 and no. 61832004, and the Shenzhen Science and Technology Program under Grant no. JCYJ20170413105929681 and no. JCYJ20170811161545863.

Author information

Authors and Affiliations

  1. Huangshan University, Huangshan, China

    Xiaoli Zhang

  2. Baidu.com, Inc, Beijing, China

    Yanfang Sun

  3. Harbin Institute of Technology, Shenzhen, China

    Linlin Liu

Authors
  1. Xiaoli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanfang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Linlin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linlin Liu.

Ethics declarations

Conflict of interest statement

No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Sun, Y. & Liu, L. An improved generative adversarial network for translating clothes from the human body to tiled image. Neural Comput & Applic 33, 8445–8457 (2021). https://doi.org/10.1007/s00521-020-05598-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05598-9

Keywords

Search

Navigation