Skip to main content
SpringerLink
Log in

A knowledge-sharing semi-supervised approach for fashion clothes classification and attribute prediction

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Fashion is a form of self-expression that permits us to manifest our personality and identity with more confidence. Visual fashion clothing analysis has attracted researchers who have sought to pioneer deep learning concepts. In this work, we introduce a semi-supervised multi-task learning approach intending to attain clothing category classification and attribute prediction. For intensifying semi-supervised fashion clothes analysis, we embrace a teacher–student (T–S) pair model that utilises weighted loss minimisation while sharing knowledge between teacher and student. Our focus in this work is on strengthening the feature representation by simultaneous learning of labelled and unlabelled samples that avoids additional training for unlabelled samples. As a result, our approach involves in gaining beneficiary performance by making use of semi-supervised learning in fashion clothing analysis. We evaluated the proposed approach on the large-scale DeepFashion-C dataset and the combined unlabelled dataset obtained from six publicly available datasets. Experimental results show that the proposed paired architecture involving deep neural networks is comparable to state-of-the-art techniques in fashion clothing analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Jingyuan, L., Lu, H.: Deep fashion analysis with feature map upsampling and landmark-driven attention. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 30–36 (2018)

  2. Li, Y., Tang, S., Ye, Y., Ma, J.: Spatial-aware non-local attention for fashion landmark detection. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), pp. 820–825 (2019)

  3. Wang,W., Xu, Y., Shen, J., Zhu, S.-C.: Attentive fashion grammar network for fashion landmark detection and clothing category classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4271–4280 (2018)

  4. Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1096–1104 (2016)

  5. Lee, S., Eun, H., Oh, S., Kim, W., Jung, C., Kim, C.: Landmark-free clothes recognition with a two-branch feature selective network. Electron. Lett. 55(13), 745 (2019)

    Article  Google Scholar 

  6. Li, P., Li, Y., Jiang, X., Zhen, X.: Two-stream multi-task network for fashion recognition. arXiv preprint arXiv:1901.10172 (2019)

  7. Lee, S., Oh, S., Jung, C., Kim, C.: A Global-local embedding module for fashion landmark detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 3153-3156 (2019)

  8. Ferreira, B.Q., Costeira, J.P., Sousa, R.G., Gui, L.Y., Gomes, J.P.: Pose Guided Attention for Multi-Label Fashion Image Classification. arXiv preprint arXiv:1911.05024 (2019)

  9. Gammerman, A., Vovk, V., Vapnik, V.: Learning by transduction. arXiv preprint arXiv:1301.7375 (2013)

  10. Zhang, Y.: Support Vector Machine Classification Algorithm and Its Application. Int. Conf. Inf. Comput. Appl (ICICA). 308, 179–186 (2012)

    Google Scholar 

  11. Sawant, S.S., Prabukumar, M.: A review on graph-based semi-supervised learning methods for hyperspectral image classification. The Egyptian Journal of Remote Sensing and Space Science. (2018)

  12. Kingma, D.P., Rezende, D.J., Mohamed, S., Welling, M.: Semi-supervised learning with deep generative models. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 3581–3589, (2014)

  13. Odena, A.: Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:1606.01583 (2016)

  14. Ma, J., Tang, W., Zhu, J., Mei, Q.: A flexible generative framework for graph-based semi-supervised learning. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 3281–3290 (2019)

  15. Lee, D.H.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, 3(2) (2013)

  16. Van Engelen, J.E., Hoos, H.H.: A survey on semi-supervised learning. Mach. Learn. 109(2), 373–440 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  17. Oliver, A., Odena, A., Raffel, C., Cubuk, E.D., Goodfellow, I.J.: Realistic evaluation of deep semi-supervised learning algorithms. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS), pp. 3239–3250 (2018)

  18. Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. In: Proceedings of the 17th International Conference on Neural Information Processing Systems (NIPS), pp. 281-296 (2005)

  19. Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 1195–1204 (2017)

  20. Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242 (2016)

  21. Miyato, T., Maeda, S.I., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1979–1993 (2018)

    Article  Google Scholar 

  22. Iscen, A., Tolias, G., Avrithis, Y., Chum, O.: Label propagation for deep semi-supervised learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5070–5079 (2019)

  23. Huang, C.-Q., Chen, J.-K., Pan, Y., Lai, H.-J., Yin, J., Huang, Q.-H.: Clothing landmark detection using deep networks with prior of key point associations. IEEE Trans. Cybern. 49(10), 3744–3754 (2019)

    Article  Google Scholar 

  24. Corbiere, C., Ben-Younes, H., Ram, A., Ollion, C.: Leveraging weakly annotated data for fashion image retrieval and label prediction. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2268–2274 (2017)

  25. Cho, H.,Ahn, C.,MinYoo, C. , Seol, J., Lee, S.G.: Leveraging class hierarchy in fashion classification. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 3197–3200 (2019)

  26. Loni, B., Cheung, L.Y., Riegler, M., Bozzon, A., Gottlieb, L., Larson, M.: Fashion 10000: an enriched social image dataset for fashion and clothing. In Proceedings of the 5th ACM Multimedia Systems Conference, pp. 41–46 (2014)

  27. Guo, S., Huang, W., Zhang, X., Srikhanta, P., Cui, Y., Li, Y., Adam, H., Scott, M.R., Belongie, S.: The imaterialist fashion attribute dataset. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW) (2019)

  28. Chen, H., Gallagher, A., Girod, B.: Describing clothing by semantic attributes. European Conference on Computer Vision (ECCV), Springer, Berlin, Heidelberg, pp. 609–623 (2012)

  29. Bossard, L., Dantone, M., Leistner, C., Wengert, C., Quack, T., Van Gool, L.: Apparel classification with style. Asian Conference on Computer Vision (ACCV), Springer, Berlin, Heidelberg, pp. 321–335 (2012)

  30. Yalniz, I.Z., Jégou, H., Chen, K., Paluri, M., Mahajan, D.: Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546 (2019)

  31. Ye, Y., Li, Y., Wu, B., Zhang, W., Duan, L., Mei, T.: Hard-Aware Fashion Attribute Classification. arXiv preprint arXiv:1907.10839. (2019)

  32. Ak, K.E., Lim, J.H., Tham, J.Y., Kassim, A.A.: Attribute manipulation generative adversarial networks for fashion images. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 10540–10549 (2019)

  33. Li, Z., Cui, Z., Wu, S., Zhang, X., Wang, L.: Semi-Supervised Compatibility Learning Across Categories for Clothing Matching. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), pp. 484-489 (2019)

  34. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. In: Advances in Neural Information Processing Systems (NIPS), pp. 2234–2242 (2016)

  35. Shajini, M., Ramanan, A.: An improved landmark-driven and spatial-channel attentive convolutional neural network for fashion clothes classification, pp. 1–10. The Visual Computer, Springer (2020)

    Google Scholar 

  36. He,K., Zhang, X., Ren, S. and Sun, J.:Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  37. Kingma, D. P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the Third International Conference on Learning Representations (2015)

  38. Hadi Kiapour, M., Han, X., Lazebnik, S., Berg, A.C., Berg, T.L.: Where to buy it: matching street clothing photos in online shops. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 3343–3351 (2015)

  39. Huang, J., Feris, R.S., Chen, Q., Yan, S.: Cross-domain image retrieval with a dual attribute-aware ranking network. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1062–1070 (2015)

  40. Zhang, Y., Zhang, P., Yuan, C., Wang, Z.: Texture and Shape Biased Two-Stream Networks for Clothing Classification and Attribute Recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13538–13547 (2020)

  41. Kendall, A., Gal, Y. Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 7482–7491 (2018)

  42. Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, 2. (2015)

  43. Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 4077–4087 (2017)

  44. Wang, X., Hu, J.F., Lai, J.H., Zhang, J., Zheng, W.S.: Progressive teacher-student learning for early action prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3556–3565 (2019)

  45. Li, Y., Wang, N., Liu, J., Hou, X.: Demystifying neural style transfer. arXiv preprint arXiv:1701.01036 (2017)

  46. Yim, J., Joo, D., Bae, J., Kim, J.: A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4133–4141 (2017)

  47. Cheng, X., Rao, Z., Chen, Y., Zhang, Q.: Explaining knowledge distillation by quantifying the knowledge. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12925–12935 (2020)

  48. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

  49. Xu, G., Liu, Z., Li, X., Loy, C.C.: Knowledge distillation meets self-supervision. In European Conference on Computer Vision (ECCV), Springer, pp. 588–604 (2020)

  50. Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 3630–3638 (2017)

  51. Flennerhag, S., Rusu, A.A., Pascanu, R., Visin, F., Yin, H., Hadsell, R.: Meta-learning with warped gradient descent. arXiv preprint arXiv:1909.00025 (2020)

  52. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inform. Proc. Syst. (NIPS) 25, 1097–1105 (2012)

    Google Scholar 

  53. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the Third International Conference on Learning Representations (ICLR) (2015)

  54. Furlanello, T., Lipton, Z., Tschannen, M., Itti, L., Anandkumar, A.: Born again neural networks. In: International Conference on Machine Learning, pp. 1607–1616 (2018)

  55. Zhu, M., Han, K., Zhang, C., Lin, J., Wang, Y.: Low-resolution visual recognition via deep feature distillation. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3762–3766 (2019)

  56. Meng, Z., Li, J., Zhao, Y., Gong, Y.: Conditional teacher-student learning. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6445-6449 (2019)

  57. Kabbai, L., Abdellaoui, M., Douik, A.: Image classification by combining local and global features. Vis. Comput. 35(5), 679–693 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Science, University of Jaffna, Jaffna, Sri Lanka

    Majuran Shajini & Amirthalingam Ramanan

Authors
  1. Majuran Shajini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amirthalingam Ramanan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Majuran Shajini.

Ethics declarations

Conflict of Interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shajini, M., Ramanan, A. A knowledge-sharing semi-supervised approach for fashion clothes classification and attribute prediction. Vis Comput 38, 3551–3561 (2022). https://doi.org/10.1007/s00371-021-02178-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02178-3

Keywords

Search

Navigation