Skip to main content
SpringerLink
Log in

DSSN: dual shallow Siamese network for fashion image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image retrieval is a way to search similar images from a given query image. The method successfully applied to different fashion image retrieval over different garments and footwear. However, research over complex fashion products such as ornaments have not got much momentum due to the complex nature of similarity and unavailability of suitable datasets. In this paper we have proposed a Dual Shallow Siamese Network (DSSN) for the task. The model has two identical Siamese networks and takes the same images as input but in a different representation. One subnetwork takes input as RGB color images and the other as identical segmented images. First, the networks are trained using positive and negative image pairs. Next, the trained model is used to find the difference between gallery and query images. The similarity score of the Siamese networks are then fused using weighted averaging. The method is applied on two public datasets, namely, RingFIR (earring dataset) and UT-Zap50K (footwear dataset). We have compared the retrieval accuracy of our method with other state-of-the-art image retrieval methods. The result shows that our method outperforms state-of-the-art methods. The source code and dataset is available publicly (https://github.com/skarifahmed/DSSN).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The source code and dataset is available publicly (https://github.com/skarifahmed/DSSN).

The datasets generated during and/or analysed during the current study are available in the github repository, (https://github.com/skarifahmed/DSSN)

References

  1. Ak K E, Kassim A A, Lim J H, Tham J Y (2018) Learning attribute representations with localization for flexible fashion search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7708–7717

  2. Ak K E, Lim J H, Tham J Y, Kassim A A (2018) Which shirt for my first date? Towards a flexible attribute-based fashion query system. Pattern Recogn Lett 112:212–218

    Article  Google Scholar 

  3. Ak K E, Lim J H, Tham J Y, Kassim A (2019) Semantically consistent hierarchical text to fashion image synthesis with an enhanced-attentional generative adversarial network. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). IEEE, pp 3121–3124

  4. Alsmadi M K (2020) Content-based image retrieval using color, shape and texture descriptors and features. Arab J Sci Eng 45(4):3317–3330

    Article  Google Scholar 

  5. Beaupre D-A, Bilodeau G-A (2019) Siamese cnns for rgb-lwir disparity estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0

  6. Bibi R, Mehmood Z, Yousaf R M, Saba T, Sardaraz M, Rehman A (2020) Query-by-visual-search: multimodal framework for content-based image retrieval. J Ambient Intell Humaniz Comput 11(11):5629–5648

    Article  Google Scholar 

  7. Caltagirone L, Bellone M, Svensson L, Wahde M (2019) Lidar–camera fusion for road detection using fully convolutional neural networks. Robot Auton Syst 111:125–131

    Article  Google Scholar 

  8. Chaudhuri U, Banerjee B, Bhattacharya A (2019) Siamese graph convolutional network for content based remote sensing image retrieval. Comput Vis Image Underst 184:22–30

    Article  Google Scholar 

  9. Cheng Z-Q, Wu X, Liu Y, Hua X-S (2017) Video2shop: exact matching clothes in videos to online shopping images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4048–4056

  10. Chung D, Tahboub K, Delp E J (2017) A two stream siamese convolutional neural network for person re-identification. In: Proceedings of the IEEE international conference on computer vision, pp 1983–1991

  11. Chung Y-A, Weng W-H (2017) Learning deep representations of medical images using siamese cnns with application to content-based image retrieval. arXiv:1711.084901711.08490

  12. Corbiere C, Ben-Younes H, Ramé A, Ollion C (2017) Leveraging weakly annotated data for fashion image retrieval and label prediction. In: Proceedings of the IEEE international conference on computer vision workshops, pp 2268–2274

  13. Dong H, Liang X, Zhang Y, Zhang X, Shen X, Xie Z, Wu B, Yin J (2020) Fashion editing with adversarial parsing learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8120–8128

  14. Erkut U, Bostancıoğlu F, Erten M, Özbayoğlu A M, Solak E (2019) Hsv color histogram based image retrieval with background elimination. In: 2019 1st International Informatics and Software Engineering Conference (UBMYK). IEEE, pp 1–5

  15. Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. Computer Vision and Pattern Recognition Workshop

  16. Gajic B, Baldrich R (2018) Cross-domain fashion image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1869–1871

  17. Ge Y, Zhang R, Wang X, Tang X, Luo P (2019) Deepfashion2: a versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5337–5345

  18. Ghojogh B, Sikaroudi M, Shafiei S, Tizhoosh H R, Karray F, Crowley M (2020) Fisher discriminant triplet and contrastive losses for training siamese networks. In: 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, pp 1–7

  19. Gu X, Gao F, Tan M, Peng P (2020) Fashion analysis and understanding with artificial intelligence. Inform Process Manag 57(5):102276

    Article  Google Scholar 

  20. Guo Y, Cheung N-M (2018) Efficient and deep person re-identification using multi-level similarity. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2335–2344

  21. Ha I, Kim H, Park S, Kim H (2018) Image retrieval using bim and features from pretrained vgg network for indoor localization. Build Environ 140:23–31

    Article  Google Scholar 

  22. Hadi Kiapour M, Han X, Lazebnik S, Berg A C, Berg T L (2015) Where to buy it: matching street clothing photos in online shops. In: Proceedings of the IEEE international conference on computer vision, pp 3343–3351

  23. Hamreras S, Boucheham B, Molina-Cabello M A, Benitez-Rochel R, Lopez-Rubio E (2020) Content based image retrieval by ensembles of deep learning object classifiers. Integr Comput-Aided Eng 27(3):317–331

    Article  Google Scholar 

  24. Hidayati S C, Hsu C-C, Chang Y-T, Hua K-L, Fu J, Cheng W-H (2018) What dress fits me best? fashion recommendation on the clothing style for personal body shape. In: Proceedings of the 26th ACM international conference on Multimedia, pp 438–446

  25. Hsiao S-C, Kao D-Y, Liu Z-Y, Tso R (2019) Malware image classification using one-shot learning with siamese networks. Procedia Comput Sci 159:1863–1871

    Article  Google Scholar 

  26. Huang C-L, Huang D-H (1998) A content-based image retrieval system. Image Vis Comput 16(3):149–163

    Article  Google Scholar 

  27. Huang J, Feris R S, Chen Q, Yan S (2015) Cross-domain image retrieval with a dual attribute-aware ranking network. In: Proceedings of the IEEE international conference on computer vision, pp 1062–1070

  28. Huang L, Chen Y (2020) Dual-path siamese cnn for hyperspectral image classification with limited training samples. IEEE Geosci Remote Sens Lett

  29. Ilhan H O, Sigirci I O, Serbes G, Aydin N (2020) A fully automated hybrid human sperm detection and classification system based on mobile-net and the performance comparison with conventional methods. Medical & Biological Engineering & Computing, 1–22

  30. Islam S M, Joardar S, Dogra D P, Sekh A A (2021) Ornament image retrieval using multimodal fusion. SN Comput Sci 2(4):1–9

    Article  Google Scholar 

  31. Islam S M, Joardar S, Sekh A A (2021) Ringfir: a large volume earring dataset for fashion image retrieval. Springer, Singapore, pp 100–111

    Google Scholar 

  32. Jaradat S (2017) Deep cross-domain fashion recommendation. In: Proceedings of the Eleventh ACM conference on recommender systems, pp 407–410

  33. Jian M, Qi Q, Dong J, Yin Y, Lam K-M (2018) Integrating qdwd with pattern distinctness and local contrast for underwater saliency detection. J Vis Commun Image Represent 53:31–41

    Article  Google Scholar 

  34. Jian M, Wang J, Yu H, Wang G, Meng X, Yang L, Dong J, Yin Y (2021) Visual saliency detection by integrating spatial position prior of object with background cues. Expert Syst Appl 168:114219

    Article  Google Scholar 

  35. Jian M, Yin Y, Dong J, Lam K-M (2018) Content-based image retrieval via a hierarchical-local-feature extraction scheme. Multimed Tools Applic 77(21):29099–29117

    Article  Google Scholar 

  36. Jian M, Zhang W, Yu H, Cui C, Nie X, Zhang H, Yin Y (2018) Saliency detection based on directional patches extraction and principal local color contrast. J Vis Commun Image Represent 57:1–11

    Article  Google Scholar 

  37. Kalaiarasi G, Thyagharajan KK (2019) Clustering of near duplicate images using bundled features. Clust Comput 22(5):11997–12007

    Article  Google Scholar 

  38. Kang W-C, Fang C, Wang Z, McAuley J (2017) Visually-aware fashion recommendation and design with generative image models. In: 2017 IEEE International Conference on Data Mining (ICDM). IEEE, pp 207–216

  39. Karczmarek P, Kiersztyn A, Pedrycz W (2018) Generalized choquet integral for face recognition. Int J Fuzzy Syst 20(3):1047–1055

    Article  Google Scholar 

  40. Khurana T, Mahajan K, Arora C, Rai A (2018) Exploiting texture cues for clothing parsing in fashion images. In: 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, pp 2102–2106

  41. Kuang Z, Gao Y, Li G, Luo P, Chen Y, Lin L, Zhang W (2019) Fashion retrieval via graph reasoning networks on a similarity pyramid. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3066–3075

  42. Lang Y, He Y, Yang F, Dong J, Xue H (2020) Which is plagiarism: fashion image retrieval based on regional representation for design protection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2595–2604

  43. Lehmann T M, Güld M O, Thies C, Fischer B, Spitzer K, Keysers D, Ney H, Kohnen M, Schubert H, Wein B B (2004) Content-based image retrieval in medical applications. Methods Inform Med 43(04):354–361

    Article  Google Scholar 

  44. Li J, Chi Z, Chen G (2004) Image retrieval based on sugeno fuzzy integral. In: Third International Conference on Image and Graphics (ICIG’04). IEEE, pp 160–163

  45. Li Z, Li Y, Tian W, Pang Y, Liu Y (2016) Cross-scenario clothing retrieval and fine-grained style recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, pp 2912–2917

  46. Liang X, Lin L, Yang W, Luo P, Huang J, Yan S (2016) Clothes co-parsing via joint image segmentation and labeling with application to clothing retrieval. IEEE Trans Multimed 18(6):1175–1186

    Article  Google Scholar 

  47. Liao Q (2016) Comparison of several color histogram based retrieval algorithms. In: 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). IEEE, pp 1670–1673

  48. Lin C-H, Chen R-T, Chan Y-K (2009) A smart content-based image retrieval system based on color and texture feature. Image Vis Comput 27 (6):658–665

    Article  Google Scholar 

  49. Lin K, Yang H-F, Liu K-H, Hsiao J-H, Chen C-S (2015) Rapid clothing retrieval via deep learning of binary codes and hierarchical search. In: Proceedings of the 5th ACM on international conference on multimedia retrieval, pp 499–502

  50. Liu G-H, Yang J-Y (2021) Deep-seated features histogram: a novel image retrieval method. Pattern Recogn 116:107926

    Article  Google Scholar 

  51. Liu K-H, Chen T-Y, Chen C-S (2016) Mvc: a dataset for view-invariant clothing retrieval and attribute prediction. In: Proceedings of the 2016 ACM on international conference on multimedia retrieval, pp 313–316

  52. Liu Y, Chen X, Peng H, Wang Z (2017) Multi-focus image fusion with a deep convolutional neural network. Inform Fus 36:191–207

    Article  Google Scholar 

  53. Liu Z, Luo P, Qiu S, Wang X, Tang X (2016) Deepfashion: powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1096–1104

  54. Liu Z, Yan S, Luo P, Wang X, Tang X (2016) Fashion landmark detection in the wild. In: European conference on computer vision. Springer, pp 229–245

  55. Long F, Zhang H, Feng D D (2003) Fundamentals of content-based image retrieval. In: Multimedia information retrieval and management. Springer, pp 1–26

  56. Melekhov I, Kannala J, Rahtu E (2016) Siamese network features for image matching. In: 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, pp 378–383

  57. Miao Y, Li G, Bao C, Zhang J, Wang J (2020) Clothingnet: cross-domain clothing retrieval with feature fusion and quadruplet loss. IEEE Access 8:142669–142679

    Article  Google Scholar 

  58. Nanni L, Minchio G, Brahnam S, Maguolo G, Lumini A (2021) Experiments of image classification using dissimilarity spaces built with siamese networks. Sensors 21(5):1573

    Article  Google Scholar 

  59. Pandey N, Savakis A (2020) Poly-gan: multi-conditioned gan for fashion synthesis. Neurocomputing 414:356–364

    Article  Google Scholar 

  60. Pelka O, Nensa F, Friedrich C M (2018) Annotation of enhanced radiographs for medical image retrieval with deep convolutional neural networks. PloS one 13(11):e0206229

    Article  Google Scholar 

  61. Peng L, Zhang J, Liu M, Hu A (2019) Deep learning based rf fingerprint identification using differential constellation trace figure. IEEE Trans Veh Technol 69(1):1091–1095

    Article  Google Scholar 

  62. Qi Y, Song Y-Z, Zhang H, Liu J (2016) Sketch-based image retrieval via siamese convolutional neural network. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE, pp 2460–2464

  63. Saxen F, Werner P, Handrich S, Othman E, Dinges L, Al-Hamadi A (2019) Face attribute detection with mobilenetv2 and nasnet-mobile. In: 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA). IEEE, pp 176–180

  64. Sharma P, Reilly R B (2003) A colour face image database for benchmarking of automatic face detection algorithms. In: Proceedings EC-VIP-MC 2003. 4th EURASIP conference focused on video/image processing and multimedia communications (IEEE Cat. No. 03EX667), vol 1. IEEE, pp 423–428

  65. Shi M, Lewis V D (2020) Using artificial intelligence to analyze fashion trends. arXiv:2005.00986

  66. Su H, Wang P, Liu L, Li H, Li Z, Zhang Y (2020) Where to look and how to describe: fashion image retrieval with an attentional heterogeneous bilinear network. IEEE Trans Circuits Syst Video Technol

  67. Sun G-L, Wu X, Chen H-H, Peng Q (2015) Clothing style recognition using fashion attribute detection. In: Proceedings of the 8th international conference on mobile multimedia communications, pp 145–148

  68. Tan M, Le Q V (2021) Efficientnetv2: smaller models and faster training. arXiv:2104.00298

  69. Ter Braak CJF, Looman CWN (1986) Weighted averaging, logistic regression and the gaussian response model. Vegetatio 65(1):3–11

    Article  Google Scholar 

  70. Thyagharajan K K, Kalaiarasi G (2018) Pulse coupled neural network based near-duplicate detection of images (pcnn–ndd). Adv Electr Comput Eng 18(3):87–97

    Article  Google Scholar 

  71. Thyagharajan KK, Kalaiarasi G (2021) A review on near-duplicate detection of images using computer vision techniques. Arch Comput Methods Eng 28 (3):897–916

    Article  MathSciNet  Google Scholar 

  72. Tu Q, Dong L (2010) An intelligent personalized fashion recommendation system. In: 2010 International Conference on Communications, Circuits and Systems (ICCCAS). IEEE, pp 479–485

  73. Verma S, Anand S, Arora C, Rai A (2018) Diversity in fashion recommendation using semantic parsing. In: 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, pp 500–504

  74. Wang W, Xu Y, Shen J, Zhu S-C (2018) Attentive fashion grammar network for fashion landmark detection and clothing category classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4271–4280

  75. Wang X, Sun Z, Zhang W, Zhou Y, Jiang Y-G (2016) Matching user photos to online products with robust deep features. In: Proceedings of the 2016 ACM on international conference on multimedia retrieval, pp 7–14

  76. Welinder P, Branson S, Mita T, Wah C, Schroff F, Belongie S, Perona P (2010) Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, Cali711 fornia Institute of Technology

  77. Wiggers K L, Britto A S, Heutte L, Koerich A L, Oliveira L S (2019) Image retrieval and pattern spotting using siamese neural network. In: 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, pp 1–8

  78. Wu L, Wang Y, Gao J, Li X (2018) Where-and-when to look: deep siamese attention networks for video-based person re-identification. IEEE Trans Multimed 21(6):1412–1424

    Article  Google Scholar 

  79. Wu Q (2020) Image retrieval method based on deep learning semantic feature extraction and regularization softmax. Multimed Tools Applic 79 (13):9419–9433

    Article  Google Scholar 

  80. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. ArXiv:1708.07747

  81. Xu N, Liu A-A, Wong Y, Zhang Y, Nie W, Su Y, Kankanhalli M (2018) Dual-stream recurrent neural network for video captioning. IEEE Trans Circuits Syst Video Technol 29(8):2482–2493

    Article  Google Scholar 

  82. Yager R R (1988) On ordered weighted averaging aggregation operators in multicriteria decisionmaking. IEEE Trans Syst Man Cybern 18(1):183–190

    Article  MathSciNet  MATH  Google Scholar 

  83. Yamaguchi K, Kiapour M H, Ortiz L E, Berg T L (2012) Parsing clothing in fashion photographs. In: 2012 IEEE Conference on computer vision and pattern recognition. IEEE, pp 3570–3577

  84. Yasmin M, Mohsin S, Sharif M (2014) Intelligent image retrieval techniques: a survey. J Appl Res Technol 12(1):87–103

    Article  Google Scholar 

  85. Yin R, Li K, Lu J, Zhang G (2019) Enhancing fashion recommendation with visual compatibility relationship. In: The world wide web conference, pp 3434–3440

  86. Yu A, Grauman K (2014) Fine-grained visual comparisons with local learning. In: Computer Vision and Pattern Recognition (CVPR)

  87. Yu A, Grauman K (2017) Semantic jitter: dense supervision for visual comparisons via synthetic images. In: International Conference on Computer Vision (ICCV)

  88. Yu L, Liu N, Zhou W, Dong S, Fan Y, Abbas K (2021) Weber’s law based multi-level convolution correlation features for image retrieval. Multimed Tools Applic 80(13):19157–19177

    Article  Google Scholar 

  89. Zhang J, Lu C, Li X, Kim H-J, Wang J (2019) A full convolutional network based on densenet for remote sensing scene classification. Math Biosci Eng 16(5):3345–3367

    Article  Google Scholar 

  90. Zhang J, Hu J (2008) Image segmentation based on 2d otsu method with histogram analysis. In: 2008 international conference on computer science and software engineering, vol 6. IEEE, pp 105–108

  91. Zhao L, Min C (2019) The rise of fashion informatics: a case of data-mining-based social network analysis in fashion. Cloth Text Res J 37(2):87–102

    Article  Google Scholar 

  92. Zheng S, Yang F, Kiapour M H, Piramuthu R (2018) Modanet: a large-scale street fashion dataset with polygon annotations. In: ACM Multimedia conference on multimedia conference. ACM, pp 1670–1678

  93. Zhou W, Mok PY, Zhou Y, Zhou Y, Shen J, Qu Q, Chau KP (2019) Fashion recommendations through cross-media information retrieval. J Vis Commun Image Represent 61:112–120

    Article  Google Scholar 

  94. Zhu S, Urtasun R, Fidler S, Lin D, Change Loy C (2017) Be your own prada: fashion synthesis with structural coherence. In: Proceedings of the IEEE international conference on computer vision, pp 1680–1688

  95. Zou X, Kong X, Wong W, Wang C, Liu Y, Cao Y (2019) Fashionai: a hierarchical dataset for fashion understanding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0

Download references

Author information

Authors and Affiliations

  1. Global Institute of Science and Technology, Purba Medinipur, India

    Sk Maidul Islam

  2. Haldia Institute of Technology, Purba Medinipur, India

    Subhankar Joardar

  3. XIM University, Bhubaneswar, India

    Arif Ahmed Sekh

Authors
  1. Sk Maidul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Subhankar Joardar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arif Ahmed Sekh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sk Maidul Islam.

Ethics declarations

Ethics approval and consent to participate

This article does not contain any studies with human participants or animals performed by any of the authors. Informed consent was obtained from all individual participants included in the study.

Conflict of Interests

The authors declare that there is no conflict of interest regarding the publication of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Islam, S.M., Joardar, S. & Sekh, A.A. DSSN: dual shallow Siamese network for fashion image retrieval. Multimed Tools Appl 82, 16501–16517 (2023). https://doi.org/10.1007/s11042-022-14204-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14204-0

Keywords

Search

Navigation