Skip to main content
SpringerLink
Log in

One-Shot Retail Product Identification Based on Improved Siamese Neural Networks

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Conventional retail stores are undergoing digital transformation, and in a typical smart retail store, automatic recognition of retail products is essential for customer experience in the checkout stage. In this paper, we propose an improved Siamese neural network to identify the product from one-shot learning. First, a spatial channel dual attention mechanism is proposed to improve the network architecture. Second, a binary cross-entropy loss function with a distance penalty is adopted to replace the conventional contrastive loss function. The proposed network can better model the details of the products. The experimental results are achieved on two public available databases. The results show that the proposed method outperforms the conventional methods, and it can solve the data insufficient problem in the training stage. Smart retail stores can change the SKUs (Stock Keeping Units) conveniently without collecting a large amount of training samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability Statement

The data sets related to this work are public available from two public databases: RPC Database and GroZi-120 Database.

References

  1. J. Bromley, J.W. Bentz, L. Bottou et al., Signature verification using a “siamese’’ time delay neural network. Int. J. Pattern Recogn. Artif. Intell. 7(04), 669–688 (1993)

    Article  Google Scholar 

  2. Q. Cai, Y. Pan, T. Yao, C. Yan, T. Mei, Memory matching networks for one-shot image recognition, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 4080–4088

  3. T. Chong, I. Bustan, M. Wee, Deep learning approach to planogram compliance in retail stores, Semant. Scholar 1–6 (2016)

  4. D. Chicco, Siamese neural networks: an overview. Artif. Neural Netw. 2190, 73–94 (2021)

    Article  Google Scholar 

  5. A. De Biasio, Retail shelf analytics through image processing and deep learning, Master Thesis, Department of Information Engineering, University of Padua, Italy (2019)

  6. D. Farren, Classifying food items by image using Convolutional Neural Networks, report, Stanford University (2017)

  7. W. Geng, F. Han, J. Lin. et al., Fine-grained grocery product recognition by one-shot learning, in Proceedings of the 26th ACM International Conference on Multimedia (2019), pp. 1706–1714

  8. E. Goldman, J. Goldberger, Large-scale classification of structured objects using a CRF with deep class embedding. arXiv:1705.07420 (2017)

  9. K. He, X. Zhang, S. Ren, et al., Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778

  10. R. Hadsell, S. Chopra, Y. LeCun, Dimensionality reduction by learning an invariant mapping, in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006), pp. 1735–1742

  11. J. Hu, L. Shen, S. Albanie, G. Sun, E. Wu, Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020)

    Article  Google Scholar 

  12. P. Jund, N. Abdo, A. Eitel, et al., The freiburg groceries dataset. Preprint arXiv:1611.05799 (2016)

  13. L. Karlinsky, J. Shtok, Y. Tzur, et al., Fine-grained recognition of thousands of object categories with single-example training, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 4113–4122

  14. G. Koch, R. Zemel, R. Salakhutdinov, Siamese neural networks for one-shot image recognition, ICML deep learning workshop (2015)

  15. D. Kingma, J. Ba, Adam: A Method for Stochastic Optimization. Preprint arXiv:1412.6980 (2014)

  16. J. Li, X. Wang, H. Su, Supermarket commodity identification using convolutional neural networks, in 2016 2nd International Conference on Cloud Computing and Internet of Things (CCIOT) (2016), pp. 115–119

  17. L. Liu, B. Zhou, Z. Zou, et al., A smart unstaffed retail shop based on artificial intelligence and IoT, in 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD) (2018), pp. 1–4

  18. C.Y. Lee, S. Xie, P. Gallagher, et al., Deeply-supervised nets, in Proc. of the 18-th International Conference on Artificial Intelligence and Statistics (AISTATS) (San Diego, CA, USA, 2015), pp. 562–570

  19. X. Li, M. He, H. Li, H. Shen, A combined loss-based multiscale fully convolutional network for high-resolution remote sensing image change detection. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2022)

    Google Scholar 

  20. C.G. Melek, E.B. Sonmez, S. Albayrak, A survey of product recognition in shelf images, in 2017 International Conference on Computer Science and Engineering (UBMK) (2017), pp. 145–150

  21. M. Merler, C. Galleguillos, S. Belongie, Recognizing groceries in situ using in vitro training data, in 2007 IEEE Conference on Computer Vision and Pattern Recognition (2007), pp. 1–8

  22. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in International Conference on Learning Representations (2015)

  23. C. Szegedy, S. Ioffe, V. Vanhoucke, et al., Inception-v4, inception-resnet and the impact of residual connections on learning, in Thirty-First AAAI Conference on Artificial Intelligence (2017)

  24. M. Shorfuzzaman, M. Hossain, MetaCOVID: a Siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients. Pattern Recogn. 113, 107700 (2021)

    Article  Google Scholar 

  25. A. Tonioni, L. Di Stefano, Domain invariant hierarchical embedding for grocery products recognition. Comput. Vis. Image Understanding 182, 81–92 (2019)

    Article  Google Scholar 

  26. A. Tonioni, E. Serra, L. Di Stefano, A deep learning pipeline for product recognition on store shelves, in 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS) (2018), pp. 25–31

  27. S. Varadarajan, M.M. Srivastava, Weakly supervised object localization on grocery shelves using simple FCN and synthetic dataset, in Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing (2018), pp. 1–7

  28. O. Vinyals, C. Blundell, T. Lillicrap et al., Matching networks for one shot learning. Adv. Neural Inf. Process. Syst. 29, 3630–3638 (2016)

    Google Scholar 

  29. Z. Wang, C. Peng, Y. Zhang, N. Wang, L. Luo, Fully convolutional siamese networks based change detection for optical aerial images with focal contrastive loss. Neurocomputing 457, 155–167 (2021)

    Article  Google Scholar 

  30. X.S. Wei, Q. Cui, L. Yang, et al., RPC: A large-scale retail product checkout dataset. Preprint arXiv:1901.07249 (2019)

  31. S. Woo, J. Park, J.Y. Lee, I.S. Kweon, Cbam: convolutional block attention module, in Proceedings of the European Conference on Computer Vision (ECCV) (2018), pp. 3–19

  32. H. Zhao, J. Shi, X. Qi, et al., Pyramid scene parsing network, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 2881–2890

Download references

Author information

Authors and Affiliations

  1. School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China

    Chunchieh Wang & Liye Zhao

  2. Zhejiang Lab, Hangzhou, China

    Chengwei Huang & Xiaoming Zhu

Authors
  1. Chunchieh Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengwei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liye Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liye Zhao.

Ethics declarations

Conflict of interest

The authors declare that there are no competing interests related to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Chunchieh Wang and Chengwei Huang are co-first author.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Huang, C., Zhu, X. et al. One-Shot Retail Product Identification Based on Improved Siamese Neural Networks. Circuits Syst Signal Process 41, 6098–6112 (2022). https://doi.org/10.1007/s00034-022-02062-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-022-02062-y

Keywords

Search

Navigation