Skip to main content
SpringerLink
Log in

Accounting for class hierarchy in object classification using Siamese neural networks

  • Published:
Computational Mathematics and Modeling Aims and scope Submit manuscript

Abstract

Siamese neural networks are an effective architecture for automatic construction of vector representations of objects, by whose comparison it is possible to solve the classification problem. The mentioned approach utilizes the training selection more effectively; it is capable of distinguishing classes by a small number of samples, and it can learn with a dynamic number of classes. We propose an improved method of tuning the Siamese neural networks for solving the classification problem, that uses information about the class hierarchy. The advantage of the mentioned method is demonstrated in examples of image and text classifications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05). vol. 1. IEEE, pp. 539–546 (2005). https://doi.org/10.1109/CVPR.2005.202

    Book  Google Scholar 

  2. Chen, W., et al.: Beyond triplet loss: a deep quadruplet network for person re-identification. Proceedings of the IEEE conference on computer vision and pattern recognition., pp. 403–412 (2017)

    Google Scholar 

  3. Ciraco, M., Rogalewski, M., Weiss, G.: Improving classifier utility by altering the misclassification cost ratio. Proceedings of the 1st international workshop on Utility-based data mining., pp. 46–52 (2005). https://doi.org/10.1145/1089827.1089833

    Book  Google Scholar 

  4. Liang, J.: Confusion matrix: Machine learning. POGIL Activity Clgh. 3(4), (2022)

  5. Masters, D., Luschi, C.: Revisiting small batch training for deep neural networks. (2018). arXiv preprint arXiv:1804.07612

  6. Bender, M.A., et al.: Lowest common ancestors in trees and directed acyclic graphs. J. Algorithms 57(2), 75–94 (2005)

    Article  MathSciNet  Google Scholar 

  7. Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. Proceedings of the European conference on computer vision (ECCV)., pp. 459–474 (2018)

    Google Scholar 

  8. Krizhevsky, A., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  9. Memon, S.A., Khan, K.A., Naveed, H.: HECNet: a hierarchical approach to enzyme function classification using a Siamese Triplet Network. Bioinformatics 36(17), 4583–4589 (2020). https://doi.org/10.1093/bioinformatics/btaa536

    Article  CAS  PubMed  Google Scholar 

  10. Barz, B., Denzler, J.: Hierarchy-based image embeddings for semantic image retrieval. 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, pp. 638–647 (2019). https://doi.org/10.1109/WACV.2019.00073

    Book  Google Scholar 

  11. Lee, N., Hong, S., Kim, H.: Single-Trace Attack Using One-Shot Learning With Siamese Network in Non-Profiled Setting. IEEE Access 10, 60778–60789 (2022). https://doi.org/10.1109/ACCESS.2022.3180742

    Article  Google Scholar 

  12. Wu, Y., et al.: A novel Siamese network object tracking algorithm based on tensor space mapping and memory-learning mechanism. J. Vis. Commun. Image. Represent. 91, 103742 (2023)

    Article  Google Scholar 

  13. Heidari, M., Fouladi-Ghaleh, K.: Using Siamese networks with transfer learning for face recognition on small-samples datasets. 2020 International Conference on Machine Vision and Image Processing (MVIP). IEEE, pp. 1–4 (2020). https://doi.org/10.1109/MVIP49855.2020.9116915

    Book  Google Scholar 

  14. Deng, J., et al.: Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848

    Book  Google Scholar 

  15. Amin, A.H.M., Khan, A.I.: One-shot classification of 2‑D leaf shapes using distributed hierarchical graph neuron (DHGN) scheme with k‑NN classifier. Procedia Comput Sci 24, 84–96 (2013)

    Article  Google Scholar 

  16. Vilcek, A., et al.: Transformer-Based Deep Siamese Network for At-Scale Product Matching and One-Shot Hierarchy Classification (2018)

    Google Scholar 

  17. Song, C., Ji, S.: Face Recognition Method Based on Siamese Networks Under Non-Restricted Conditions. IEEE Access 10, 40432–40444 (2022). https://doi.org/10.1109/ACCESS.2022.3167143

    Article  Google Scholar 

  18. Melekhov, I., Kannala, J., Rahtu, E.: Siamese network features for image matching. 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp. 378–383 (2016). https://doi.org/10.1109/ICPR.2016.7899663

    Book  Google Scholar 

  19. Schroff, F., Kalenichenko, D., Facenet, P.J.: A unified embedding for face recognition and clustering. Proceedings of the IEEE conference on computer vision and pattern recognition., pp. 815–823 (2015)

    Google Scholar 

  20. Daudt, R.C., Le Saux, B., Boulch, A.: Fully convolutional siamese networks for change detection. 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, pp. 4063–4067 (2018). https://doi.org/10.1109/ICIP.2018.8451652

    Book  Google Scholar 

  21. Wu, Y., et al.: Siamese capsule networks with global and local features for text classification. Neurocomputing 390, 88–98 (2020)

    Article  Google Scholar 

  22. Wang, C., Tzanetakis, G.: Singing style investigation by residual siamese convolutional neural networks. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 116–120 (2018). https://doi.org/10.1109/ICASSP.2018.8461660

    Book  Google Scholar 

  23. He, A., et al.: A twofold siamese network for real-time object tracking. Proceedings of the IEEE conference on computer vision and pattern recognition., pp. 4834–4843 (2018)

    Google Scholar 

  24. Barz, B., Denzler, J.: Hierarchy-based image embeddings for semantic image retrieval. 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, pp. 638–647 (2019). https://doi.org/10.1109/WACV.2019.00073

    Book  Google Scholar 

  25. Memon, S.A., Khan, K.A., Naveed, H.: HECNet: a hierarchical approach to enzyme function classification using a Siamese Triplet Network. Bioinformatics 36(17), 4583–4589 (2020). https://doi.org/10.1093/bioinformatics/btaa536

    Article  CAS  PubMed  Google Scholar 

  26. Ge, W.: Deep metric learning with hierarchical triplet loss. Proceedings of the European conference on computer vision (ECCV)., pp. 269–285 (2018)

    Google Scholar 

  27. Haque, T.U., Saber, N.N., Shah, F.M.: Sentiment analysis on large scale Amazon product reviews. 2018 IEEE international conference on innovative research and development (ICIRD). IEEE, pp. 1–6 (2018). https://doi.org/10.1109/ICIRD.2018.8376299

    Book  Google Scholar 

  28. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. (2017). arXiv preprint arXiv:1708.07747

  29. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. – (2012). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  30. He, K., et al.: Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition., pp. 770–778 (2016)

    Google Scholar 

  31. Sak, H., Senior, A., Beaufays, F.: Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. (2014). arXiv preprint arXiv:1402.1128

  32. Kingma, D.P., Ba, J.A.: A method for stochastic optimization. (2014). arXiv preprint arXiv:1412.6980

  33. Mikolov, T., et al.: Efficient estimation of word representations in vector space. (2013). arXiv preprint arXiv:1301.3781

  34. Maćkiewicz, A., Ratajczak, W.: Principal components analysis (PCA). Comput. Geosci. 19(3), 303–342 (1993)

    Article  ADS  Google Scholar 

  35. Gan, Y., Yang, J., Lai, W.: Video object forgery detection algorithm based on VGG-11 convolutional neural network. 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS). IEEE, pp. 575–580 (2019). https://doi.org/10.1109/ICICAS48597.2019.00126

    Book  Google Scholar 

  36. Mannor, S., Peleg, D., Rubinstein, R.: The cross entropy method for classification. Proceedings of the 22nd international conference on Machine learning, vol. 2005, pp. 561–568

Download references

Funding

This research was performed in the framework of the state task in the field of scientific activity of the Ministry of Science and Higher Education of the Russian Federation, project “Models, methods, and algorithms of artificial intelligence in the problems of economics for the analysis and style transfer of multidimensional datasets, time series forecasting, and recommendation systems design”, grant no. FSSW-2023-0004.

Author information

Authors and Affiliations

  1. LLC Yandex.Technologies, Moscow, Russian Federation

    V. V. Ponamaryov

  2. Artificial Intelligence Laboratory, Plekhanov Russian University of Economics, Moscow, Russian Federation

    V. V. Kitov & V. A. Kitov

  3. Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, Moscow, Russian Federation

    V. V. Kitov

Authors
  1. V. V. Ponamaryov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. V. Kitov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. A. Kitov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. V. Ponamaryov.

Additional information

Translated from Prikladnaya Matematika i Informatika, No. 73, 2023, pp. 38–57.

This article is a translation of the original article published in Russian in the journal Prikladnaya Matematika i Informatika. The translation was done with the help of an artificial intelligence machine translation tool, and subsequently reviewed and revised by an expert with knowledge of the field. Springer Nature works continuously to further the development of tools for the production of journals, books and on the related technologies to support the authors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ponamaryov, V., Kitov, V. & Kitov, V. Accounting for class hierarchy in object classification using Siamese neural networks. Comput Math Model (2024). https://doi.org/10.1007/s10598-024-09593-w

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10598-024-09593-w

Keywords

Search

Navigation