Skip to main content

Advertisement

SpringerLink
Log in

Generalized Few-Shot Classification with Knowledge Graph

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Few-shot learning aims to discriminate images from novel categories using only a few available training examples. While existing few-shot methods can adapt quickly and precisely to new categories, they often struggle to retain knowledge of base categories that were used in the training phase. To address this challenge and support lifelong learning, generalized few-shot learning has been introduced to enable few-shot models to classify both base and novel categories. However, as the number of categories increases, few-shot models can lose efficiency due to the limited amount of visual information available for each category. To address this limitation, we propose the knowledge-augmented weight generation (KAWG) method, which incorporates semantic information in addition to visual features. Specifically, KAWG combines textual descriptions and entity relationships extracted from knowledge graphs and visual features to generate more robust classifiers for generalized few-shot learning tasks. Through our meta-training strategy, KAWG can retain the knowledge learned from base categories to the greatest extent when transferring to novel classes. Experiments show that our approach achieves state-of-the-art performance on some generalized few-shot benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. Advances in neural information processing systems 30

  2. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. In: Advances in neural information processing systems, vol 29

  3. Gidaris S, Komodakis N (2019) Generating classification weights with GNN denoising autoencoders for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 21–30

  4. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1199–1208

  5. Ye H-J, Hu H, Zhan D-C, Sha F (2020) Few-shot learning via embedding adaptation with set-to-set functions. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8808–8817

  6. Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3(1):1–40

    Article  Google Scholar 

  7. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning. PMLR, pp 1126–1135

  8. Wang Y, Yao Q, Kwok JT, Ni LM (2020) Generalizing from a few examples: a survey on few-shot learning. ACM Comput Surv (CSUR) 53(3):1–34

    Article  Google Scholar 

  9. Chen W-Y, Liu Y-C, Kira Z, Wang Y-CF, Huang J-B (2019) A closer look at few-shot classification. arXiv preprint arXiv:1904.04232

  10. Hospedales T, Antoniou A, Micaelli P, Storkey A (2021) Meta-learning in neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(9):5149–5169

    Google Scholar 

  11. Li A, Luo T, Xiang T, Huang W, Wang L (2019) Few-shot learning with global class representations. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9715–9724

  12. Gidaris S, Komodakis N (2018) Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4367–4375

  13. Qi H, Brown M, Lowe DG (2018) Low-shot learning with imprinted weights. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5822–5830

  14. Chen F, Zhang D, Han M, Chen X, Shi J, Xu S, Xu B (2022) VLP: a survey on vision-language pre-training. arXiv preprint arXiv:2202.09061

  15. Zhou L, Palangi H, Zhang L, Hu H, Corso J, Gao J (2020) Unified vision-language pre-training for image captioning and VQA. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 13041–13049

  16. Lin Y, Chi Y, Han H, Han M, Guo Y (2022) Multimodal orthodontic corpus construction based on semantic tag classification method. Neural Process Lett 54(4):2817–2830

    Article  Google Scholar 

  17. Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng AY (2011) Multimodal deep learning. In: ICML

  18. Miller GA (1995) Wordnet: a lexical database for English. Commun ACM 38(11):39–41

    Article  Google Scholar 

  19. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207

  20. Zhao J, Tang T, Yu Y, Wang J, Yang T, Chen M, Wu J (2022) Adaptive meta transfer learning with efficient self-attention for few-shot bearing fault diagnosis. Neural Process Lett 2020:1–20

    Google Scholar 

  21. He Y, Zang C, Zeng P, Dong Q, Liu D, Liu Y (2022) Convolutional shrinkage neural networks based model-agnostic meta-learning for few-shot learning. Neural Process Lett 2022:1–14

    Google Scholar 

  22. Wang Q, Wang G, Kou G, Zang M, Wang H (2021) Application of meta-learning framework based on multiple-capsule intelligent neural systems in image classification. Neural Process Lett 53(4):2581–2602

    Article  Google Scholar 

  23. Oreshkin B, Rodríguez López P, Lacoste A (2018) Tadam: task dependent adaptive metric for improved few-shot learning. In: Advances in neural information processing systems, vol 31

  24. Ravi S, Larochelle H (2016) Optimization as a model for few-shot learning

  25. Lyu B, Wen S, Shi K, Huang T (2023) Multiobjective reinforcement learning-based neural architecture search for efficient portrait parsing. IEEE Trans Cybern 53(2):1158–1169. https://doi.org/10.1109/TCYB.2021.3104866

  26. Lyu B, Hamdi M, Yang Y, Cao Y, Yan Z, Li K, Wen S, Huang T (2023) Efficient spectral graph convolutional network deployment on memristive crossbars. IEEE Trans Emerg Top Comput Intell 7(2):415–425. https://doi.org/10.1109/TETCI.2022.3210998

  27. Zhu Z, Lin X (2021) Kan: knowledge-augmented networks for few-shot learning. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 1735–1739

  28. Chen R, Chen T, Hui X, Wu H, Li G, Lin L (2020) Knowledge graph transfer network for few-shot recognition. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 10575–10582

  29. Kampffmeyer M, Chen Y, Liang X, Wang H, Zhang Y, Xing EP (2019) Rethinking knowledge graph propagation for zero-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition pp 11487–11496

  30. Yang S, Liu Y, Zhang Y, Zhu J (2022) A word-concept heterogeneous graph convolutional network for short text classification. Neural Process Lett 2022:1–16

    Google Scholar 

  31. Tian Y, Wang Y, Krishnan D, Tenenbaum JB, Isola P (2020) Rethinking few-shot image classification: a good embedding is all you need? In: European conference on computer vision. Springer, pp 266–282

  32. Rajasegaran J, Khan S, Hayat M, Khan FS, Shah M (2020) Self-supervised knowledge distillation for few-shot learning. arXiv preprint arXiv:2006.09785

  33. Rizve MN, Khan S, Khan FS, Shah M (2021) Exploring complementary strengths of invariant and equivariant representations for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10836–10846

  34. Hinton G, Vinyals O, Dean J et al (2015) Distilling the knowledge in a neural network, vol. 2, no. 7. arXiv preprint arXiv:1503.02531

  35. Paulheim H (2017) Knowledge graph refinement: a survey of approaches and evaluation methods. Semant Web 8(3):489–508

    Article  Google Scholar 

  36. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  37. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1532–1543

  38. Ren M, Liao R, Fetaya E, Zemel R (2019) Incremental few-shot learning with attention attractor networks. In: Advances in neural information processing systems, vol 32

  39. Kukleva A, Kuehne H, Schiele B (2021) Generalized and incremental few-shot learning by explicit learning and calibration without forgetting. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9020–9029

  40. Zhao J, Yang Y, Lin X, Yang J, He L (2021) Looking wider for better adaptive representation in few-shot learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 10981–10989

  41. Liu C, Fu Y, Xu C, Yang S, Li J, Wang C, Zhang L (2021) Learning a few-shot embedding model with contrastive learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 8635–8643

  42. Zhang C, Ding H, Lin G, Li R, Wang C, Shen C (2021) Meta navigator: search for a good adaptation policy for few-shot learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9435–9444

  43. Gao Z, Wu Y, Jia Y, Harandi M (2021) Curvature generation in curved spaces for few-shot learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 8691–8700

  44. Zhou Z, Qiu X, Xie J, Wu J, Zhang C (2021) Binocular mutual learning for improving few-shot classification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 8402–8411

  45. Xu W, Wang H, Tu Z (2020) Attentional constellation nets for few-shot learning. In: International conference on learning representations

Download references

Author information

Authors and Affiliations

  1. College of Systems Engineering, National University of Defense Technology, Changsha, 410000, China

    Dianqi Liu, Liang Bai & Tianyuan Yu

Authors
  1. Dianqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianyuan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DL is the first author, LB is the second author and the corresponding author, TY is the third author

Corresponding author

Correspondence to Liang Bai.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., Bai, L. & Yu, T. Generalized Few-Shot Classification with Knowledge Graph. Neural Process Lett 55, 7649–7666 (2023). https://doi.org/10.1007/s11063-023-11278-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11278-1

Keywords

Search

Navigation