Skip to main content
SpringerLink
Log in

Generalized few-shot learning under large scope by using episode-wise regularizing imprinting

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Few-shot learning explores machine learning tasks under input scarce conditions. In the past few years, meta-learning has demonstrated some advantages, but most of the meta-learning-based methods proposed in this field can only be applied to the restricted version of the problem, namely, the small-scope novel-only classification. It faces serious challenges when extending the problem from two practical direction. Firstly, the meta-learning conducted only improves sensitivity on a few novel target classes but do not maintain accuracy on all-class-oriented situation or more complicated visual learning tasks. Secondly, meta-learning performs poorly when recognizing objects from a large scope either based-classes are involved or not. In this paper, we focus on metric-based meta-learning. A key characteristic of this branch of models is its training stage relies on support-set to generate classifier on-the-fly in each iteration, which in turn limited their application on more complicated tasks. To overcome these limitations, we introduce a method that accomplishes few-shot oriented learning by iteratively using a traditional training routine and a parameter imprinting routine. Our approach closes the gap between small-scope and large-scope few-shot classifier by boosting the performances of previous approaches under large-scope settings. This is the first approach that achieves the effect of meta-learning by using traditional learning routings. The proposed approach is comparatively evaluated with a number of recent approaches on popular few-shot classification benchmarks and demonstrates better performance consistently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available in the ImageNet and CUB repository: https://www.image-net.org/, http://www.vision.caltech.edu/visipedia/CUB-200-2011.html.

Notes

  1. Meta-learning performance on GFSL settings has not been reported in previous publications. The result in Fig.1 is obtained by using naive approach introduced in Sect. 4.2.

  2. Works such as [5, 14] which apply meta-learning based on fixed feature extractor are discussed in terms of non-meta-learning approaches in this paper (Section 2.1).

References

  1. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. pp. 1097–1105 (2012)

  2. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  3. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  4. Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3018–3027 (2017)

  5. Wang, Y.-X., Girshick, R., Hebert, M., Hariharan, B.: Low-shot learning from imaginary data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7278–7286 (2018)

  6. Zhou, B., Cui, Q., Wei, X.-S., Chen, Z.-M.: Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In: CVPR, pp. 9719–9728 (2020)

  7. Van Horn, G., Perona, P.: The devil is in the tails: Fine-grained classification in the wild. arXiv preprint arXiv:1709.01450 (2017)

  8. Nguyen, V.N., Jenssen, R., Roverso, D.: Ls-net: fast single-shot line-segment detector. Mach. Vis. Appl. 32(1), 12 (2021)

    Article  Google Scholar 

  9. Kaur, T., Gandhi, T.K.: Deep convolutional neural networks with transfer learning for automated brain image classification. Mach. Vis. Appl. 31(3), 20 (2020)

    Article  Google Scholar 

  10. Vo, D.M., Sugimoto, A.: Two-stream fcns to balance content and style for style transfer. Mach. Vis. Appl. 31(5), 37 (2020)

    Article  Google Scholar 

  11. Garcia, V., Estrach, J.B.: Few-shot learning with graph neural networks. In: ICLR (2018)

  12. Chen, Z., Fu, Y., Wang, Y.-X., Ma, L., Liu, W., Hebert, M.: Image deformation meta-networks for one-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8680–8689 (2019)

  13. Gidaris, S., Komodakis, N.: Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4367–4375 (2018)

  14. Gidaris, S., Komodakis, N.: Generating classification weights with gnn denoising autoencoders for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 21–30 (2019)

  15. Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X.: Finding task-relevant features for few-shot learning by category traversal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–10 (2019)

  16. Sun, N., Yang, P.: T2l: Trans-transfer learning for few-shot fine-grained visual categorization with extended adaptation. Knowl. Based Syst. 264, 110329 (2023)

    Article  Google Scholar 

  17. Lifchitz, Y., Avrithis, Y., Picard, S., Bursuc, A.: Dense classification and implanting for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9258–9267 (2019)

  18. Oreshkin, B., López, P.R., Lacoste, A.: Tadam: Task dependent adaptive metric for improved few-shot learning. Adv. Neural Inf. Process. Syst. pp. 721–731 (2018)

  19. Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. IEEE Trans. Patt. Anal. Mach. Intell. 28(4), 594–611 (2006)

    Article  Google Scholar 

  20. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. Adv. Neural Inf. Process. Syst. pp. 3630–3638 (2016)

  21. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, pp. 4077–4087 (2017)

  22. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1126–1135 (2017). JMLR. org

  23. Chen, W.-Y., Liu, Y.-C., Kira, Z., Wang, Y.-C.F., Huang, J.-B.: A closer look at few-shot classification. ICLR (2019)

  24. Ye, H., Hu, H., Zhan, D., Sha, F.: Learning embedding adaptation for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8680–8689 (2020)

  25. Bronskill, J., Massiceti, D., Patacchiola, M., Hofmann, K., Nowozin, S., Turner, R.E.: Memory efficient meta-learning with large images. CoRR arXiv:2107.01105 (2021)

  26. Li, A., Luo, T., Lu, Z., Xiang, T., Wang, L.: Large-scale few-shot learning: Knowledge transfer with class hierarchy. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7212–7220 (2019)

  27. Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A baseline for few-shot image classification. In: ICLR (2020)

  28. Cao, T., Law, M., Fidler, S.: A theoretical analysis of the number of shots in few-shot learning. ICLR (2020)

  29. Niu, S., Liu, Y., Wang, J., Song, H.: A decade survey of transfer learning. IEEE Trans. Artif. Intell. 1(2), 151–166 (2020)

    Article  Google Scholar 

  30. Tseng, H.-Y., Lee, H.-Y., Huang, J.-B., Yang, M.-H.: Cross-domain few-shot classification via learned feature-wise transformation. In: ICLR (2020)

  31. Qi, H., Brown, M., Lowe, D.G.: Low-shot learning with imprinted weights. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5822–5830 (2018)

  32. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208 (2018)

  33. Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 539–546 (2005). IEEE

  34. Zhang, C., Cai, Y., Lin, G., Shen, C.: Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: CVPR, pp. 12203–12213 (2020)

  35. Li, A., Luo, T., Lu, Z., Xiang, T., Wang, L.: Large-scale few-shot learning: Knowledge transfer with class hierarchy. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7212–7220 (2019)

  36. Nichol, A., Schulman, J.: Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.029992 (2018)

  37. Sun, Q., Liu, Y., Chua, T.-S., Schiele, B.: Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 403–412 (2019)

  38. Qiao, S., Liu, C., Shen, W., Yuille, A.L.: Few-shot image recognition by predicting parameters from activations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7229–7238 (2018)

  39. Li, A., Luo, T., Xiang, T., Huang, W., Wang, L.: Few-shot learning with global class representations. In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pp. 9714–9723. IEEE (2019)

  40. Shi, X., Salewski, L., Schiegg, M., Welling, M.: Relational generalized few-shot learning. In: 31st British Machine Vision Conference 2020, BMVC 2020, Virtual Event, UK, September 7-10, 2020. BMVA Press (2020)

  41. Kukleva, A., Kuehne, H., Schiele, B.: Generalized and incremental few-shot learning by explicit learning and calibration without forgetting. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pp. 9000–9009. IEEE (2021)

  42. Liu, Y., Lee, J., Park, M., Kim, S., Yang, E., Hwang, S.J., Yang, Y.: Learning to propagate labels: Transductive propagation network for few-shot learning. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 (2019)

  43. Ren, M., Triantafillou, E., Ravi, S., Snell, J., Swersky, K., Tenenbaum, J.B., Larochelle, H., Zemel, R.S.: Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 (2018)

  44. Wang, Y., Xu, C., Liu, C., Zhang, L., Fu, Y.: Instance credibility inference for few-shot learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 12833–12842 (2020)

  45. Lichtenstein, M., Sattigeri, P., Feris, R., Giryes, R., Karlinsky, L.: TAFSSL: task-adaptive feature sub-space learning for few-shot classification. In: ECCV (2020). https://doi.org/10.1007/978-3-030-58571-6_31

  46. Wang, Y., Zhang, L., Yao, Y., Fu, Y.: How to trust unlabeled data? instance credibility inference for few-shot learning. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 6240–6253 (2022)

    Article  Google Scholar 

  47. Ravichandran, A., Bhotika, R., Soatto, S.: Few-shot learning with embedded class models and shot-free meta training. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 331–339 (2019)

  48. Seo, J., Jung, H., Lee, S.: Self-augmentation: generalizing deep networks to unseen classes for few-shot learning. Neural Netw. 138, 140–149 (2021)

    Article  MATH  Google Scholar 

  49. Xian, Y., Sharma, S., Schiele, B., Akata, Z.: f-vaegan-d2: A feature generating framework for any-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10275–10284 (2019)

  50. Tokmakov, P., Wang, Y.-X., Hebert, M.: Learning compositional representations for few-shot recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6372–6381 (2019)

  51. Schwartz, E., Karlinsky, L., Feris, R., Giryes, R., Bronstein, A.M.: Baby steps towards few-shot learning with multiple semantics. arXiv preprint arXiv:1906.01905 (2019)

  52. Schaul, T., Schmidhuber, J.: Metalearning. Scholarpedia 5(6), 4650 (2010)

    Article  Google Scholar 

  53. Lilian, W.: Meta-Learning: Learning to Learn Fast. https://lilianweng.github.io/posts/2018-11-30-meta-learning/

  54. Passalis, N., Iosifidis, A., Gabbouj, M., Tefas, A.: Hypersphere-based weight imprinting for few-shot learning on embedded devices. IEEE Trans. Neural Netw. Learn. Syst. 32(2), 925–930 (2021)

    Article  Google Scholar 

  55. Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On large-batch training for deep learning: Generalization gap and sharp minima. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings (2017)

  56. LeCun, Y., Bottou, L., Orr, G.B., Müller, K.: Efficient backprop. In: Neural Networks: Tricks of the Trade - Second Edition. Lecture Notes in Computer Science, vol. 7700, pp. 9–48. Springer (2012)

  57. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML (2015)

  58. Wang, P., Cheng, J., Hao, F., Wang, L., Feng, W.: Embedded adaptive cross-modulation neural network for few-shot learning. Neural Comput. Appl. 32(10), 5505–5515 (2020)

    Article  Google Scholar 

  59. Goodfellow, I.J., Bengio, Y., Courville, A.C.: Deep learning (2016)

  60. Arnold, S.M.R., Iqbal, S., Sha, F.: When MAML can adapt fast and how to assist when it cannot. In: Banerjee, A., Fukumizu, K. (eds.) The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event

  61. Hou, M., Sato, I.: A closer look at prototype classifier for few-shot image classification. CoRR arXiv:2110.05076 (2021)

  62. Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10657–10665 (2019)

  63. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset. Technical report (2011)

  64. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. In: Proceedings of the IEEE International Conference on Computer Vision (2015)

  65. Ren, M., Liao, R., Fetaya, E., Zemel, R.S.: Incremental few-shot learning with attention attractor networks. In: Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 5276–5286 (2019)

  66. Rusu, A.A., Rao, D., Sygnowski, J., Vinyals, O., Pascanu, R., Osindero, S., Hadsell, R.: Meta-learning with latent embedding optimization. ICLR (2019)

  67. Schonfeld, E., Ebrahimi, S., Sinha, S., Darrell, T., Akata, Z.: Generalized zero-and few-shot learning via aligned variational autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8247–8255 (2019)

  68. Triantafillou, E., Zhu, T., Dumoulin, V., Lamblin, P., Evci, U., Xu, K., Goroshin, R., Gelada, C., Swersky, K., Manzagol, P., Larochelle, H.: Meta-dataset: A dataset of datasets for learning to learn from few examples. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 (2020)

  69. Dvornik, N., Schmid, C., Mairal, J.: Selecting relevant features from a multi-domain representation for few-shot classification. In: Computer Vision-ECCV 2020-16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part X, vol. 12355, pp. 769–786 (2020)

  70. Sun, X., Xv, H., Dong, J., Li, Q., Chen, C.: Few-shot learning for domain-specific fine-grained image classification. IEEE Trans. Ind. Electron. (2019)

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Kunming, 650500, Yunnan, China

    Nan Sun & Yunxuan Tang

  2. School of Information, Yunnan University of Finance and Economics, Kunming, 650000, Yunnan, China

    Kun Wang

Authors
  1. Nan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunxuan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kun Wang.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, N., Tang, Y. & Wang, K. Generalized few-shot learning under large scope by using episode-wise regularizing imprinting. Machine Vision and Applications 34, 112 (2023). https://doi.org/10.1007/s00138-023-01445-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01445-8

Keywords

Search

Navigation