Skip to main content
SpringerLink
Log in

Learned Gaussian ProtoNet for improved cross-domain few-shot classification and generalization

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

To imitate intelligent human behaviour, computer vision introduces a fundamental task called Few-Shot learning (FSL) that carries the promise of alleviating the need for exhaustively labeled data. Using prior knowledge few-shot learning aims to learn and generalize to novel tasks containing limited examples with supervised information. Although metric-based methods demonstrated promising performance but due to the large disparity of feature distributions across domains they often fail to generalize. In this work, we propose a learned Gaussian ProtoNet model for fine-grained few-shot classification via meta-learning for both in-domain and cross-domain scenarios. Gaussian ProtoNet encoder helps to map an image into an embedding vector and Gaussian covariance matrix predicts the confidence region about individual data points. Direction and class-dependent distance metrics are adopted to estimate the distances to distinct class prototypes. Feature-wise modulated layers are embedded in the encoder to augment the feature distribution of images. The learning-to-learn approach is adopted for fine-tuning the hyper-parameters of incorporated feature-wise modulated layers for better generalization on unseen domains. Experimental results justify that our proposed model performs better than many state-of-the-art models and feature-wise modulation improves the performance under domain shifts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availibility Statement

All data generated or analysed during this study has been properly cited in this published article (see reference section). If found difficulty in finding the data links, same can be available from the corresponding author on reasonable request.

References

  1. Turing AM, & Haugelan J (1950) Computing machinery and intelligence, The Turing Test: Verbal Behavior as the Hallmark of Intelligence, pp. 29–56

  2. Khanday NY, Sofi SA (2021) Taxonomy, state-of-the-art, challenges and applications of visual understanding: a review. Comput Sci Rev 40:100374

    Article  Google Scholar 

  3. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  4. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  5. Khanday NY, Sofi SA (2021) Deep insight: Convolutional neural network and its applications for covid-19 prognosis. Biomed Signal Process Control 69:102814

    Article  Google Scholar 

  6. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning, Advances in neural information processing systems, vol. 29

  7. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning, Advances in neural information processing systems, vol. 30

  8. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks, In: International conference on machine learning, pp. 1126–1135, PMLR

  9. Ravi S, Larochelle H (2016) Optimization as a model for few-shot learning, International conference on learning representations (ICLR), 2017

  10. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1199–1208,

  11. Garcia V, Bruna J (2017) Few-shot learning with graph neural networks, arXiv preprint arXiv:1711.04043,

  12. Hochreiter S, Younger AS, Conwell PR (2001) Learning to learn using gradient descent, In: International conference on artificial neural networks, pp. 87–94, Springer

  13. Hospedales TM, Antoniou A, Micaelli P, Storkey AJ (2021) Meta-learning in neural networks: a survey, In: IEEE transactions on pattern analysis and machine intelligence

  14. Cubuk ED, Zoph B, Mane D, Vasudevan V, Le Q V (2018) Autoaugment: Learning augmentation policies from data, arXiv preprint arXiv:1805.09501,

  15. W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang, A closer look at few-shot classification, arXiv preprint arXiv:1904.04232, 2019

  16. Tseng HY, Lee HY, Huang JB, Yang MH (2020) Cross-domain few-shot classification via learned feature-wise transformation, arXiv preprint arXiv:2001.08735,

  17. Bateni P, Goyal R, Masrani V, Wood F, Sigal L (2020) Improved few-shot visual classification, In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14493–14502

  18. Fort S (2017) Gaussian prototypical networks for few-shot learning on omniglot, arXiv preprint arXiv:1708.02735,

  19. Rusu AA, Rao D, Sygnowski J, Vinyals O, Pascanu R, Osindero S, Hadsell R (2018) Meta-learning with latent embedding optimization, arXiv preprint arXiv:1807.05960

  20. Vuorio R, Sun SH, Hu H, Lim JJ (2019) Multimodal model-agnostic meta-learning via task-aware modulation. Advances in neural information processing systems, 32

  21. Rezende D, Danihelka I, Gregor K, Wierstra D et al (2016) One-shot generalization in deep generative models, In: International conference on machine learning, pp. 1521–1529, PMLR

  22. Santoro A, Bartunov S, Botvinick, M, Wierstra D, Lillicra T (2016) Meta-learning with memory-augmented neural networks, In: International conference on machine learning, pp. 1842–1850, PMLR

  23. Koch G, Zemel R, Salakhutdinov R et al (2015) Siamese neural networks for one-shot image recognition, In: ICML deep learning workshop, vol. 2, p. 0, Lille

  24. Oreshkin B, Rodríguez López P, Lacoste A (2018) Tadam: Task dependent adaptive metric for improved few-shot learning. Advances in neural information processing systems, vol. 31

  25. Lifchitz,Y, Avrithi Y, Picard S, Bursuc A (2019) Dense classification and implanting for few-shot learning, In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9258–9267

  26. Banerjee A, Merugu S, Dhillon IS, Ghosh J, Lafferty J (2005) Clustering with bregman divergences. J Mach Learn Res, vol. 6, no. 10,

  27. Bertinetto L, Henriques JF, Torr PH, Vedaldi A (2018) Meta-learning with differentiable closed-form solvers, arXiv preprint arXiv:1805.08136,

  28. Lemley J, Bazrafkan S, Corcoran P (2017) Smart augmentation learning an optimal data augmentation strategy. IEEE Access 5:5858–5869

    Article  Google Scholar 

  29. Sixt L, Wild B, Landgraf T (2018) Rendergan: Generating realistic labeled data. Front Robot AI 5:66

    Article  Google Scholar 

  30. Tran T, Pham T, Carneiro G, Palmer L, Reid I (2017) A bayesian data augmentation approach for learning deep models. Advances in neural information processing systems, vol. 30

  31. Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2018) Autoaugment: Learning augmentation policies from data, arXiv preprint arXiv:1805.09501

  32. Shankar S, Piratla V, Chakrabarti S, Chaudhuri S, Jyothi P, Sarawag S (2018) Generalizing across domains via cross-gradient training, arXiv preprint arXiv:1804.10745

  33. Volpi R, Namkoon H, Sener O, Duchi JC, Murino V, Savarese S (2018) Generalizing to unseen domains via adversarial data augmentation. Advances in neural information processing systems, vol. 31

  34. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17(1):2026–2030

    MATH  Google Scholar 

  35. Tzeng E, Hoffman J, Saenko K, Darrell T (2017)Adversarial discriminative domain adaptation, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7167–7176

  36. Hsu HK, Yao CH, Tsai Y H, . Hung WC, Tseng HY, Singh M, Yang M.H. (2020) Progressive domain adaptation for object detection, In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 749–757

  37. Hoffman J, Tzeng E, Park T, Zhu JY, P. Isola, K. Saenko, Efros A, Darrell T. (2018) Cycada: Cycle-consistent adversarial domain adaptation, In: International conference on machine learning, pp. 1989–1998, PMLR

  38. Chen YC, Lin YY, Yang MH, Huang JB (2019) Crdoco: Pixel-level domain transfer with cross-domain consistency, In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1791–1800

  39. Dong N, Xing EP (2018) Domain adaption in one-shot learning, In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 573–588, Springer

  40. Li H, Pan SJ, Wang S, Kot AC (2018) Domain generalization with adversarial feature learning, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5400–5409

  41. Li D, Yang Y, Song YZ Hospedales TM (2017) Deeper, broader and artier domain generalization, In: Proceedings of the IEEE international conference on computer vision, pp. 5542–5550

  42. Balaji Y, Sankaranarayanan S, Chellappa R (2018) Metareg: Towards domain generalization using meta-regularization. Advances in neural information processing systems, vol. 31

  43. Li Y, Yang Y, Zhou W, Hospedales T (2019) Feature-critic networks for heterogeneous domain generalization, In: International Conference on Machine Learning, pp. 3915–3924, PMLR

  44. Requeima J, Gordon J, Bronskill J, Nowozin S, Turner RE (2019) Fast and flexible multi-task classification using conditional neural adaptive processes. Advances in neural information processing systems, vol. 32

  45. Triantafillou E, Zhu T, Dumoulin V, Lamblin P, Evci U, Xu K, Goroshin R, Gelada C, Swersky K, Manzagol PA et al (2019) Meta-dataset: a dataset of datasets for learning to learn from few examples, arXiv preprint arXiv:1903.03096

  46. Lake BM, Salakhutdinov R, Tenenbaum JB (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338

    Article  MATH  Google Scholar 

  47. Ali H (2020) Uhat: Urdu handwritten text dataset

  48. Gidaris S, Komodakis N (2018) Dynamic few-shot visual learning without forgetting, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4367–4375

  49. Edwards H, Storkey A (2016) Towards a neural statistician, arXiv preprint arXiv:1606.02185

  50. Mishra N, Rohaninejad M, Chen X, Abbeel P (2017) Meta-learning with temporal convolutions, arXiv preprint arXiv:1707.03141, 2(7) 23

  51. Munkhdalai T, Yu H (2017) Meta networks, In: International conference on machine learning, pp. 2554–2563, PMLR

  52. Wang Y, Yao Q, Kwok JT, Ni LM (2020) Generalizing from a few examples: a survey on few-shot learning. ACM Comput Surv (csur) 53(3):1–34

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Technology, Srinagar, India

    Nadeem Yousuf Khanday & Shabir Ahmad Sofi

Authors
  1. Nadeem Yousuf Khanday
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shabir Ahmad Sofi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nadeem Yousuf Khanday.

Ethics declarations

Conflict of interest

We have no conflicts of interest to disclose. Also there is no any funding support from any source to mention.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khanday, N.Y., Sofi, S.A. Learned Gaussian ProtoNet for improved cross-domain few-shot classification and generalization. Neural Comput & Applic 35, 3435–3448 (2023). https://doi.org/10.1007/s00521-022-07897-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07897-9

Keywords

Search

Navigation