Abstract
Popular few-shot Meta-learning (ML) methods presume that a task’s support and query data are drawn from a common distribution. Recently, Bennequin et al. [4] relaxed this assumption to propose a few-shot setting where the support and query distributions differ, with disjoint yet related meta-train and meta-test support-query shifts (SQS). We relax this assumption further to a more pragmatic SQS setting (SQS+) where the meta-test SQS is anonymous and need not be related to the meta-train SQS. The state-of-the-art solution to address SQS is transductive, requiring unlabelled meta-test query data to bridge the support and query distribution gap. In contrast, we propose a theoretically grounded inductive solution - Adversarial Query Projection (AQP) for addressing SQS+ and SQS that is applicable when unlabeled meta-test query instances are unavailable. AQP can be easily integrated into the popular ML frameworks. Exhaustive empirical investigations on benchmark datasets and their extensions, different ML approaches, and architectures establish AQP’s efficacy in handling SQS+ and SQS.
A. Aimen and B. Ladrecha—Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aimen, A., Sidheekh, S., Ladrecha, B., Krishnan, N.C.: Task attended meta-learning for few-shot learning. In: Fifth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems (2021)
Antoniou, A., Storkey, A.J.: Learning to learn by self-critique. Adv. Neural Inf. Process. Syst. 32, 1–11 (2019)
Ben-David, S., Blitzer, J., Crammer, K., Pereira, F.: Analysis of representations for domain adaptation. Adv. Neural Inf. Process. Syst. 19, 1–8 (2006)
Bennequin, E., Bouvier, V., Tami, M., Toubhans, A., Hudelot, C.: Bridging few-shot learning and adaptation: new challenges of support-query shift. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 554–569 (2021)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, Heidelberg (2013). https://doi.org/10.1007/978-1-4612-1394-9
Boudiaf, M., Ziko, I., Rony, J., Dolz, J., Piantanida, P., Ben Ayed, I.: Information maximization for few-shot learning. Adv. Neural Inf. Process. Syst. 33, 2445–2457 (2020)
Chen, W., Liu, Y., Kira, Z., Wang, Y.F., Huang, J.: A closer look at few-shot classification. In: International Conference on Learning Representations (2019)
Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A baseline for few-shot image classification. In: International Conference on Learning Representations (2020)
Du, Y., Zhen, X., Shao, L., Snoek, C.G.: Metanorm: learning to normalize few-shot batches across domains. In: International Conference on Learning Representations (2020)
Finn, C., Xu, K., Levine, S.: Probabilistic model-agnostic meta-learning. Adv. Neural Inf. Process. Syst. 31, 1–12 (2018)
Flamary, R., Courty, N., Tuia, D., Rakotomamonjy, A.: Optimal transport for domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. 1, 1–40 (2016)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International Conference on Machine Learning, pp. 1180–1189 (2015)
Goldblum, M., Fowl, L., Goldstein, T.: Adversarially robust few-shot learning: a meta-learning approach. Adv. Neural Inf. Process. Syst. 33, 17886–17895 (2020)
Goldblum, M., Reich, S., Fowl, L., Ni, R., Cherepanova, V., Goldstein, T.: Unraveling meta-learning: understanding feature representations for few-shot tasks. In: International Conference on Machine Learning, pp. 3607–3616 (2020)
Guo, Y., et al.: A broader study of cross-domain few-shot learning. In: European Conference on Computer Vision, pp. 124–141 (2020)
Laenen, S., Bertinetto, L.: On episodes, prototypical networks, and few-shot learning. Adv. Neural Inf. Process. Syst. 34, 24581–24592 (2021)
Li, Y., Yang, Y., Zhou, W., Hospedales, T.M.: Feature-critic networks for heterogeneous domain generalization. In: International Conference on Machine Learning, pp. 3915–3924 (2019)
Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J.E., Stoica, I.: Tune: a research platform for distributed model selection and training. arXiv preprint \({\rm arXiv}\):1807.05118 (2018)
Liu, B., Zhao, Z., Li, Z., Jiang, J., Guo, Y., Ye, J.: Feature transformation ensemble model with batch spectral regularization for cross-domain few-shot classification. arXiv preprint \({\rm arXiv}\):2005.08463 (2020)
Liu, Y., et al.: Learning to propagate labels: transductive propagation network for few-shot learning. In: International Conference on Learning Representations (2019)
Peyré, G., Cuturi, M., et al.: Computational optimal transport: with applications to data science. Found. Trends® Mach. Learn. 11, 355–607 (2019)
Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: International Conference on Learning Representations (2018)
Sinha, A., Namkoong, H., Duchi, J.C.: Certifying some distributional robustness with principled adversarial training. In: International Conference on Learning Representations (2018)
Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. Adv. Neural Inf. Process. Syst. 30, 1–11 (2017)
Sun, J., Lapuschkin, S., Samek, W., Zhao, Y., Cheung, N., Binder, A.: Explanation-guided training for cross-domain few-shot classification. In: International Conference on Pattern Recognition, pp. 7609–7616 (2020)
Tseng, H., Lee, H., Huang, J., Yang, M.: Cross-domain few-shot classification via learned feature-wise transformation. In: International Conference on Learning Representations (2020)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. Adv. Neural Inf. Process. Syst. 29, 1–9 (2016)
Volpi, R., Namkoong, H., Sener, O., Duchi, J.C., Murino, V., Savarese, S.: Generalizing to unseen domains via adversarial data augmentation. Adv. Neural Inf. Process. Syst. 31, 5339–5349 (2018)
Wang, H., Deng, Z.: Cross-domain few-shot classification via adversarial task augmentation. In: International Joint Conference on Artificial Intelligence, pp. 1075–1081 (2021)
Yeh, J.F., et al.: Large margin mechanism and pseudo query set on cross-domain few-shot learning. arXiv preprint \({\rm arXiv}\):2005.09218 (2020)
Acknowledgements
The resources provided by ‘PARAM Shivay Facility’ under the National Supercomputing Mission, Government of India at the Indian Institute of Technology, Varanasi are gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Aimen, A., Ladrecha, B., Krishnan, N.C. (2023). Adversarial Projections to Tackle Support-Query Shifts in Few-Shot Meta-Learning. In: Amini, MR., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2022. Lecture Notes in Computer Science(), vol 13715. Springer, Cham. https://doi.org/10.1007/978-3-031-26409-2_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-26409-2_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26408-5
Online ISBN: 978-3-031-26409-2
eBook Packages: Computer ScienceComputer Science (R0)
