Skip to main content

A Geometric View on the Role of Nonlinear Feature Maps in Few-Shot Learning

Part of the Lecture Notes in Computer Science book series (LNCS,volume 14071)


We investigate the problem of successfully learning from just a few examples of data points in a binary classification problem, and present a brief overview of some recent results on the role of nonlinear feature maps in this challenging task. Our main conclusion is that successful learning and generalisation may be expected to occur with high probability, despite the small training sample, when the nonlinear feature map induces certain fundamental geometric properties in the mapped data.


  • Few-shot learning
  • High dimensional data
  • Nonlinear feature maps

The authors are grateful for financial support by the UKRI and EPSRC (UKRI Turing AI Fellowship ARaISE EP/V025295/1). I.Y.T. is also grateful for support from the UKRI Trustworthy Autonomous Systems Node in Verifiability EP/V026801/1.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Bartlett, P., Long, P., Lugosi, G., Tsigler, A.: Benign overfitting in linear regression. Proc. Natl. Acad. Sci. 117(48), 30063–30070 (2020)

    CrossRef  MathSciNet  MATH  Google Scholar 

  2. Bastounis, A., et al.: The crisis of empirical risk in verifiable, robust and accurate learning (2023, in preparation)

    Google Scholar 

  3. Bastounis, A., Hansen, A.C., Vlačić, V.: The mathematics of adversarial attacks in AI - why deep learning is unstable despite the existence of stable neural networks (2021).

  4. Gorban, A.N., Tyukin, I.Y.: Stochastic separation theorems. Neural Netw. 94, 255–259 (2017)

    CrossRef  MATH  Google Scholar 

  5. Gorban, A., Tyukin, I., Prokhorov, D., Sofeikov, K.: Approximation with random bases: pro et contra. Inf. Sci. 364–365, 129–145 (2016)

    CrossRef  MATH  Google Scholar 

  6. Kainen, P.C., Kůrková, V.: Quasiorthogonal dimension. In: Kosheleva, O., Shary, S.P., Xiang, G., Zapatrin, R. (eds.) Beyond Traditional Probabilistic Data Processing Techniques: Interval, Fuzzy etc. Methods and Their Applications. SCI, vol. 835, pp. 615–629. Springer, Cham (2020).

    CrossRef  Google Scholar 

  7. Kainen, P.C., Kurková, V.: Quasiorthogonal dimension of Euclidean spaces. Appl. Math. Lett. 6(3), 7–10 (1993)

    Google Scholar 

  8. Ledoux, M.: The Concentration of Measure Phenomenon. No. 89, American Mathematical Soc. (2001)

    Google Scholar 

  9. Mallinar, N., Simon, J.B., Abedsoltan, A., Pandit, P., Belkin, M., Nakkiran, P.: Benign, tempered, or catastrophic: a taxonomy of overfitting. arXiv preprint arXiv:2207.06569 (2022)

  10. Smola, A., Gretton, A., Song, L., Schölkopf, B.: A Hilbert space embedding for distributions. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 13–31. Springer, Heidelberg (2007).

    CrossRef  Google Scholar 

  11. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, pp. 4077–4087 (2017)

    Google Scholar 

  12. Sutton, O.J., Gorban, A.N., Tyukin, I.Y.: Towards a mathematical understanding of learning from few examples with nonlinear feature maps (2022).

  13. Tyukin, I.Y., Gorban, A.N., Alkhudaydi, M.H., Zhou, Q.: Demystification of few-shot and one-shot learning. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2021)

    Google Scholar 

  14. Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)

    Google Scholar 

  15. Wang, Y., Yao, Q., Kwok, J.T., Ni, L.M.: Generalizing from a few examples: a survey on few-shot learning. ACM Comput. Surv. 53(3) (2020).

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Oliver J. Sutton .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sutton, O.J., Gorban, A.N., Tyukin, I.Y. (2023). A Geometric View on the Role of Nonlinear Feature Maps in Few-Shot Learning. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2023. Lecture Notes in Computer Science, vol 14071. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-38270-3

  • Online ISBN: 978-3-031-38271-0

  • eBook Packages: Computer ScienceComputer Science (R0)