Skip to main content
SpringerLink
Log in

TAFSSL: Task-Adaptive Feature Sub-Space Learning for Few-Shot Classification

  • Conference paper
  • First Online:
Book cover Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12352))

Included in the following conference series:

Abstract

Recently, Few-Shot Learning (FSL), or learning from very few (typically 1 or 5) examples per novel class (unseen during training), has received a lot of attention and significant performance advances. While number of techniques have been proposed for FSL, several factors have emerged as most important for FSL performance, awarding SOTA even to the simplest of techniques. These are: the backbone architecture (bigger is better), type of pre-training (meta-training vs multi-class), quantity and diversity of the base classes (the more the merrier), and using auxiliary self-supervised tasks (a proxy for increasing the diversity). In this paper we propose TAFSSL, a simple technique for improving the few shot performance in cases when some additional unlabeled data accompanies the few-shot task. TAFSSL is built upon the intuition of reducing the feature and sampling noise inherent to few-shot tasks comprised of novel classes unseen during pre-training. Specifically, we show that on the challenging miniImageNet and tieredImageNet benchmarks, TAFSSL can improve the current state-of-the-art in both transductive and semi-supervised FSL settings by more than \(5\%\), while increasing the benefit of using unlabeled data in FSL to above \(10\%\) performance gain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alfassy, A., et al.: LaSO: Label-Set Operations networks for multi-label few-shot learning. In: CVPR (2019)

    Google Scholar 

  2. Antoniou, A., Storkey, A., Edwards, H.: Data Augmentation Generative Adversarial Networks. arXiv:1711.04340 (2017). https://arxiv.org/pdf/1711.04340.pdf

  3. Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C., Huang, J.B.: A closer look at few-shot classification. In: ICLR (2019)

    Google Scholar 

  4. Chen, Z., Fu, Y., Zhang, Y., Jiang, Y.G., Xue, X., Sigal, L.: Multi-level semantic feature augmentation for one-shot learning. IEEE Trans. Image Process. 28(9), 4594–4605 (2019). https://doi.org/10.1109/tip.2019.2910052

    Article  MathSciNet  MATH  Google Scholar 

  5. Comon, P.: Independent component analysis, A new concept? Technical Report (1994)

    Google Scholar 

  6. Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learning Augmentation Policies from Data. https://arxiv.org/pdf/1805.09501v1.pdf

  7. Devos, A., Grossglauser, M.: Subspace Networks for Few-shot Classification. Technical Report (2019)

    Google Scholar 

  8. Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A Baseline For Few-Shot Image Classification. Technical Report (2019)

    Google Scholar 

  9. Dosovitskiy, A., Springenberg, J.T., Tatarchenko, M., Brox, T.: Learning to generate chairs, tables and cars with convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 692–705 (2017). https://doi.org/10.1109/TPAMI.2016.2567384

    Article  Google Scholar 

  10. Doveh, S., et al.: MetAdapt: Meta-Learned Task-Adaptive Architecture for Few-Shot Classification. Technical Report (2019)

    Google Scholar 

  11. Dvornik, N., Schmid, C., Mairal, J.: Diversity with cooperation: ensemble methods for few-shot classification. In: The IEEE International Conference on Computer Vision (ICCV) (2019). http://arxiv.org/abs/1903.11341

  12. Finn, C., Abbeel, P., Levine, S.: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400 (2017). http://arxiv.org/abs/1703.03400

  13. Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory 21(1), 32–40 (1975). https://doi.org/10.1109/TIT.1975.1055330

    Article  MathSciNet  MATH  Google Scholar 

  14. Garcia, V., Bruna, J.: Few-Shot Learning with Graph Neural Networks, pp. 1–13. arXiv:1711.04043 (2017). http://arxiv.org/abs/1711.04043

  15. Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting Few-Shot Visual Learning with Self-Supervision, 6 2019. http://arxiv.org/abs/1906.05186

  16. Guu, K., Hashimoto, T.B., Oren, Y., Liang, P.: Generating Sentences by Editing Prototypes. Arxiv:1709.08878 (2017). https://arxiv.org/pdf/1709.08878.pdf

  17. Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: IEEE International Conference on Computer Vision (ICCV) (2017). https://arxiv.org/pdf/1606.02819.pdf

  18. He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385 (2015). https://arxiv.org/pdf/1512.03385.pdf

  19. Huang, G., Liu, Z., Maaten, V.D.L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243, https://arxiv.org/pdf/1608.06993.pdf

  20. Jiang, X., Havaei, M., Varno, F., Chartrand, G.: Learning To Learn With Conditional Class Dependencies, pp. 1–11 (2019)

    Google Scholar 

  21. Kim, J., Kim, T., Kim, S., Yoo, C.D.: Edge-Labeling Graph Neural Network for Few-shot Learning. Technical Report

    Google Scholar 

  22. Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-Learning with Differentiable Convex Optimization. In: CVPR (2019). https://github.com/kjunelee/MetaOptNet

  23. Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X.: Finding Task-Relevant Features for Few-Shot Learning by Category Traversal, vol. 1 (2019). http://arxiv.org/abs/1905.11116

  24. Li, X., et al.: Learning to Self-Train for Semi-Supervised Few-Shot Classification, 6 2019. http://arxiv.org/abs/1906.00562

  25. Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: Learning to Learn Quickly for Few-Shot Learning. arXiv:1707.09835 (2017). http://arxiv.org/abs/1707.09835

  26. Lim, S., Kim, I., Kim, T., Kim, C., Brain, K., Kim, S.: Fast AutoAugment. Technical Report (2019)

    Google Scholar 

  27. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  28. Liu, Y., et al.: Learning To Propagate Labels: Transductive Propagation Networ For Few-Shot Learning (2019)

    Google Scholar 

  29. Lloyd, S.P., Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28, 129–137 (1982). https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1338

  30. Munkhdalai, T., Yu, H.: Meta Networks. arXiv:1703.00837 (2017). https://doi.org/10.1093/mnrasl/slx008, http://arxiv.org/abs/1703.00837

  31. Nakamura, A., Harada, T.: Revisiting Fine-Tuning for Few-Shot Learning. Technical Report

    Google Scholar 

  32. Oreshkin, B.N., Rodriguez, P., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. NeurIPS, 5 2018. http://arxiv.org/abs/1805.10123

  33. Park, D., Ramanan, D.: Articulated pose estimation with tiny synthetic videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 58–66, October 2015. https://doi.org/10.1109/CVPRW.2015.7301337

  34. Pearson, K.: On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901). https://doi.org/10.1080/14786440109462720

    Article  MATH  Google Scholar 

  35. Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., Tian, Y.: Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning (2019). http://arxiv.org/abs/1910.02224

  36. Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: ICLR, pp. 1–11 (2017). https://openreview.net/pdf?id=rJY0-Kcll

  37. Reed, S., et al.: Few-shot autoregressive density estimation: towards learning to learn distributions, pp. 1–11 (2018). arXiv:1710.10304 (2016)

  38. Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: ICLR, 3 2018. http://arxiv.org/abs/1803.00676, http://bair.berkeley.edu/blog/2017/07/18/

  39. Rippel, O., Paluri, M., Dollar, P., Bourdev, L.: Metric Learning with Adaptive Density Discrimination, pp. 1–15. arXiv:1511.05939 (2015). http://arxiv.org/abs/1511.05939

  40. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. In: IJCV, 9 2015. http://arxiv.org/abs/1409.0575

  41. Rusu, A.A., et al.: Meta-learning with latent embedding optimization. In: ICLR, 7 2018. http://arxiv.org/abs/1807.05960

  42. Saito, K., Kim, D., Sclaroff, S., Darrell, T., Saenko, K.: Semi-supervised domain adaptation via minimax entropy. In: ICCV, 4 2019. http://arxiv.org/abs/1904.06487

  43. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. J. Mach. Learn. Res. (Proceedings of The 33rd International Conference on Machine Learning), vol. 48, pp. 1842–1850 (2016). https://doi.org/10.1002/2014GB005021

  44. Schwartz, E., Karlinsky, L., Feris, R., Giryes, R., Bronstein, A.M.: Baby steps towards few-shot learning with multiple semantics, pp. 1–11 (2019). http://arxiv.org/abs/1906.01905

  45. Schwartz, E., et al.: Delta-Encoder: an effective sample synthesis method for few-shot object recognition. NeurIPS (2018). https://arxiv.org/pdf/1806.04734.pdf

  46. Simon, C., Koniusz, P., Harandi, M.: Projective sub-space networks for few-shot learning. In: ICLR 2019 OpenReview. https://openreview.net/pdf?id=rkzfuiA9F7

  47. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: NIPS (2017). http://arxiv.org/abs/1703.05175

  48. Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN viewpoint estimation in images using CNNs trained with rendered 3D model views.pdf. In: IEEE International Conference on Computer Vision (ICCV), pp. 2686–2694 (2015)

    Google Scholar 

  49. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. https://arxiv.org/pdf/1711.06025.pdf

  50. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. arXiv:1711.06025 (2017), http://arxiv.org/abs/1711.06025

  51. Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.:Matching networks for one shot learning. In: NIPS (2016).https://doi.org/10.1109/CVPR.2016.95, http://arxiv.org/abs/1606.04080

  52. Wang, Y., Chao, W.L., Weinberger, K.Q., van der Maaten, L.: SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning, 11 2019. http://arxiv.org/abs/1911.04623

  53. Wang, Y.X., Girshick, R., Hebert, M., Hariharan, B.: Low-Shot Learning from Imaginary Data. arXiv:1801.05401 (2018). http://arxiv.org/abs/1801.05401

  54. Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009). https://doi.org/10.1126/science.277.5323.215

    Article  MATH  Google Scholar 

  55. Xing, C., Rostamzadeh, N., Oreshkin, B.N., Pinheiro, P.O.: Adaptive Cross-Modal Few-Shot Learning (2019). https://arxiv.org/pdf/1902.07104.pdf, http://arxiv.org/abs/1902.07104

  56. Yu, A., Grauman, K.: Semantic jitter: dense supervision for visual comparisons via synthetic images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5571–5580, October 2017. https://doi.org/10.1109/ICCV.2017.594

  57. Zhang, J., Zhao, C., Ni, B., Xu, M., Yang, X.: Variational few-shot learning. In: IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  58. Zhou, F., Wu, B., Li, Z.: Deep Meta-Learning: Learning to Learn in the Concept Space. Technical Report, 2 2018. http://arxiv.org/abs/1802.03596

Download references

Acknowledgment

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-19-C-1001. Any opinions, ndings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reect the views of DARPA. Raja Giryes is supported by ERC-StG grant no. 757497 (SPADE).

Author information

Authors and Affiliations

  1. IBM Research AI, Cambridge, USA

    Moshe Lichtenstein, Prasanna Sattigeri, Rogerio Feris & Leonid Karlinsky

  2. Tel Aviv University, Tel Aviv, Israel

    Raja Giryes

Authors
  1. Moshe Lichtenstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prasanna Sattigeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rogerio Feris
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raja Giryes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Leonid Karlinsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonid Karlinsky .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (zip 5598 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lichtenstein, M., Sattigeri, P., Feris, R., Giryes, R., Karlinsky, L. (2020). TAFSSL: Task-Adaptive Feature Sub-Space Learning for Few-Shot Classification. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12352. Springer, Cham. https://doi.org/10.1007/978-3-030-58571-6_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58571-6_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58570-9

  • Online ISBN: 978-3-030-58571-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation