TAFSSL: Task-Adaptive Feature Sub-Space Learning for Few-Shot Classification

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12352)


Recently, Few-Shot Learning (FSL), or learning from very few (typically 1 or 5) examples per novel class (unseen during training), has received a lot of attention and significant performance advances. While number of techniques have been proposed for FSL, several factors have emerged as most important for FSL performance, awarding SOTA even to the simplest of techniques. These are: the backbone architecture (bigger is better), type of pre-training (meta-training vs multi-class), quantity and diversity of the base classes (the more the merrier), and using auxiliary self-supervised tasks (a proxy for increasing the diversity). In this paper we propose TAFSSL, a simple technique for improving the few shot performance in cases when some additional unlabeled data accompanies the few-shot task. TAFSSL is built upon the intuition of reducing the feature and sampling noise inherent to few-shot tasks comprised of novel classes unseen during pre-training. Specifically, we show that on the challenging miniImageNet and tieredImageNet benchmarks, TAFSSL can improve the current state-of-the-art in both transductive and semi-supervised FSL settings by more than \(5\%\), while increasing the benefit of using unlabeled data in FSL to above \(10\%\) performance gain.


Transductive Semi-supervised Few-Shot Learning 



This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-19-C-1001. Any opinions, ndings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reect the views of DARPA. Raja Giryes is supported by ERC-StG grant no. 757497 (SPADE).

Supplementary material (5.5 mb)
Supplementary material 1 (zip 5598 KB)


  1. 1.
    Alfassy, A., et al.: LaSO: Label-Set Operations networks for multi-label few-shot learning. In: CVPR (2019)Google Scholar
  2. 2.
    Antoniou, A., Storkey, A., Edwards, H.: Data Augmentation Generative Adversarial Networks. arXiv:1711.04340 (2017).
  3. 3.
    Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C., Huang, J.B.: A closer look at few-shot classification. In: ICLR (2019)Google Scholar
  4. 4.
    Chen, Z., Fu, Y., Zhang, Y., Jiang, Y.G., Xue, X., Sigal, L.: Multi-level semantic feature augmentation for one-shot learning. IEEE Trans. Image Process. 28(9), 4594–4605 (2019). Scholar
  5. 5.
    Comon, P.: Independent component analysis, A new concept? Technical Report (1994)Google Scholar
  6. 6.
    Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learning Augmentation Policies from Data.
  7. 7.
    Devos, A., Grossglauser, M.: Subspace Networks for Few-shot Classification. Technical Report (2019)Google Scholar
  8. 8.
    Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A Baseline For Few-Shot Image Classification. Technical Report (2019)Google Scholar
  9. 9.
    Dosovitskiy, A., Springenberg, J.T., Tatarchenko, M., Brox, T.: Learning to generate chairs, tables and cars with convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 692–705 (2017). Scholar
  10. 10.
    Doveh, S., et al.: MetAdapt: Meta-Learned Task-Adaptive Architecture for Few-Shot Classification. Technical Report (2019)Google Scholar
  11. 11.
    Dvornik, N., Schmid, C., Mairal, J.: Diversity with cooperation: ensemble methods for few-shot classification. In: The IEEE International Conference on Computer Vision (ICCV) (2019).
  12. 12.
    Finn, C., Abbeel, P., Levine, S.: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400 (2017).
  13. 13.
    Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory 21(1), 32–40 (1975). Scholar
  14. 14.
    Garcia, V., Bruna, J.: Few-Shot Learning with Graph Neural Networks, pp. 1–13. arXiv:1711.04043 (2017).
  15. 15.
    Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting Few-Shot Visual Learning with Self-Supervision, 6 2019.
  16. 16.
    Guu, K., Hashimoto, T.B., Oren, Y., Liang, P.: Generating Sentences by Editing Prototypes. Arxiv:1709.08878 (2017).
  17. 17.
    Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: IEEE International Conference on Computer Vision (ICCV) (2017).
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385 (2015).
  19. 19.
    Huang, G., Liu, Z., Maaten, V.D.L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2261–2269 (2017).,
  20. 20.
    Jiang, X., Havaei, M., Varno, F., Chartrand, G.: Learning To Learn With Conditional Class Dependencies, pp. 1–11 (2019)Google Scholar
  21. 21.
    Kim, J., Kim, T., Kim, S., Yoo, C.D.: Edge-Labeling Graph Neural Network for Few-shot Learning. Technical ReportGoogle Scholar
  22. 22.
    Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-Learning with Differentiable Convex Optimization. In: CVPR (2019).
  23. 23.
    Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X.: Finding Task-Relevant Features for Few-Shot Learning by Category Traversal, vol. 1 (2019).
  24. 24.
    Li, X., et al.: Learning to Self-Train for Semi-Supervised Few-Shot Classification, 6 2019.
  25. 25.
    Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: Learning to Learn Quickly for Few-Shot Learning. arXiv:1707.09835 (2017).
  26. 26.
    Lim, S., Kim, I., Kim, T., Kim, C., Brain, K., Kim, S.: Fast AutoAugment. Technical Report (2019)Google Scholar
  27. 27.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar
  28. 28.
    Liu, Y., et al.: Learning To Propagate Labels: Transductive Propagation Networ For Few-Shot Learning (2019)Google Scholar
  29. 29.
    Lloyd, S.P., Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28, 129–137 (1982).
  30. 30.
  31. 31.
    Nakamura, A., Harada, T.: Revisiting Fine-Tuning for Few-Shot Learning. Technical ReportGoogle Scholar
  32. 32.
    Oreshkin, B.N., Rodriguez, P., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. NeurIPS, 5 2018.
  33. 33.
    Park, D., Ramanan, D.: Articulated pose estimation with tiny synthetic videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 58–66, October 2015.
  34. 34.
    Pearson, K.: On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901). Scholar
  35. 35.
    Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., Tian, Y.: Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning (2019).
  36. 36.
    Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: ICLR, pp. 1–11 (2017).
  37. 37.
    Reed, S., et al.: Few-shot autoregressive density estimation: towards learning to learn distributions, pp. 1–11 (2018). arXiv:1710.10304 (2016)
  38. 38.
    Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: ICLR, 3 2018.,
  39. 39.
    Rippel, O., Paluri, M., Dollar, P., Bourdev, L.: Metric Learning with Adaptive Density Discrimination, pp. 1–15. arXiv:1511.05939 (2015).
  40. 40.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. In: IJCV, 9 2015.
  41. 41.
    Rusu, A.A., et al.: Meta-learning with latent embedding optimization. In: ICLR, 7 2018.
  42. 42.
    Saito, K., Kim, D., Sclaroff, S., Darrell, T., Saenko, K.: Semi-supervised domain adaptation via minimax entropy. In: ICCV, 4 2019.
  43. 43.
    Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. J. Mach. Learn. Res. (Proceedings of The 33rd International Conference on Machine Learning), vol. 48, pp. 1842–1850 (2016).
  44. 44.
    Schwartz, E., Karlinsky, L., Feris, R., Giryes, R., Bronstein, A.M.: Baby steps towards few-shot learning with multiple semantics, pp. 1–11 (2019).
  45. 45.
    Schwartz, E., et al.: Delta-Encoder: an effective sample synthesis method for few-shot object recognition. NeurIPS (2018).
  46. 46.
    Simon, C., Koniusz, P., Harandi, M.: Projective sub-space networks for few-shot learning. In: ICLR 2019 OpenReview.
  47. 47.
    Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: NIPS (2017).
  48. 48.
    Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN viewpoint estimation in images using CNNs trained with rendered 3D model views.pdf. In: IEEE International Conference on Computer Vision (ICCV), pp. 2686–2694 (2015)Google Scholar
  49. 49.
    Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning.
  50. 50.
    Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. arXiv:1711.06025 (2017),
  51. 51.
    Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.:Matching networks for one shot learning. In: NIPS (2016).,
  52. 52.
    Wang, Y., Chao, W.L., Weinberger, K.Q., van der Maaten, L.: SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning, 11 2019.
  53. 53.
    Wang, Y.X., Girshick, R., Hebert, M., Hariharan, B.: Low-Shot Learning from Imaginary Data. arXiv:1801.05401 (2018).
  54. 54.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009). Scholar
  55. 55.
    Xing, C., Rostamzadeh, N., Oreshkin, B.N., Pinheiro, P.O.: Adaptive Cross-Modal Few-Shot Learning (2019).,
  56. 56.
    Yu, A., Grauman, K.: Semantic jitter: dense supervision for visual comparisons via synthetic images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5571–5580, October 2017.
  57. 57.
    Zhang, J., Zhao, C., Ni, B., Xu, M., Yang, X.: Variational few-shot learning. In: IEEE International Conference on Computer Vision (ICCV) (2019)Google Scholar
  58. 58.
    Zhou, F., Wu, B., Li, Z.: Deep Meta-Learning: Learning to Learn in the Concept Space. Technical Report, 2 2018.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.IBM Research AICambridgeUSA
  2. 2.Tel Aviv UniversityTel AvivIsrael

Personalised recommendations