Advertisement

TAFSSL: Task-Adaptive Feature Sub-Space Learning for Few-Shot Classification

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12352)

Abstract

Recently, Few-Shot Learning (FSL), or learning from very few (typically 1 or 5) examples per novel class (unseen during training), has received a lot of attention and significant performance advances. While number of techniques have been proposed for FSL, several factors have emerged as most important for FSL performance, awarding SOTA even to the simplest of techniques. These are: the backbone architecture (bigger is better), type of pre-training (meta-training vs multi-class), quantity and diversity of the base classes (the more the merrier), and using auxiliary self-supervised tasks (a proxy for increasing the diversity). In this paper we propose TAFSSL, a simple technique for improving the few shot performance in cases when some additional unlabeled data accompanies the few-shot task. TAFSSL is built upon the intuition of reducing the feature and sampling noise inherent to few-shot tasks comprised of novel classes unseen during pre-training. Specifically, we show that on the challenging miniImageNet and tieredImageNet benchmarks, TAFSSL can improve the current state-of-the-art in both transductive and semi-supervised FSL settings by more than \(5\%\), while increasing the benefit of using unlabeled data in FSL to above \(10\%\) performance gain.

Keywords

Transductive Semi-supervised Few-Shot Learning 

Notes

Acknowledgment

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-19-C-1001. Any opinions, ndings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reect the views of DARPA. Raja Giryes is supported by ERC-StG grant no. 757497 (SPADE).

Supplementary material

504444_1_En_31_MOESM1_ESM.zip (5.5 mb)
Supplementary material 1 (zip 5598 KB)

References

  1. 1.
    Alfassy, A., et al.: LaSO: Label-Set Operations networks for multi-label few-shot learning. In: CVPR (2019)Google Scholar
  2. 2.
    Antoniou, A., Storkey, A., Edwards, H.: Data Augmentation Generative Adversarial Networks. arXiv:1711.04340 (2017). https://arxiv.org/pdf/1711.04340.pdf
  3. 3.
    Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C., Huang, J.B.: A closer look at few-shot classification. In: ICLR (2019)Google Scholar
  4. 4.
    Chen, Z., Fu, Y., Zhang, Y., Jiang, Y.G., Xue, X., Sigal, L.: Multi-level semantic feature augmentation for one-shot learning. IEEE Trans. Image Process. 28(9), 4594–4605 (2019).  https://doi.org/10.1109/tip.2019.2910052MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Comon, P.: Independent component analysis, A new concept? Technical Report (1994)Google Scholar
  6. 6.
    Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learning Augmentation Policies from Data. https://arxiv.org/pdf/1805.09501v1.pdf
  7. 7.
    Devos, A., Grossglauser, M.: Subspace Networks for Few-shot Classification. Technical Report (2019)Google Scholar
  8. 8.
    Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A Baseline For Few-Shot Image Classification. Technical Report (2019)Google Scholar
  9. 9.
    Dosovitskiy, A., Springenberg, J.T., Tatarchenko, M., Brox, T.: Learning to generate chairs, tables and cars with convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 692–705 (2017).  https://doi.org/10.1109/TPAMI.2016.2567384CrossRefGoogle Scholar
  10. 10.
    Doveh, S., et al.: MetAdapt: Meta-Learned Task-Adaptive Architecture for Few-Shot Classification. Technical Report (2019)Google Scholar
  11. 11.
    Dvornik, N., Schmid, C., Mairal, J.: Diversity with cooperation: ensemble methods for few-shot classification. In: The IEEE International Conference on Computer Vision (ICCV) (2019). http://arxiv.org/abs/1903.11341
  12. 12.
    Finn, C., Abbeel, P., Levine, S.: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400 (2017). http://arxiv.org/abs/1703.03400
  13. 13.
    Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory 21(1), 32–40 (1975).  https://doi.org/10.1109/TIT.1975.1055330MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Garcia, V., Bruna, J.: Few-Shot Learning with Graph Neural Networks, pp. 1–13. arXiv:1711.04043 (2017). http://arxiv.org/abs/1711.04043
  15. 15.
    Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting Few-Shot Visual Learning with Self-Supervision, 6 2019. http://arxiv.org/abs/1906.05186
  16. 16.
    Guu, K., Hashimoto, T.B., Oren, Y., Liang, P.: Generating Sentences by Editing Prototypes. Arxiv:1709.08878 (2017). https://arxiv.org/pdf/1709.08878.pdf
  17. 17.
    Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: IEEE International Conference on Computer Vision (ICCV) (2017). https://arxiv.org/pdf/1606.02819.pdf
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385 (2015). https://arxiv.org/pdf/1512.03385.pdf
  19. 19.
    Huang, G., Liu, Z., Maaten, V.D.L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2261–2269 (2017).  https://doi.org/10.1109/CVPR.2017.243, https://arxiv.org/pdf/1608.06993.pdf
  20. 20.
    Jiang, X., Havaei, M., Varno, F., Chartrand, G.: Learning To Learn With Conditional Class Dependencies, pp. 1–11 (2019)Google Scholar
  21. 21.
    Kim, J., Kim, T., Kim, S., Yoo, C.D.: Edge-Labeling Graph Neural Network for Few-shot Learning. Technical ReportGoogle Scholar
  22. 22.
    Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-Learning with Differentiable Convex Optimization. In: CVPR (2019). https://github.com/kjunelee/MetaOptNet
  23. 23.
    Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X.: Finding Task-Relevant Features for Few-Shot Learning by Category Traversal, vol. 1 (2019). http://arxiv.org/abs/1905.11116
  24. 24.
    Li, X., et al.: Learning to Self-Train for Semi-Supervised Few-Shot Classification, 6 2019. http://arxiv.org/abs/1906.00562
  25. 25.
    Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: Learning to Learn Quickly for Few-Shot Learning. arXiv:1707.09835 (2017). http://arxiv.org/abs/1707.09835
  26. 26.
    Lim, S., Kim, I., Kim, T., Kim, C., Brain, K., Kim, S.: Fast AutoAugment. Technical Report (2019)Google Scholar
  27. 27.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10602-1_48CrossRefGoogle Scholar
  28. 28.
    Liu, Y., et al.: Learning To Propagate Labels: Transductive Propagation Networ For Few-Shot Learning (2019)Google Scholar
  29. 29.
    Lloyd, S.P., Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28, 129–137 (1982). https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1338
  30. 30.
  31. 31.
    Nakamura, A., Harada, T.: Revisiting Fine-Tuning for Few-Shot Learning. Technical ReportGoogle Scholar
  32. 32.
    Oreshkin, B.N., Rodriguez, P., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. NeurIPS, 5 2018. http://arxiv.org/abs/1805.10123
  33. 33.
    Park, D., Ramanan, D.: Articulated pose estimation with tiny synthetic videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 58–66, October 2015.  https://doi.org/10.1109/CVPRW.2015.7301337
  34. 34.
    Pearson, K.: On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901).  https://doi.org/10.1080/14786440109462720CrossRefzbMATHGoogle Scholar
  35. 35.
    Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., Tian, Y.: Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning (2019). http://arxiv.org/abs/1910.02224
  36. 36.
    Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: ICLR, pp. 1–11 (2017). https://openreview.net/pdf?id=rJY0-Kcll
  37. 37.
    Reed, S., et al.: Few-shot autoregressive density estimation: towards learning to learn distributions, pp. 1–11 (2018). arXiv:1710.10304 (2016)
  38. 38.
    Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: ICLR, 3 2018. http://arxiv.org/abs/1803.00676, http://bair.berkeley.edu/blog/2017/07/18/
  39. 39.
    Rippel, O., Paluri, M., Dollar, P., Bourdev, L.: Metric Learning with Adaptive Density Discrimination, pp. 1–15. arXiv:1511.05939 (2015). http://arxiv.org/abs/1511.05939
  40. 40.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. In: IJCV, 9 2015. http://arxiv.org/abs/1409.0575
  41. 41.
    Rusu, A.A., et al.: Meta-learning with latent embedding optimization. In: ICLR, 7 2018. http://arxiv.org/abs/1807.05960
  42. 42.
    Saito, K., Kim, D., Sclaroff, S., Darrell, T., Saenko, K.: Semi-supervised domain adaptation via minimax entropy. In: ICCV, 4 2019. http://arxiv.org/abs/1904.06487
  43. 43.
    Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. J. Mach. Learn. Res. (Proceedings of The 33rd International Conference on Machine Learning), vol. 48, pp. 1842–1850 (2016).  https://doi.org/10.1002/2014GB005021
  44. 44.
    Schwartz, E., Karlinsky, L., Feris, R., Giryes, R., Bronstein, A.M.: Baby steps towards few-shot learning with multiple semantics, pp. 1–11 (2019). http://arxiv.org/abs/1906.01905
  45. 45.
    Schwartz, E., et al.: Delta-Encoder: an effective sample synthesis method for few-shot object recognition. NeurIPS (2018). https://arxiv.org/pdf/1806.04734.pdf
  46. 46.
    Simon, C., Koniusz, P., Harandi, M.: Projective sub-space networks for few-shot learning. In: ICLR 2019 OpenReview. https://openreview.net/pdf?id=rkzfuiA9F7
  47. 47.
    Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: NIPS (2017). http://arxiv.org/abs/1703.05175
  48. 48.
    Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN viewpoint estimation in images using CNNs trained with rendered 3D model views.pdf. In: IEEE International Conference on Computer Vision (ICCV), pp. 2686–2694 (2015)Google Scholar
  49. 49.
    Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. https://arxiv.org/pdf/1711.06025.pdf
  50. 50.
    Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. arXiv:1711.06025 (2017), http://arxiv.org/abs/1711.06025
  51. 51.
    Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.:Matching networks for one shot learning. In: NIPS (2016). https://doi.org/10.1109/CVPR.2016.95, http://arxiv.org/abs/1606.04080
  52. 52.
    Wang, Y., Chao, W.L., Weinberger, K.Q., van der Maaten, L.: SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning, 11 2019. http://arxiv.org/abs/1911.04623
  53. 53.
    Wang, Y.X., Girshick, R., Hebert, M., Hariharan, B.: Low-Shot Learning from Imaginary Data. arXiv:1801.05401 (2018). http://arxiv.org/abs/1801.05401
  54. 54.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009).  https://doi.org/10.1126/science.277.5323.215CrossRefzbMATHGoogle Scholar
  55. 55.
    Xing, C., Rostamzadeh, N., Oreshkin, B.N., Pinheiro, P.O.: Adaptive Cross-Modal Few-Shot Learning (2019). https://arxiv.org/pdf/1902.07104.pdf, http://arxiv.org/abs/1902.07104
  56. 56.
    Yu, A., Grauman, K.: Semantic jitter: dense supervision for visual comparisons via synthetic images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5571–5580, October 2017.  https://doi.org/10.1109/ICCV.2017.594
  57. 57.
    Zhang, J., Zhao, C., Ni, B., Xu, M., Yang, X.: Variational few-shot learning. In: IEEE International Conference on Computer Vision (ICCV) (2019)Google Scholar
  58. 58.
    Zhou, F., Wu, B., Li, Z.: Deep Meta-Learning: Learning to Learn in the Concept Space. Technical Report, 2 2018. http://arxiv.org/abs/1802.03596

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.IBM Research AICambridgeUSA
  2. 2.Tel Aviv UniversityTel AvivIsrael

Personalised recommendations