Skip to main content
Book cover

CAAI International Conference on Artificial Intelligence

CICAI 2021: Artificial Intelligence pp 176–187Cite as

Boosting Few-Shot Learning with Task-Adaptive Multi-level Mixed Supervision

  • 923 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 13070)

Abstract

In this paper, we propose a novel task-adaptive few-shot learning (FSL) method called Multi-Level Mixed Supervision (MLMS), which adapts a classifier specifically for each task by mixed supervision. Our method complements the supervised training with a multi-level unsupervised loss including the instance-level certainty term, set-level divergence term, and group-level consistency term. We further modify the set-level divergence term under the unbalanced prior situation where different classes of the unlabeled set contain different numbers of samples. Besides, we propose an approximate solution of minimizing our MLMS loss which is faster than the gradient-based method. Extensive experiments on multiple FSL datasets demonstrate that our method outperforms several recent models by an obvious margin on both transductive FSL and semi-supervised FSL tasks. Codes and trained models are available at https://github.com/Wangduo428/few-shot-learning-mlms.

Keywords

  • Few-shot learning
  • Task-adaptive
  • Multi-level mixed supervision
  • Transductive FSL
  • Semi-supervised FSL
  • Unbalanced prior

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-93049-3_15
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   69.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-93049-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   89.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.

References

  1. Boudiaf, M., Masud, Z.I., Rony, J., Dolz, J., Piantanida, P., Ayed, I.B.: Transductive information maximization for few-shot learning (2020). arXiv preprint arXiv:2008.11297

  2. Chen, W., Liu, Y., Kira, Z., Wang, Y.F., Huang, J.: A closer look at few-shot classification. In: International Conference on Learning Representations, ICLR (2019)

    Google Scholar 

  3. Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A baseline for few-shot image classification. In: International Conference on Learning Representations, ICLR (2020)

    Google Scholar 

  4. Hou, R., Chang, H., Ma, B., Shan, S., Chen, X.: Cross attention network for few-shot classification. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 4005–4016 (2019)

    Google Scholar 

  5. Hu, S.X., et al.: Empirical bayes transductive meta-learning with synthetic gradients. In: International Conference on Learning Representations, ICLR (2020)

    Google Scholar 

  6. Kim, J., Kim, T., Kim, S., Yoo, C.D.: Edge-labeling graph neural network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11–20 (2019)

    Google Scholar 

  7. Kye, S.M., Lee, H.B., Kim, H., Hwang, S.J.: Transductive few-shot learning with meta-learned confidence (2020). arXiv preprint arXiv:2002.12017

  8. Li, X., et al.: Learning to self-train for semi-supervised few-shot classification. In: Advances in Neural Information Processing Systems: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, December, vol. 8, pp. 1–11 (1906)

    Google Scholar 

  9. Lichtenstein, M., Sattigeri, P., Feris, R., Giryes, R., Karlinsky, L.: TAFSSL: task-adaptive feature sub-space learning for few-shot classification. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12352, pp. 522–539. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_31

    CrossRef  Google Scholar 

  10. Liu, J., Song, L., Qin, Y.: Prototype rectification for few-shot learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 741–756. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_43

    CrossRef  Google Scholar 

  11. Liu, Y., et al.: Learning to propagate labels: transductive propagation network for few-shot learning. In: International Conference on Learning Representations, ICLR (2019)

    Google Scholar 

  12. Ma, Y., et al.: Transductive relation-propagation network for few-shot learning. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 804–810 (2020). https://doi.org/10.24963/ijcai.2020/112

  13. Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., Tian, Y.: Transductive episodic-wise adaptive metric for few-shot learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3603–3612 (2019)

    Google Scholar 

  14. Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: International Conference on Learning Representations, ICLR (2017)

    Google Scholar 

  15. Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: International Conference on Learning Representations, ICLR (2018)

    Google Scholar 

  16. Simon, C., Koniusz, P., Nock, R., Harandi, M.: Adaptive subspaces for few-shot learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  17. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 4077–4087 (2017)

    Google Scholar 

  18. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 3630–3638 (2016)

    Google Scholar 

  19. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset (2011)

    Google Scholar 

  20. Wang, Y., Chao, W., Weinberger, K.Q., van der Maaten, L.: Simpleshot: revisiting nearest-neighbor classification for few-shot learning. arXiv e-prints (2019)

    Google Scholar 

  21. Yang, L., Li, L., Zhang, Z., Zhou, X., Zhou, E., Liu, Y.: DPGN: distribution propagation graph network for few-shot learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  22. Ye, H.J., Hu, H., Zhan, D.C., Sha, F.: Few-shot learning via embedding adaptation with set-to-set functions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 8808–8817 (2020)

    Google Scholar 

  23. Yu, Z., Chen, L., Cheng, Z., Luo, J.: Transmatch: a transfer-learning scheme for semi-supervised few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12856–12864 (2020)

    Google Scholar 

  24. Ziko, I., Dolz, J., Granger, E., Ayed, I.B.: Laplacian regularized few-shot learning. In: International Conference on Machine Learning, pp. 11660–11670. PMLR (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Automation, Tsinghua University, Beijing, China

    Duo Wang, Qianxia Ma & Tao Zhang

  2. College of Engineering and Physical Sciences, Aston University, Birmingham, UK

    Ming Zhang

  3. Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China

    Tao Zhang

Authors
  1. Duo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qianxia Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Zhang .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Lu Fang

  2. Duke University, Durham, NC, USA

    Yiran Chen

  3. Shanghai Jiao Tong University, Shanghai, China

    Prof. Guangtao Zhai

  4. University of British Columbia, Vancouver, BC, Canada

    Jane Wang

  5. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Prof. Ruiping Wang

  6. Xidian University, Xi'an, China

    Weisheng Dong

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Wang, D., Ma, Q., Zhang, M., Zhang, T. (2021). Boosting Few-Shot Learning with Task-Adaptive Multi-level Mixed Supervision. In: Fang, L., Chen, Y., Zhai, G., Wang, J., Wang, R., Dong, W. (eds) Artificial Intelligence. CICAI 2021. Lecture Notes in Computer Science(), vol 13070. Springer, Cham. https://doi.org/10.1007/978-3-030-93049-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-93049-3_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-93048-6

  • Online ISBN: 978-3-030-93049-3

  • eBook Packages: Computer ScienceComputer Science (R0)