Abstract
Few-shot learning aims to train a classifier which can recognize a new class from a few examples like a human. Recently, some works have leveraged auxiliary information in few-shot learning, such as textual data, unlabeled visual data. But these data are positive data, they are close related to the training data. Different from such experimental settings, people can also get knowledge from negative data to better recognize a new class. Inspired by this, we exposure a few unlabeled outliers in each few-shot learning tasks to assist the learning of the classifier. To the best of our knowledge, we are the first ones who propose to utilize outliers to improve few-shot learning. We propose a novel method based on meta-learning paradigms to utilize unlabeled outliers. We not only utilize unlabeled outliers to optimize the meta-embedding network but also adaptively leverage them to enhance the class prototypes. Experiments show that our outlier exposure network can improve few-shot learning performance with a few unlabeled outliers exposure.
H. Wang and J. Lian–Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML, vol. 70, pp. 1126–1135 (2017)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS, vol. 15, pp. 315–323 (2011)
Hendrycks, D., Mazeika, M., Dietterich, T.G.: Deep anomaly detection with outlier exposure. In: 7th International Conference on Learning Representations, ICLR (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR (2015)
Kohonen, T.: Learning vector quantization. In: Self-Organizing Maps, pp. 175–189 (1995)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835 (2017)
Oreshkin, B., López, P.R., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 721–731 (2018)
Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: 5th International Conference on Learning Representations, ICLR (2017)
Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: 6th International Conference on Learning Representations, ICLR (2018)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Rusu, A.A., et al: Meta-learning with latent embedding optimization. In: 7th International Conference on Learning Representations, ICLR (2019)
Simon, C., Koniusz, P., Nock, R., Harandi, M.: Adaptive subspaces for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 4136–4145 (2020)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 4077–4087 (2017)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1199–1208 (2018)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)
Xing, C., Rostamzadeh, N., Oreshkin, B.N., Pinheiro, P.O.: Adaptive cross-modal few-shot learning. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 4848–4858. Curran Associates, Inc. (2019)
Ye, H.J., Hu, H., Zhan, D.C., Sha, F.: Few-shot learning via embedding adaptation with set-to-set functions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 8808–8817 (2020)
Zhang, R., Che, T., Ghahramani, Z., Bengio, Y., Song, Y.: MetaGAN: an adversarial approach to few-shot learning. In: Advances in Neural Information Processing Systems, NeurIPS, pp. 2371–2380 (2018)
Acknowledgments
This work was in part supported by the National Key Research and Development Program of China (Grant No. 2017YFB1402203), the National Natural Science Foundation of China (Grant No. 61702386). the Defense Industrial Technology Development Program (Grant No. JCKY2018110C165), Major Technological Innovation Projects in Hubei Province (Grant No. 2019AAA024).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, H., Lian, J., Xiong, S. (2021). Few-Shot Learning with Unlabeled Outlier Exposure. In: Lokoč, J., et al. MultiMedia Modeling. MMM 2021. Lecture Notes in Computer Science(), vol 12572. Springer, Cham. https://doi.org/10.1007/978-3-030-67832-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-67832-6_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67831-9
Online ISBN: 978-3-030-67832-6
eBook Packages: Computer ScienceComputer Science (R0)