Abstract
Recently, Graph-based Few Shot Learning (FSL) methods exhibit good generalization by mining relations among the few examples with Graph Neural Networks. However, most Graph-based FSL methods consider only binary relations and ignore the multi-semantic information of the global context knowledge. We propose a framework of Multi-Semantic Hypergraph for FSL (MSH-FSL) to explore complex latent high-order multi-semantic relations among the few examples. Specifically, we first build up a novel Multi-Semantic Hypergraph by identifying associated examples with various semantic features from different receptive fields. With the constructed hypergraph, we then develop the Hyergraph Neural Network along with a novel multi-generation hypergraph message passing so as to better leverage the complex latent semantic relations among examples. Finally, after a number of generations, the hyper-node representations embedded in the learned hypergraph become more accurate for obtaining few-shot predictions. In the 5-way 1-shot task on miniImagenet dataset, the multi-semantic hypergraph outperforms the single-semantic graph by 3.1%, and with the proposed semantic-distribution message passing, the improvement can further reach 6.5%.
This work was supported by the Natural Science Foundation of China (No. 61876121).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bandyopadhyay, S., Das, K., Murty, M.N.: Line hypergraph convolution network: applying graph convolution for hypergraphs (2020)
Chen, C., Yang, X., Xu, C., Huang, X., Ma, Z.: ECKPN: explicit class knowledge propagation network for transductive few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 6596–6605 (2021)
Du, K., et al.: AGCN: augmented graph convolutional network for lifelong multi-label image recognition. arXiv preprint arXiv:2203.05534 (2022)
Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565 (2019)
Jiang, J., Wei, Y., Feng, Y., Cao, J., Gao, Y.: Dynamic hypergraph neural networks. In: IJCAI, pp. 2635–2641 (2019)
Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 10657–10665 (2019)
Lifchitz, Y., Avrithis, Y., Picard, S., Bursuc, A.: Dense classification and implanting for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 9258–9267 (2019)
Lyu, F., Wang, S., Feng, W., Ye, Z., Hu, F., Wang, S.: Multi-domain multi-task rehearsal for lifelong learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 8819–8827 (2021)
Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., Hajishirzi, H.: ESPNet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 561–580. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_34
Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P.: A simple neural attentive meta-learner. In: International Conference on Learning Representations, ICLR (2018)
Oreshkin, B.N., Rodriguez, P., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 719–729 (2018)
Ravichandran, A., Bhotika, R., Soatto, S.: Few-shot learning with embedded class models and shot-free meta training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 331–339 (2019)
Rizve, M.N., Khan, S., Khan, F.S., Shah, M.: Exploring complementary strengths of invariant and equivariant representations for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 10836–10846 (2021)
Sun, Q., Liu, Y., Chua, T.S., Schiele, B.: Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 403–412 (2019)
Tang, S., Chen, D., Bai, L., Liu, K., Ge, Y., Ouyang, W.: Mutual CRF-GNN for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2329–2339 (2021)
Xia, X., Yin, H., Yu, J., Wang, Q., Cui, L., Zhang, X.: Self-supervised hypergraph convolutional networks for session-based recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4503–4511 (2021)
Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., Talukdar, P.: HyperGCN: a new method of training graph convolutional networks on hypergraphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NeurIPS, pp. 1511–1522 (2019)
Yang, L., Li, L., Zhang, Z., Zhou, X., Zhou, E., Liu, Y.: DPGN: distribution propagation graph network for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 13390–13399 (2020)
Ye, Z., Hu, F., Lyu, F., Li, L., Huang, K.: Disentangling semantic-to-visual confusion for zero-shot learning. IEEE Trans. Multimedia (2021)
Ye, Z., Lyu, F., Li, L., Fu, Q., Ren, J., Hu, F.: SR-GAN: semantic rectifying generative adversarial network for zero-shot learning. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 85–90. IEEE (2019)
Yoon, S.W., Seo, J., Moon, J.: TapNet: neural network augmented with task-adaptive projection for few-shot learning. In: International Conference on Machine Learning, ICML, pp. 7115–7123. PMLR (2019)
Zhang, C., Cai, Y., Lin, G., Shen, C.: DeepEMD: few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp. 12203–12213 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, H. et al. (2022). Harnessing Multi-Semantic Hypergraph for Few-Shot Learning. In: Yu, S., et al. Pattern Recognition and Computer Vision. PRCV 2022. Lecture Notes in Computer Science, vol 13534. Springer, Cham. https://doi.org/10.1007/978-3-031-18907-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-18907-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18906-7
Online ISBN: 978-3-031-18907-4
eBook Packages: Computer ScienceComputer Science (R0)