Abstract
In the case of insufficient labeled data, active learning performs very well by actively querying more valuable samples. Traditional active learning models usually use the target model to select samples in the image classification tasks. The target model is not only used to find out the crucial unlabeled samples with massive information but also used to perform prediction. This method does not use the information of the remaining unlabeled samples, and the efficiency will also be restricted when using a single model to solve these two tasks simultaneously. We put forward a self-supervised active learning framework to help query samples in this paper. First, we introduce contrastive learning to construct a new feature space. Second, a special active sampling strategy is proposed based on the distance between unlabeled samples and the center of the category clusters. We update the clusters at each iteration to ensure that the examples most needed can be selected for labeling. Experiments show the success of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Settles, B.: Active learning literature survey (2009)
Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Active learning with Gaussian processes for object categorization. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)
Chattopadhyay, R., Wang, Z., Fan, W., Davidson, I., Panchanathan, S., Ye, J.: Batch mode active sampling based on marginal probability distribution matching. In: KDD Proceedings of International Conference on Knowledge Discovery and Data Mining 2012, pp. 741–749 (2013)
Copa, L., Tuia, D., Volpi, M., et al.: Unbiased query-by-bagging active learning for VHR image classification. In: Image and Signal Processing for Remote Sensing XVI. International Society for Optics and Photonics, vol. 7830, p. 78300K (2010)
Jaiswal, A., Babu, A.R., Zadeh, M.Z., et al.: A survey on contrastive self-supervised learning. Technologies (2021)
Wang, L.: Smoothness, disagreement coefficient, and the label complexity of agnostic active learning. J. Mach. Learn. Res. 12, 2269–2292 (2011)
Zhang, L., Chen, C., Bu, J.: Active learning based on locally linear reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 2026–2038 (2011)
Cai, W., Zhang, Y., Zhou, S.: Active learning for support vector machines with maximum model change. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 211–226. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44848-9_14
Yang, Z., Tang, J., Zhang, Y.: Active learning for streaming networked data. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1129–1138. ACM (2014)
Li, S., Xue, Y., Wang, Z.: Active learning for cross-domain sentiment classification. In: Proceeding of the 23rd International Joint Conference on Artificial Intelligence, pp. 2127–2133 (2013)
Zhang, H.T., Huang, M.L., Zhu, X.Y.: A unified active learning framework for biomedical relation extraction. J. Comput. Sci. Technol. 27(6), 1302–1313 (2014)
Wang, D., Yan, C., Shan, S., et al.: Active learning for interactive segmentation with expected confidence change. In: Proceedings of the 11th Asian Conference on Computer Vision, pp.790–802. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-37331-2_59
King, R.D., Whelan, K.E., Jones, F.M.: Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 427(6971), 247 (2004)
Liu, Y.: Active learning with support vector machine applied to gene expression data for cancer classification. J. Chem. Inf. Comput. Sci. 44(6), 1936–1941 (2004)
Lewis, D., Gale, W.: A sequential algorithm for training text classifiers. In: Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 3–12. ACM/Springer (1994)
Beluch, W.H., Genewein, T., Nurnberger, A., Kohler, J.: The power of ensembles for active learning in image classification. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9368–9377 (2018)
Gilad-Bachrach, R., Navot, A., Tishby, N.: Query by committee made real. In: NIPS (2005)
Settles, B., Craven, M., Ray, S.: Multiple-instance active learning. In: Advances in Neural Information Processing Systems (NIPS), vol. 20, pp. 1289–1296. MIT Press (2008)
Roy, N., McCallum, A.: Toward optimal active learning through sampling estimation of error reduction. In: Proceedings of the International Conference on Machine Learning (ICML), pp. 441–448. Morgan Kaufmann (2001)
Gilad-Bachrach, R., Navot, A., Tishby, N.: Query by committee made real. In: Advances in Neural Information Processing Systems, pp. 443–450 (2006)
Tran, T., Do, T.T., Reid, I.: Bayesian generative active deep learning. In: International Conference on Machine Learning. PMLR, pp. 6295–6304 (2019)
Saito, K., Watanabe, K., Ushiku, Y.: Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3723–3732 (2018)
Higgins, I., Matthey, L., Pal, A.: beta-vae: learning basic visual concepts with a constrained variational framework (2016)
Goodfellow, I., Pouget-Abadie, J., Mirza, M.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Bojanowski, P., Joulin, A.: Unsupervised learning by predicting noise (2017)
Alexey, D., Fischer, P., Tobias, J., Springenberg, M.R., Brox, T.: Discriminative unsupervised feature learning with exemplar convolutional neural networks (2014)
Wu, Z., Xiong, Y., Yu, S., Lin, D.: Unsupervised feature learning via non-parametric instance-level discrimination (2018)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments (2020)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning (2019)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations (2020)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Liu, P., Wang, L., He, G.: A survey on active deep learning: from model-driven to data-driven. arXiv preprint arXiv:2101.09933 (2021)
Gutmann, M., Hyvärinen, A.: Noise-contrastive estimation: a new estimation principle for unnormalized statistical models. In: AISTATS (2010)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
Dagan, I., Engelson, S.P.: Committee-based sampling for training probabilistic classifiers. In: Machine Learning Proceedings 1995, pp. 150–157. Elsevier(1995)
Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3733–3742 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Luo, SF. (2022). Research on Active Sampling with Self-supervised Model. In: Tian, Y., Ma, T., Khan, M.K., Sheng, V.S., Pan, Z. (eds) Big Data and Security. ICBDS 2021. Communications in Computer and Information Science, vol 1563. Springer, Singapore. https://doi.org/10.1007/978-981-19-0852-1_54
Download citation
DOI: https://doi.org/10.1007/978-981-19-0852-1_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-0851-4
Online ISBN: 978-981-19-0852-1
eBook Packages: Computer ScienceComputer Science (R0)