Abstract
Annotating data for supervised learning can be costly. When the annotation budget is limited, active learning can be used to select and annotate those observations that are likely to give the most gain in model performance. We propose an active learning algorithm that, in addition to selecting which observation to annotate, selects the precision of the annotation that is acquired. Assuming that annotations with low precision are cheaper to obtain, this allows the model to explore a larger part of the input space, with the same annotation budget. We build our acquisition function on the previously proposed BALD objective for Gaussian Processes, and empirically demonstrate the gains of being able to adjust the annotation precision in the active learning loop.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Code provided at https://github.com/AOlmin/active_learning_weak_sup.
References
Chakraborty, S.: Asking the right questions to the right users: active learning with imperfect oracles. In: AAAI (2020)
Cover, T.M., Thomas, J.A.: Elements of Information Theory, vol. 2. Wiley, Hoboken (2006)
Dua, D., Graff, C.: UCI Machine Learning Repository (2019). http://archive.ics.uci.edu/ml
Gao, R., Saar-Tsechansky, M.: Cost-accuracy aware adaptive labeling for active learning. In: AAAI (2020)
Hamidieh, K.: A data-driven statistical model for predicting the critical temperature of a superconductor. Comput. Mater. Sci. 154, 346–354 (2018)
Herde, M., Kottke, D., Huseljic, D., Sick, B.: Multi-annotator probabilistic active learning. In: ICPR (2020)
Houlsby, N., Huszár, F., Ghahramani, Z., Lengyel, M.: Bayesian Active Learning for Classification and Preference Learning. arXiv preprint arXiv: 1112.5745 (2011)
Huang, S.J., Chen, J.L., Mu, X., Zhou, Z.H.: Cost-effective active learning from diverse labelers. In: IJCAI (2017)
Li, S., Kirby, R.M., Zhe, S.: Deep multi-fidelity active learning of high-dimensional outputs. In: AISTATS (2022)
Li, S., Xing, W., Kirby, R.M., Zhe, S.: Multi-fidelity bayesian optimization via deep neural networks. In: NeurIPS (2020)
Pellegrini, R., Wackers, J., Broglia, R., Diez, M., Serani, A., Visonneau, M.: A Multi-fidelity active learning method for global design optimization problems with noisy evaluations. arXiv preprint arXiv:2202.06902 (2022)
Picheny, V., Ginsbourger, D., Richet, Y., Caplin, G.: Quantile-based optimization of noisy computer experiments with tunable precision. Technometrics 55(1), 2–13 (2013)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Boston (2006)
Settles, B.: Active Learning Literature Survey. Technical Report. Computer Sciences Technical Report 1648, University of Wisconsin, Madison (2010)
Song, J., Chen, Y., Yue, Y.: A general framework for multi-fidelity bayesian optimization with gaussian processes. In: AISTATS (2019)
Takeno, S., et al.: Multi-fidelity bayesian optimization with max-value entropy search and its parallelization. In: ICML (2020)
Tian, K., Li, Z., Ma, X., Zhao, H., Zhang, J., Wang, B.: Toward the robust establishment of variable-fidelity surrogate models for hierarchical stiffened shells by two-step adaptive updating approach. Struct. Multidisc. Optim. 61, 1515–1528 (2020)
Wu, Y., Hu, J., Zhou, Q., Wang, S., Jin, P.: An active learning multi-fidelity metamodeling method based on the bootstrap estimator. Aeros. Sci. Technol. 106, 106116 (2020)
Yan, Y., Rosales, R., Fung, G., Dy, J.G.: Active learning from crowds. In: ICML (2011)
Yeh, I.C.: Modeling of strength of high-performance concrete using artificial neural networks. Cement Conc. Res. 28(12), 1797–1808 (1998)
Acknowledgements
This research is financially supported by the Swedish Research Council (project 2020-04122), the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, and the Excellence Center at Linköping–Lund in Information Technology (ELLIIT).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Olmin, A., Lindqvist, J., Svensson, L., Lindsten, F. (2023). Active Learning with Weak Supervision for Gaussian Processes. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1792. Springer, Singapore. https://doi.org/10.1007/978-981-99-1642-9_17
Download citation
DOI: https://doi.org/10.1007/978-981-99-1642-9_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1641-2
Online ISBN: 978-981-99-1642-9
eBook Packages: Computer ScienceComputer Science (R0)