Abstract
Hyperparameter tuning is a time-consuming task for deep learning models. Meta-learning offers a promising solution to reduce the time required for this task. In this work, we propose a meta-learning approach to simulate a set of experiments and select a hyperparameter configuration (HC) that achieves high accuracy using a deep model. Our formulation involves conducting a grid search over hyperparameters to train a convolutional neural network and get an overview of their space. Then, a meta-regressor was trained using the experiment data to predict accuracy as a function of hyperparameter sets. Subsequently, the trained meta-regressor was employed to simulate diverse HCs and assess the corresponding deep model performance. Our approach was tested across two different domains: COVID-19 detection using X-ray images, and lung detection in computer tomography volumes. Furthermore, we evaluated the proposed approach with two different architectures. Our results show that the proposed method can simulate a set of experiments using the meta-regressor, saving time and computing resources during hyperparameter tuning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balaji, Y., Sankaranarayanan, S., Chellappa, R.: Metareg: Towards domain generalization using meta-regularization. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 1–11. Curran Associates, Inc. (2018)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Ceron, J.S.O., Castro, P.S.: Revisiting rainbow: promoting more insightful and inclusive deep reinforcement learning research. In: International Conference on Machine Learning, pp. 1373–1383. PMLR (2021)
Chowdhury, M.E.: Can AI help in screening viral and COVID-19 pneumonia? IEEE Access 8, 132665–132676 (2020)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)
Ding, Y., et al.: Learning to learn by jointly optimizing neural architecture and weights. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 129–138 (2022)
Franceschi, L., Frasconi, P., Salzo, S., Grazzi, R., Pontil, M.: Bilevel programming for hyperparameter optimization and meta-learning. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 1568–1577. PMLR (2018)
Garouani, M., Ahmad, A., Bouneffa, M., Hamlich, M., Bourguin, G., Lewandowski, A.: Using meta-learning for automated algorithms selection and configuration: an experimental framework for industrial big data. J. Big Data 9(1), 57 (2022)
Hessel, M., et al.: Rainbow: combining improvements in deep reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)
Hospedales, T., Antoniou, A., Micaelli, P., Storkey, A.: Meta-learning in neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 5149–5169 (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Liang, H., et al.: Training interpretable convolutional neural networks by differentiating class-specific filters. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 622–638. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_37
Liashchynskyi, P., Liashchynskyi, P.: Grid search, random search, genetic algorithm: a big comparison for NAS. arXiv preprint arXiv:1912.06059 (2019)
Occorso, M., Sabbioni, L., Metelli, A.M., Restelli, M.: Trust region meta learning for policy optimization. In: ECMLPKDD Workshop on Meta-Knowledge Transfer, pp. 62–74. PMLR (2022)
Oshiro, T.M., Perez, P.S., Baranauskas, J.A.: How many trees in a random forest? In: Perner, P. (ed.) MLDM 2012. LNCS (LNAI), vol. 7376, pp. 154–168. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31537-4_13
Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Raschka, S.: Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808 (2018)
Roth, H.R.: Rapid artificial intelligence solutions in a pandemic-the COVID-19-20 lung CT lesion segmentation challenge. Med. Image Anal. 82, 102605 (2022)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)
Xiao, X., Yan, M., Basodi, S., Ji, C., Pan, Y.: Efficient hyperparameter optimization in deep learning using a variable length genetic algorithm. arXiv preprint arXiv:2006.12703 (2020)
Acknowledgments
This work was supported by the Universidad Nacional Autónoma de México by means of PAPIIT grants TA101121 and IV100420. Rodrigo Ramos Díaz acknowledges CONACYT for the scholarship that supports his PhD studies associated with CVU number 927245.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
García-Ramírez, J., Ramos Díaz, R., Olveres, J., Escalante-Ramírez, B. (2023). Meta-Learning for Hyperparameters Tuning in CNNs for Chest Images. In: García Bringas, P., et al. Hybrid Artificial Intelligent Systems. HAIS 2023. Lecture Notes in Computer Science(), vol 14001. Springer, Cham. https://doi.org/10.1007/978-3-031-40725-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-40725-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40724-6
Online ISBN: 978-3-031-40725-3
eBook Packages: Computer ScienceComputer Science (R0)