Abstract
When learning from images, it is desirable to augment the dataset with plausible transformations of its images. Unfortunately, it is not always intuitive for the user how much shear or translation to apply. For this reason, training multiple models through hyperparameter search is required to find the best augmentation policies. But these methods are computationally expensive. Furthermore, since they generate static policies, they do not take advantage of smoothly introducing more aggressive augmentation transformations. In this work, we propose repeating each epoch twice with a small difference in data augmentation intensity, walking towards the best policy. This process doubles the number of epochs, but avoids having to train multiple models. The method is compared against random and Bayesian search for classification and segmentation tasks. The proposal improved twice over random search and was on par with Bayesian search for 4% of the training epochs.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Intel & MobileODT cervical cancer screening, Kaggle competition (2017). https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening. Accessed 1 Dec 2018
Retinal CHASE DB1 database (2017). https://blogs.kingston.ac.uk/retinal/chasedb1/. Accessed 1 Dec 2018
Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 41–48. ACM (2009). https://doi.org/10.1145/1553374.1553380
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012)
Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Advances in Neural Information Processing Systems, pp. 2546–2554 (2011)
Cardoso, J.S., Cardoso, M.J.: Towards an intelligent medical system for the aesthetic evaluation of breast cancer conservative treatment. Artif. Intell. Med. 40(2), 115–126 (2007). https://doi.org/10.1016/j.artmed.2007.02.007
Coates, A., Ng, A., Lee, H.: An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 215–223 (2011)
Codella, N.C., Gutman, D., Celebi, M.E., Helba, B., et al.: Skin lesion analysis toward melanoma detection: a challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI), pp. 168–172. IEEE (2018). https://doi.org/10.1109/ISBI.2018.8363547
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: learning augmentation policies from data (2018). arXiv preprint: arXiv:1805.09501
Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015). https://doi.org/10.1007/s11263-014-0733-5
Fernandes, K., Cardoso, J.S., Fernandes, J.: Transfer learning with partial observability applied to cervical cancer screening. In: Alexandre, L.A., Salvador Sánchez, J., Rodrigues, J.M.F. (eds.) IbPRIA 2017. LNCS, vol. 10255, pp. 243–250. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58838-4_27
Fernandez, K., Chang, C.: Teeth/Palate and interdental segmentation using artificial neural networks. In: Mana, N., Schwenker, F., Trentin, E. (eds.) ANNPR 2012. LNCS (LNAI), vol. 7477, pp. 175–185. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33212-8_16
Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19(3), 203–210 (2000)
Jamieson, K., Talwalkar, A.: Non-stochastic best arm identification and hyperparameter optimization. In: Artificial Intelligence and Statistics, pp. 240–248 (2016)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, Citeseer (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Larsen, J., Hansen, L.K., Svarer, C., Ohlsson, M.: Design and regularization of neural networks: the optimal use of a validation set. In: Proceedings of the 1996 IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing VI, pp. 62–71. IEEE (1996)
LeCun, Y., Cortes, C., Burges, C.: MNIST handwritten digit database. AT&T Labs [Online]. http://yann.lecun.com/exdb/mnist 2 (2010). https://doi.org/10.1109/MSP.2012.2211477
Mendonça, T., Ferreira, P.M., Marques, J.S., Marcal, A.R., Rozeira, J.: PH\(^{2}\)-a dermoscopic image database for research and benchmarking. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5437–5440. IEEE (2013). https://doi.org/10.1109/EMBC.2013.6610779
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning, vol. 2011, p. 5 (2011)
Olson, R.S., Urbanowicz, R.J., Andrews, P.C., Lavender, N.A., Kidd, L.C., Moore, J.H.: Automating biomedical data science through tree-based pipeline optimization. In: Squillero, G., Burelli, P. (eds.) EvoApplications 2016, Part I. LNCS, vol. 9597, pp. 123–137. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31204-0_9
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sequeira, A.F., Monteiro, J.C., Rebelo, A., Oliveira, H.P.: MobBIO: a multimodal database captured with a portable handheld device. In: 2014 International Conference on Computer Vision Theory and Applications (VISAPP), vol. 3, pp. 133–139. IEEE (2014). https://doi.org/10.5220/0004679601330139
Staal, J., Abràmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23(4), 501–509 (2004). https://doi.org/10.1109/TMI.2004.825627
Vasconcelos, M.J.M., Rosado, L., Ferreira, M.: Principal axes-based asymmetry assessment methodology for skin lesion image analysis. In: Bebis, G., et al. (eds.) ISVC 2014, Part II. LNCS, vol. 8888, pp. 21–31. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-14364-4_3
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms (2017). arXiv preprint: arXiv:1708.07747
Acknowledgments
This work is financed by National Funds through the Portuguese funding agency, FCT – Fundação para a Ciência e a Tecnologia within project: UID/EEA/50014/2019, and the PhD grant “SFRH/BD/122248/2016” from FCT supported by POCH and the EU.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cruz, R., Pinto Costa, J.F., Cardoso, J.S. (2019). Automatic Augmentation by Hill Climbing. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning. ICANN 2019. Lecture Notes in Computer Science(), vol 11728. Springer, Cham. https://doi.org/10.1007/978-3-030-30484-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-30484-3_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30483-6
Online ISBN: 978-3-030-30484-3
eBook Packages: Computer ScienceComputer Science (R0)