Data Augmentation for Building an Ensemble of Convolutional Neural Networks

  • Loris Nanni
  • Sheryl BrahnamEmail author
  • Gianluca Maguolo
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 145)


Bioimage classification is important in subcellular localization, accurate cell phenotype recognition, and histopathological classification, to name a few applications. In this paper, we propose an ensemble of deep learning methods built using different batch sizes, different learning rates, and different methods of data augmentation. Our main goal is to test different methods of data augmentation for building an ensemble that boosts the performance of Convolutional Neural Networks (CNN). Our method is evaluated on a diverse set of bioimage classification problems, with each represented by a benchmark dataset and with each bioimage classification task representing a typical cellular or tissue-level classification problem. The results on these datasets demonstrate that the proposed ensemble does indeed boost the performance of the standard CNN. The MATLAB code of all the descriptors and experiments reported in this paper is available at


Microscopy imaging classification Deep learning Convolutional neural networks Support vector machines 



We gratefully acknowledge the support of NVIDIA Corporation with their donation of the Titan XP GPU used in this research.


  1. 1.
    Schermelleh, L., Heintzmann, R., Leonhardt, H.: A guide to super-resolution fluorescence microscopy. J. Cell Biol. 190, 165–175 (2010)CrossRefGoogle Scholar
  2. 2.
    Zhou, J., Lamichhane, S., Sterne, G., Ye, B., Peng, H.: BIOCAT: a pattern recognition platform for customizable biological image classification and annotation. BMC Bioinformatics 14, 291 (2013)CrossRefGoogle Scholar
  3. 3.
    Misselwitz, B., Strittmatter, G., Periaswamy, B., Schlumberger, M.C., Rout, S., Horvath, P., Kozak, K., Hardt, W.D.: Enhanced cell classifier: a multi-class classification tool for microscopy images. BMC Bioinformat. 11 (2010)Google Scholar
  4. 4.
    Pau, G., Fuchs, F., Sklyar, O., Boutros, M., Huber, W.: EBImage—an R package for image processing with applications to cellular phenotypes. Bioinformatics 26 (2010)CrossRefGoogle Scholar
  5. 5.
    Uhlmann, V., Singh, S., Carpenter, A.E.: CP-CHARM: segmentation-free image classification made accessible. BMC Bioinformatics 17, 51 (2016)CrossRefGoogle Scholar
  6. 6.
    Rui, Y., Huang, T.S., Chang, S.F.: Image retrieval: current techniques, promising directions, and open issues. J. Vis. Commun. Image Represent. 10, 39–62 (1999)CrossRefGoogle Scholar
  7. 7.
    Vailaya, A., Figueiredo, M.A.T., Jain, A.K., Zhang, H.J.: Image classification for content-based indexing. IEEE Trans. Image Process. 10, 117–130 (2001)CrossRefGoogle Scholar
  8. 8.
    Huang, K., Murphy, R.F.: Automated classification of subcellular patterns in multicell images without segmentation into single cells. IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI’04), pp. 1139–11342. IEEE, Arlington (2004)Google Scholar
  9. 9.
    Glory, E., Murphy, R.: Automated subcellular location determination and high throughput microscopy. Dev. Cell 12, 7–16 (2007)CrossRefGoogle Scholar
  10. 10.
    Yang, F., Xu, Y.Y., Shen, H.B.: Many local pattern texture features: which is better for image-based multilabel human protein subcellular localization classification. Scientif. World J. 2014, 429049 (2014)Google Scholar
  11. 11.
    Fogel, I., Sagi, D.: Gabor filters as texture discriminator. Biol. Cybern. 61, 103–113 (1989)CrossRefGoogle Scholar
  12. 12.
    Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Systems Man Cybernet. 3, 610–621 (1973)CrossRefGoogle Scholar
  13. 13.
    Nanni, L., Brahnam, S., Ghidoni, S., Menegatti, E., Barrier, T.: A comparison of methods for extracting information from the co-occurrence matrix for subcellular classification. Expert Syst. Appl. 40, 7457–7467 (2013)CrossRefGoogle Scholar
  14. 14.
    Xu, Y., Zhu, J., Chang, E.I., Lai, M., Tu, Z.: Weakly supervised histopathology cancer image segmentation and classification. Med. Image Anal. 18, 591–604 (2014)CrossRefGoogle Scholar
  15. 15.
    Barker, J., Hoogi, A., Depeursinge, A., Rubin, D.L.: Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med. Image Anal. 30, 60–71 (2016)CrossRefGoogle Scholar
  16. 16.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge, UK (2000)CrossRefGoogle Scholar
  17. 17.
    Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Networks 61, 85–117 (2015)CrossRefGoogle Scholar
  18. 18.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  19. 19.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How Transferable are Features in Deep Neural Networks? Cornell University (2014)Google Scholar
  20. 20.
    Araújo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Polónia, A., Campilho, A.: Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12, 1–14 (2017)Google Scholar
  21. 21.
    Mateen, M., Wen, J., Nasrullah, Song, S., Huang, Z.: Fundus image classification using VGG-19 architecture with PCA and SVD. Symmetry 11 (2018)CrossRefGoogle Scholar
  22. 22.
    Badejo, J.A., Adetiba, E., Akinrinmade, A., Akanle, M.B.: medical image classification with hand-designed or machine-designed texture descriptors: a performance evaluation. In: Bioinformatics and Biomedical Engineering, pp. 266–275. Springer International PublishingGoogle Scholar
  23. 23.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: COPY ImageNet classification with deep convolutional neural networks. In: Bartlett, P.L., Pereira, F.C.N., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Adv Neural Inf Process Syst, pp. 1106–1114. Curran Associates Inc, Lake Tahoe, NV (2012)Google Scholar
  24. 24.
    Shijie, J., Ping, W., Peiyi, J., Siping, H.: Research on data augmentation for image classification based on convolution neural networks. Chinese Automation Congress (CAC) 2017, pp. 4165–4170, Jinan (2017)Google Scholar
  25. 25.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015 LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015)Google Scholar
  26. 26.
    Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)CrossRefGoogle Scholar
  27. 27.
    Zhao, D., Zhu, D., Lu, J., Luo, Y., Zhang, G.: Synthetic medical images using F&BGAN for improved lung nodules classification by multi-scale VGG16. Symmetry 10, 519 (2018)CrossRefGoogle Scholar
  28. 28.
    Ding, J., Chen, B., Liu, H., Huang, M.: Convolutional neural network with data augmentation for sar target recognition. IEEE Geosci. Remote Sens. Lett. 13, 364–368 (2016)Google Scholar
  29. 29.
    Castro, E., Cardoso, J.S., Pereira, J.C.: Elastic Deformations for Data Augmentation in Breast Cancer Mass Detection, pp. 230–234 (2018)Google Scholar
  30. 30.
    Tajbakhsh, N., Shin, J.Y., Gurudu, S.R., Hurst, R.T., Kendall, C.B., Gotway, M.B., Liang, J.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35, 1299–1312 (2016)CrossRefGoogle Scholar
  31. 31.
    Xu, Y., Jia, Z., Wang, L.-B., Ai, Y., Zhang, F., Lai, M., Chang, E.I.C.: Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinformatics 18, 281 (2017)CrossRefGoogle Scholar
  32. 32.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE, Las Vegas, NV (2016)Google Scholar
  33. 33.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely Connected Convolutional Networks. CVPR 1, 3 (2017)Google Scholar
  34. 34.
    Boland, M.V., Murphy, R.F.: A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells. Bioinformatics 17, 1213–1223 (2001)CrossRefGoogle Scholar
  35. 35.
    Shamir, L., Orlov, N.V., Eckley, D.M., Goldberg, I.: IICBU 2008: a proposed benchmark suite for biological image analysis. Med. Biol. Eng. Compu. 46, 943–947 (2008)CrossRefGoogle Scholar
  36. 36.
    Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. Analysis and Modelling of Faces and Gestures LNCS 4778, 168–182 (2007)zbMATHGoogle Scholar
  37. 37.
    Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19, 1657–1663 (2010)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Nosaka, R., Fukui, K.: HEp-2 cell classification using rotation invariant co-occurrence among local binary patterns. Pattern Recognition in Bioinformatics 47, 2428–2436 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Department of Information EngineeringUniversity of PaduaPaduaItaly
  2. 2.Computer Information SystemsMissouri State UniversitySpringfieldUSA

Personalised recommendations