Skip to main content
SpringerLink
Log in

Influence of Data Augmentation Strategies on the Segmentation of Oral Histological Images Using Fully Convolutional Neural Networks

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Segmentation of tumor regions in H &E-stained slides is an important task for a pathologist while diagnosing different types of cancer, including oral squamous cell carcinoma (OSCC). Histological image segmentation is often constrained by the availability of labeled training data since labeling histological images is a highly skilled, complex, and time-consuming task. Thus, data augmentation strategies become essential to train convolutional neural networks models to overcome the overfitting problem when only a few training samples are available. This paper proposes a new data augmentation strategy, named Random Composition Augmentation (RCAug), to train fully convolutional networks (FCN) to segment OSCC tumor regions in H &E-stained histological images. Given the input image and their corresponding label, a pipeline with a random composition of geometric, distortion, color transfer, and generative image transformations is executed on the fly. Experimental evaluations were performed using an FCN-based method to segment OSCC regions through a set of different data augmentation transformations. By using RCAug, we improved the FCN-based segmentation method from 0.51 to 0.81 of intersection-over-union (IOU) in a whole slide image dataset and from 0.65 to 0.69 of IOU in a tissue microarray images dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. WHO: The Website of the World Health Organization (WHO). Accessed: 2022-01-16. https://www.who.int/health-topics/cancer#tab=tab_1

  2. Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R., Torre, L., Jemal, A.: Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries: Global cancer statistics 2018. CA: A Cancer Journal for Clinicians 68 (2018). https://doi.org/10.3322/caac.21492

  3. INCA: The Website of the Instituto Nacional de Câncer José Alencar Gomes Da Silva (INCA). Accessed: 2022-01-16. https://www.inca.gov.br/tipos-de-cancer/cancer-de-boca

  4. Boxberg, M., Bollwein, C., Jöhrens, K., Kuhn, P.-H., Haller, B., Steiger, K., Wolff, K.-D., Kolk, A., Jesinghaus, M., Weichert, W.: Novel prognostic histopathological grading system in oral squamous cell carcinoma based on tumour budding and cell nest size shows high interobserver and intraobserver concordance. Journal of Clinical Pathology 72(4), 285–294 (2019). https://doi.org/10.1136/jclinpath-2018-205454

  5. Hashibe, M., Brennan, P., Benhamou, S., Castellsague, X., Chen, C., Curado, M.P., Maso, L.D., Daudt, A.W., Fabianova, E., Wünsch-Filho, V., Franceschi, S., Hayes, R.B., Herrero, R., Koifman, S., Vecchia, C.L., Lazarus, P., Levi, F., Mates, D., Matos, E., Menezes, A., Muscat, J., Eluf-Neto, J., Olshan, A.F., Rudnai, P., Schwartz, S.M., Smith, E., Sturgis, E.M., Szeszenia-Dabrowska, N., Talamini, R., Wei, Q., Winn, D.M., Zaridze, D., Zatonski, W., Zhang, Z.-F., Berthiller, J., Boffetta, P.: Alcohol Drinking in Never Users of Tobacco, Cigarette Smoking in Never Drinkers, and the Risk of Head and Neck Cancer: Pooled Analysis in the International Head and Neck Cancer Epidemiology Consortium. JNCI: Journal of the National Cancer Institute 99(10), 777–789 (2007). https://doi.org/10.1093/jnci/djk179

  6. Boffetta, P., Hecht, S., Gray, N., Gupta, P., Straif, K.: Smokeless tobacco and cancer. The lancet oncology 9, 667–75 (2008). https://doi.org/10.1016/S1470-2045(08)70173-6

  7. Chaturvedi, A.K., Engels, E.A., Pfeiffer, R.M., Hernandez, B.Y., Xiao, W., Kim, E., Jiang, B., Goodman, M.T., Sibug-Saber, M., Cozen, W., Liu, L., Lynch, C.F., Wentzensen, N., Jordan, R.C., Altekruse, S., Anderson, W.F., Rosenberg, P.S., Gillison, M.L.: Human papillomavirus and rising oropharyngeal cancer incidence in the united states. Journal of Clinical Oncology 29(32), 4294–4301 (2011). https://doi.org/10.1200/JCO.2011.36.4596. PMID: 21969503

  8. Mahmood, H., Shaban, M., Indave, B., Santos-Silva, A., Rajpoot, N., Khurram, S.A.: Use of artificial intelligence in diagnosis of head and neck precancerous and cancerous lesions: A systematic review. Oral Oncology 110, 104885 (2020). https://doi.org/10.1016/j.oraloncology.2020.104885

  9. Pulte, D., Brenner, H.: Changes in survival in head and neck cancers in the late 20th and early 21st century: A period analysis. The oncologist 15, 994–1001 (2010). https://doi.org/10.1634/theoncologist.2009-0289

  10. Liao, L.-J., Hsu, W.-L., Lo, W.-C., Cheng, P.-W., Shueng, P.-W., Hsieh, C.-H.: Health-related quality of life and utility in head and neck cancer survivors. BMC Cancer 19 (2019). https://doi.org/10.1186/s12885-019-5614-4

  11. Baik, J., Ye, Q., Zhang, L., Poh, C., Rosin, M., Macaulay, C., Guillaud, M.: Automated classification of oral premalignant lesions using image cytometry and random forests-based algorithms. Cellular Oncology 37, 193–202 (2014). https://doi.org/10.1007/s13402-014-0172-x

  12. Das, D.K., Chakraborty, C., Sawaimoon, S., Maiti, A., Chatterjee, S.: Automated identification of keratinisation and keratin pearl area from in situ oral histological images. Tissue and Cell 47 (2015). https://doi.org/10.1016/j.tice.2015.04.009

  13. de Andrea, C.E., Bleggi-Torres, L.F., de Seixas Alves, M.T.: Anáise da morfometria nuclear: descrição da metodologia e o papel dos softwares de edição de imagem. Jornal Brasileiro de Patologia e Medicina Laboratorial 44, 51–57 (2008)

  14. Bell, E.S., Lammerding, J.: Causes and consequences of nuclear envelope alterations in tumour progression. European journal of cell biology 95, 449–464 (2016). https://doi.org/10.1016/j.ejcb.2016.06.007

  15. Shaban, M., Khurram, S.A., Fraz, M.M., Alsubaie, N., Masood, I., Mushtaq, S., Hassan, M., Loya, A., Rajpoot, N.M.: A novel digital score for abundance of tumour infiltrating lymphocytes predicts disease free survival in oral squamous cell carcinoma. Scientific Reports 9(1) (2019). https://doi.org/10.1038/s41598-019-49710-z

  16. Filipczuk, P., Kowal, M., Obuchowicz, A.: Automatic breast cancer diagnosis based on k-means clustering and adaptive thresholding hybrid segmentation. In: IP &C (2011)

  17. Heijmans, H.J.A.M.: Mathematical morphology: Basic principles. In: Proceedings of Summer School on Morphological Image and Signal Processing, pp. 228–231 (1995)

  18. Phoulady, H.A., Goldgof, D.B., Hall, L.O., Mouton, P.R.: Nucleus segmentation in histology images with hierarchical multilevel thresholding. In: Medical Imaging 2016: Digital Pathology, vol. 9791, pp. 280–285. International Society for Optics and Photonics, SPIE (2016). https://doi.org/10.1117/12.2216632

  19. Gurcan, M.N., Boucheron, L.E., Can, A., Madabhushi, A., Rajpoot, N.M., Yener, B.: Histopathological image analysis: A review. IEEE Reviews in Biomedical Engineering 2, 147–171 (2009). https://doi.org/10.1109/RBME.2009.2034865

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

  21. Xing, F., Xie, Y., Su, H., Liu, F., Yang, L.: Deep learning in microscopy image analysis: A survey. IEEE Transactions on Neural Networks and Learning Systems 29(10), 4550–4568 (2018)

  22. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. Journal of Big Data 6(1), 60 (2019). https://doi.org/10.1186/s40537-019-0197-0

  23. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural information processing systems 27 (2014)

  24. Martino, F., Bloisi, D.D., Pennisi, A., Fawakherji, M., Ilardi, G., Russo, D., Nardi, D., Staibano, S., Merolla, F.: Deep learning-based pixel-wise lesion segmentation on oral squamous cell carcinoma images. Applied Sciences 10(22) (2020). https://doi.org/10.3390/app10228285

  25. Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: A survey. Medical Image Analysis 67, 101813 (2021). https://doi.org/10.1016/j.media.2020.101813

  26. Das, D.K., Bose, S., Maiti, A., Mitra, B., Mukherjee, G., Dutta, P.: Automatic identification of clinically relevant regions from oral tissue histological images for oral squamous cell carcinoma diagnosis. Tissue and Cell 53 (2018). https://doi.org/10.1016/j.tice.2018.06.004

  27. Halicek, M., Shahedi, M., Little, J., Chen, A., Myers, L., Sumer, B., Fei, B.: Head and neck cancer detection in digitized whole-slide histology using convolutional neural networks. Scientific Reports 9 (2019). https://doi.org/10.1038/s41598-019-50313-x

  28. dos Santos, D.F.D., de Faria, P.R., Travençolo, B.A.N., do Nascimento, M.Z.: Automated detection of tumor regions from oral histological whole slide images using fully convolutional neural networks. Biomedical Signal Processing and Control 69, 102921 (2021). https://doi.org/10.1016/j.bspc.2021.102921

  29. Bejnordi, B.E., Veta, M., van Diest, P.J., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J.A.W.M., the CAMELYON16 Consortium: Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA 318(22), 2199–2210 (2017). https://doi.org/10.1001/jama.2017.14585

  30. TCGA: The Website of The Cancer Genome Atlas Program (TCGA). Accessed: 2021-03-17. https://www.cancer.gov/about-nci/organization/ccg/research/structural-genomics/tcga

  31. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., Ginneken, B., Sánchez, C.I.: A survey on deep learning in medical image analysis. Medical image analysis 42, 60–88 (2017)

  32. dos Santos, D.F.D., Loyola, A., Cardoso, S., de Faria, P.R., Travençolo, B.A.N., do Nascimento, M.Z.: H &E-stained oral squamous cell carcinoma histological images dataset. Mendeley Data V1 (2022). https://doi.org/10.17632/9bsc36jyrt.1

  33. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). LNCS, vol. 9351, pp. 234–241. Springer (2015). http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a, (available on arXiv:1505.04597 [cs.CV]).

  34. Falk, T., Mai, D., Bensch, R., Ozgün ÇiçSek, Abdulkadir, A., Marrakchi, Y., Böhm, A., Deubner, J., Jäckel, Z., Seiwald, K., Dovzhenko, A., Tietz, O., Bosco, C.D., Walsh, S., Saltukoglu, D., Tay, T.L., Prinz, M., Palme, K., Simons, M., Diester, I., Brox, T., Ronneberger, O.: U-net – deep learning for cell counting, detection, and morphometry. Nat. Methods 16, 67–70 (2019)

  35. Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., Kalinin, A.A.: Albumentations: Fast and flexible image augmentations. Information 11(2) (2020). https://doi.org/10.3390/info11020125

  36. Xiao, Y., Decenciére, E., Velasco-Forero, S., Burdin, H., Bornschlögl, T., Bernerd, F., Warrick, E., Baldeweck, T.: A new color augmentation method for deep learning segmentation of histological images. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 886–890 (2019). https://doi.org/10.1109/ISBI.2019.8759591

  37. Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence 34 (2017). https://doi.org/10.1609/aaai.v34i07.7000

  38. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative Image Inpainting with Contextual Attention (2018)

  39. Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Information Processing & Management 45(4), 427–437 (2009). https://doi.org/10.1016/j.ipm.2009.03.002

  40. Wang, Z., Wang, E., Zhu, Y.: Image segmentation evaluation: a survey of methods. Artif. Intell. Rev. 53(8), 5637–5674 (2020). https://doi.org/10.1007/s10462-020-09830-9

  41. Xun, S., Li, D., Zhu, H., Chen, M., Wang, J., Li, J., Chen, M., Wu, B., Zhang, H., Chai, X., Jiang, Z., Zhang, Y., Huang, P.: Generative adversarial networks in medical image segmentation: A review. Computers in Biology and Medicine 140, 105063 (2021). https://doi.org/10.1016/j.compbiomed.2021.105063

  42. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in PyTorch. In: NIPS Autodiff Workshop (2017)

  43. Garcia, S., Herrera, F.: An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. Journal of Machine Learning Research - JMLR 9 (2008)

  44. Kendall, A., Badrinarayanan, V., Cipolla, R.: Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. (2017). https://doi.org/10.5244/C.31.57

  45. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition (2015)

  46. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

Download references

Funding

The authors gratefully acknowledge the financial support of National Council for Scientific and Technological Development CNPq (Grants 311404/2021-9, 306436/2022-1, 307318/2022-2) and the State of Minas Gerais Research Foundation - FAPEMIG (Grant APQ-00578-18).

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Federal University of Uberlândia, Brazil and Institute of Biomedical Science, Federal University of Uberlândia, Uberlândia, Brazil

    Dalí F. D. dos Santos, Paulo R. de Faria, Bruno A. N. Travençolo & Marcelo Z. do Nascimento

Authors
  1. Dalí F. D. dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paulo R. de Faria
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno A. N. Travençolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcelo Z. do Nascimento
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dalí F. D. dos Santos.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

dos Santos, D.F.D., de Faria, P.R., Travençolo, B.A.N. et al. Influence of Data Augmentation Strategies on the Segmentation of Oral Histological Images Using Fully Convolutional Neural Networks. J Digit Imaging 36, 1608–1623 (2023). https://doi.org/10.1007/s10278-023-00814-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-023-00814-z

Keywords

Search

Navigation