Skip to main content
SpringerLink
Log in

Joint Learning with Local and Global Consistency for Improved Medical Image Segmentation

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13413))

Included in the following conference series:

  • 2530 Accesses

Abstract

Semantic segmentation has been one of the key components in subsequent image-based decision-making across computer vision and biomedical imaging. While a lot of progress has been made with the advent of deep learning, segmentation models rely heavily on large labeled datasets for optimal results. Moreover, with added challenges due to varying imaging conditions, abnormalities, etc., it becomes relatively a harder problem to solve, even by the most sophisticated models. Additionally, segmentation models when employed at small patch-level lose the global context and when employed at the full image-level may lose focus to closely located and small objects-of-interest. In order to resolve such issues and thereby improve the segmentation performance, we propose a novel joint patch- and image-level training framework namely Image-to-Patch w/ Patch-to-Image (IPPI) which at the same time preserves the global context and pays attention to local details. Accommodating the joint training, our proposed IPPI technique can be incorporated with any segmentation network for improved performance and local-global consistency. Our experimentation with three different segmentation networks (U-Net, U-Net++, and NodeU-Net) in segmenting cell nuclei and retinal vessel demonstrates the effectiveness of the proposed IPPI method. The segmentation improvements—13.35% over U-Net, 5.56% over U-Net++, and 4.59% over NodeU-Net IoU (Intersection over Union) make it a potentially beneficial tool in challenging segmentation tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahamed, M.A., Ali Hossain, M., Al Mamun, M.: Semantic segmentation of self-supervised dataset and medical images using combination of u-net and neural ordinary differential equations. In: 2020 IEEE Region 10 Symposium (TENSYMP), pp. 238–241 (2020)

    Google Scholar 

  2. Bamford, P., Lovell, B.: Unsupervised cell nucleus segmentation with active contours. Signal Process. 71(2), 203–213 (1998)

    Article  Google Scholar 

  3. Beheshti, N., Johnsson, L.: Squeeze u-net: a memory and energy efficient image segmentation network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 364–365 (2020)

    Google Scholar 

  4. Caicedo, J.C., et al.: Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat. Methods 16(12), 1247–1253 (2019)

    Article  Google Scholar 

  5. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  6. Chen, T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: Advances in Neural Information Processing Systems, pp. 6571–6583 (2018)

    Google Scholar 

  7. Farnell, D.J., Hatfield, F., Knox, P., Reakes, M., Spencer, S., Parry, D., Harding, S.P.: Enhancement of blood vessels in digital fundus photographs via the application of multiscale line operators. J. Franklin Inst. 345(7), 748–765 (2008)

    Article  Google Scholar 

  8. Gerard, G., Piastra, M.: Conditional deep convolutional neural networks for improving the automated screening of histopathological images. arXiv preprint arXiv:2105.14338 (2021)

  9. Goyzueta, C.A.R., De la Cruz, J.E.C., Machaca, W.A.M.: Integration of u-net, resu-net and deeplab architectures with intersection over union metric for cells nuclei image segmentation. In: 2021 IEEE Engineering International Research Conference (EIRCON), pp. 1–4. IEEE (2021)

    Google Scholar 

  10. Hamilton, B.A.: 2018 data science bowl (2018). https://www.kaggle.com/c/data-science-bowl-2018. Accessed 09 Jan 2022

  11. Hassan, G., El-Bendary, N., Hassanien, A.E., Fahmy, A., Snasel, V., et al.: Retinal blood vessel segmentation approach based on mathematical morphology. Procedia Comput. Sci. 65, 612–622 (2015)

    Article  Google Scholar 

  12. Imran, A.A.Z.: From Fully-Supervised, Single-Task to Scarcely-Supervised, Multi-Task Deep Learning for Medical Image Analysis. Ph.D. thesis, UCLA (2020)

    Google Scholar 

  13. Imran, A.A.Z., et al.: Fully-automated analysis of scoliosis from spinal x-ray images. In: 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS), pp. 114–119 (2020)

    Google Scholar 

  14. Imran, A.A.Z., Terzopoulos, D.: Progressive adversarial semantic segmentation. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 4910–4917 (2021)

    Google Scholar 

  15. Irshad, S., Akram, M.U.: Classification of retinal vessels into arteries and veins for detection of hypertensive retinopathy. In: 2014 Cairo International Biomedical Engineering Conference (CIBEC), pp. 133–136 (2014)

    Google Scholar 

  16. Jin, Q., Meng, Z., Pham, T.D., Chen, Q., Wei, L., Su, R.: Dunet: a deformable network for retinal vessel segmentation. Knowl.-Based Syst. 178, 149–162 (2019)

    Article  Google Scholar 

  17. Kulbear: Residual networks (2017). https://github.com/Kulbear/deep-learning-coursera/blob/master/Convolutional Neural Networks/Residual Networks-v1.ipynb. Accessed 19 Aug 2019

  18. Lin, C.K., Chang, J., Huang, C.C., Wen, Y.F., Ho, C.C., Cheng, Y.C.: Effectiveness of convolutional neural networks in the interpretation of pulmonary cytologic images in endobronchial ultrasound procedures. Cancer Med. 10(24), 9047–9057 (2021)

    Article  Google Scholar 

  19. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  20. Maji, D., Santara, A., Mitra, P., Sheet, D.: Ensemble of deep convolutional neural networks for learning to detect retinal vessels in fundus images. arXiv preprint arXiv:1603.04833 (2016)

  21. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. Adv. Neural. Inf. Process. Syst. 32, 8026–8037 (2019)

    Google Scholar 

  22. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  23. Sekou, T.B., Hidane, M., Olivier, J., Cardot, H.: From patch to image segmentation using fully convolutional networks-application to retinal images. arXiv preprint arXiv:1904.03892 (2019)

  24. Subramanya, S.K.: Deep Learning Models to Characterize Smooth Muscle Fibers in Hematoxylin and Eosin Stained Histopathological Images of the Urinary Bladder. Ph.D. thesis, Rochester Institute of Technology (2021)

    Google Scholar 

  25. Thoma, M.: A survey of semantic segmentation. arXiv preprint arXiv:1602.06541 (2016)

  26. Wang, H., et al.: Patch-free 3D medical image segmentation driven by super-resolution technique and self-supervised guidance. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 131–141. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_13

    Chapter  Google Scholar 

  27. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  28. Xing, F., Xie, Y., Yang, L.: An automatic learning-based framework for robust nucleus segmentation. IEEE Trans. Med. Imaging 35(2), 550–566 (2015)

    Article  Google Scholar 

  29. Zana, F., Klein, J.C.: Segmentation of vessel-like patterns using mathematical morphology and curvature evaluation. IEEE Trans. Image Process. 10(7), 1010–1019 (2001)

    Article  Google Scholar 

  30. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested u-net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1

    Chapter  Google Scholar 

  31. Zhu, H., Cao, Z., Lian, L., Ye, G., Gao, H., Wu, J.: Cariesnet: a deep learning approach for segmentation of multi-stage caries lesion from oral panoramic x-ray image. Neural Comput. Appl. 1–9 (2022). https://doi.org/10.1007/s00521-021-06684-2

Download references

Author information

Authors and Affiliations

  1. Green University of Bangladesh, Dhaka, 01207, Bangladesh

    Md. Atik Ahamed

  2. Stanford University, Stanford, CA, 94305, USA

    Abdullah Al Zubaer Imran

Authors
  1. Md. Atik Ahamed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdullah Al Zubaer Imran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md. Atik Ahamed .

Editor information

Editors and Affiliations

  1. Imperial College London, London, UK

    Guang Yang

  2. University of Cambridge, Cambridge, UK

    Angelica Aviles-Rivero

  3. University of Cambridge, Cambridge, UK

    Michael Roberts

  4. University of Cambridge, Cambridge, UK

    Carola-Bibiane Schönlieb

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ahamed, M.A., Imran, A.A.Z. (2022). Joint Learning with Local and Global Consistency for Improved Medical Image Segmentation. In: Yang, G., Aviles-Rivero, A., Roberts, M., Schönlieb, CB. (eds) Medical Image Understanding and Analysis. MIUA 2022. Lecture Notes in Computer Science, vol 13413. Springer, Cham. https://doi.org/10.1007/978-3-031-12053-4_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-12053-4_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-12052-7

  • Online ISBN: 978-3-031-12053-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation