Skip to main content
SpringerLink
Log in

Medical Image Segmentation: A Review of Modern Architectures

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13807))

Included in the following conference series:

Abstract

Medical image segmentation involves identifying regions of interest in medical images. In modern times, there is a great need to develop robust computer vision algorithms to perform this task in order to reduce the time and cost of diagnosis and thus to aid quicker prevention and treatment of a variety of diseases. The approaches presented so far, mainly follow the U-type architecture proposed along with the UNet model, they implement encoder-decoder type architectures with fully convolutional networks, and also transformer architectures, exploiting both attention mechanisms and residual learning, and emphasizing information gathering at different resolution scales. Many of these architectural variants achieve significant improvements in quantitative and qualitative results in comparison to the pioneer UNet, while some fail to outperform it. In this work, 11 models designed for medical image segmentation, as well as other types of segmentation, are trained, tested and evaluated on specific evaluation metrics, on four publicly available datasets related to gastric polyps and cell nuclei, which are first augmented to increase their size in an attempt to address the problem of the lack of a large amount of medical data. In addition, their generalizability and the effect of data augmentation on the scores of the experiments are also examined. Finally, conclusions on the performance of the models are provided and future extensions that can improve their performance in the task of medical image segmentation are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). Software available: https://www.tensorflow.org/

  2. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on u-net (R2U-Net) for medical image segmentation (2018). https://doi.org/10.48550/ARXIV.1802.06955, https://arxiv.org/abs/1802.06955

  3. Asadi-Aghbolaghi, M., Azad, R., Fathy, M., Escalera, S.: Multi-level context gating of embedded collective knowledge for medical image segmentation (2020). https://doi.org/10.48550/ARXIV.2003.05056, https://arxiv.org/abs/2003.05056

  4. Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., Gil, D., Rodríguez, C., Vilariño, F.: WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput. Med. Imaging Graph. 43, 99–111 (2015). https://doi.org/10.1016/j.compmedimag.2015.02.007, https://doi.org/10.1016/j.compmedimag.2015.02.007

  5. Caicedo, J.C., et al.: Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat. Methods 16(12), 1247–1253 (2019). https://doi.org/10.1038/s41592-019-0612-7

  6. Caliva, F., et al.: A deep learning approach to anomaly detection in nuclear reactors. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)

    Google Scholar 

  7. Cao, H., et al.: Swin-unet: unet-like pure transformer for medical image segmentation (2021). https://doi.org/10.48550/ARXIV.2105.05537, https://arxiv.org/abs/2105.05537

  8. Chen, J., et al.: Transunet: transformers make strong encoders for medical image segmentation (2021). https://doi.org/10.48550/ARXIV.2102.04306, https://arxiv.org/abs/2102.04306

  9. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs (2016). https://doi.org/10.48550/ARXIV.1606.00915, https://arxiv.org/abs/1606.00915

  10. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation (2018). https://doi.org/10.48550/ARXIV.1802.02611, https://arxiv.org/abs/1802.02611

  11. Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras

  12. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  13. Diakogiannis, F.I., Waldner, F., Caccetta, P., Wu, C.: Resunet-A: a deep learning framework for semantic segmentation of remotely sensed data (2019). https://doi.org/10.48550/ARXIV.1904.00592, https://arxiv.org/abs/1904.00592

  14. Gehlot, S., Gupta, A., Gupta, R.: EDNFC-net: convolutional neural network with nested feature concatenation for nuclei-instance segmentation. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, May 2020. https://doi.org/10.1109/icassp40776.2020.9053633

  15. Giusti, A., Cireşan, D.C., Masci, J., Gambardella, L.M., Schmidhuber, J.: Fast image scanning with deep max-pooling convolutional neural networks (2013). https://doi.org/10.48550/ARXIV.1302.1700, https://arxiv.org/abs/1302.1700

  16. GRNET: Aris documentation - hardware overview. https://doc.aris.grnet.gr/system/hardware/

  17. Gupta, A., et al.: GCTI-SN: geometry-inspired chemical and tissue invariant stain normalization of microscopic medical images. Med. Image Anal. 65, 101788 (2020). https://doi.org/10.1016/j.media.2020.101788

  18. Gupta, A., Gupta, R., Gehlot, S., Gehlot, S.: Segpc-2021: segmentation of multiple myeloma plasma cells in microscopic images (2021). https://doi.org/10.21227/7NP1-2Q42, https://ieee-dataport.org/open-access/segpc-2021-segmentation-multiple-myeloma-plasma-cells-microscopic-images

  19. Gupta, A., Mallick, P., Sharma, O., Gupta, R., Duggal, R.: PCSeg: color model driven probabilistic multiphase level set based tool for plasma cell segmentation in multiple myeloma. PLOS ONE 13(12), e0207908 (2018). https://doi.org/10.1371/journal.pone.0207908

  20. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks (2017). https://doi.org/10.48550/ARXIV.1709.01507, https://arxiv.org/abs/1709.01507

  21. Huang, H., et al.: Unet 3+: a full-scale connected unet for medical image segmentation (2020). https://doi.org/10.48550/ARXIV.2004.08790, https://arxiv.org/abs/2004.08790

  22. Ibtehaz, N., Rahman, M.S.: Multiresunet: rethinking the u-net architecture for multimodal biomedical image segmentation (2019). https://doi.org/10.48550/ARXIV.1902.04049, https://arxiv.org/abs/1902.04049

  23. Jha, D., et al.: A comprehensive study on colorectal polyp segmentation with resunet, conditional random field and test-time augmentation. IEEE J. Biomed. Health Inform. 25(6), 2029–2040 (2021). https://doi.org/10.1109/jbhi.2021.3049304

  24. Jha, D., et al.: Kvasir-SEG: a segmented polyp dataset. In: Ro, Y.M., et al. (eds.) MMM 2020. LNCS, vol. 11962, pp. 451–462. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_37

    Chapter  Google Scholar 

  25. Jha, D., et al.: Resunet++: an advanced architecture for medical image segmentation (2019). https://doi.org/10.48550/ARXIV.1911.07067, https://arxiv.org/abs/1911.07067

  26. Kollia, I., Stafylopatis, A.G., Kollias, S.: Predicting Parkinson’s disease using latent information extracted from deep neural networks. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)

    Google Scholar 

  27. Kollias, D., Arsenos, A., Kollias, S.: AI-Mia: Covid-19 detection & severity analysis through medical imaging. arXiv preprint arXiv:2206.04732 (2022)

  28. Kollias, D., Arsenos, A., Soukissian, L., Kollias, S.: Mia-Cov19d: Covid-19 detection through 3-D chest CT image analysis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 537–544 (2021)

    Google Scholar 

  29. Kollias, D., Tagaris, A., Stafylopatis, A., Kollias, S., Tagaris, G.: Deep neural architectures for prediction in healthcare. Complex Intell. Syst. 4(2), 119–131 (2018)

    Article  Google Scholar 

  30. Kollias, D., Yu, M., Tagaris, A., Leontidis, G., Stafylopatis, A., Kollias, S.: Adaptation and contextualization of deep neural network models. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2017)

    Google Scholar 

  31. Kollias, D., et al.: Transparent adaptation in deep medical image diagnosis. In: Heintz, F., Milano, M., O’Sullivan, B. (eds.) TAILOR 2020. LNCS (LNAI), vol. 12641, pp. 251–267. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73959-1_22

    Chapter  Google Scholar 

  32. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation (2014). https://doi.org/10.48550/ARXIV.1411.4038, https://arxiv.org/abs/1411.4038

  33. Holschneider, M., Kronland-Martinet, R., Morlet, J., Tchamitchian, P.: A real-time algorithm for signal analysis with the help of the wavelet transform. In: Combes, J.M., Grossmann, A., Tchamitchian, P. (eds.) Wavelets. Inverse Problems and Theoretical Imaging, pp. 289–297. Springer, Heidelberg (1990). http://kronland.fr/wp-content/uploads/2015/05/RealTimeAlgo_Springer89.pdf. https://doi.org/10.1007/978-3-642-75988-8_28

  34. Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation (2016). https://doi.org/10.48550/ARXIV.1606.04797, https://arxiv.org/abs/1606.04797

  35. Oktay, O., et al.: Attention U-net: learning where to look for the pancreas (2018). https://doi.org/10.48550/ARXIV.1804.03999, https://arxiv.org/abs/1804.03999

  36. Papandreou, G., Kokkinos, I., Savalle, P.A.: Modeling local and global deformations in deep learning: epitomic convolution, multiple instance learning, and sliding window detection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2015. https://doi.org/10.1109/cvpr.2015.7298636

  37. Psaroudakis, A., Kollias, D.: Mixaugment & mixup: augmentation methods for facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2367–2375 (2022)

    Google Scholar 

  38. Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O.R., Jagersand, M.: U\(^2\)-net: going deeper with nested U-structure for salient object detection (2020). https://doi.org/10.48550/ARXIV.2005.09007, https://arxiv.org/abs/2005.09007

  39. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  40. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks (2013). https://doi.org/10.48550/ARXIV.1312.6229, https://arxiv.org/abs/1312.6229

  41. Srivastava, A., et al.: MSRF-net: a multi-scale residual fusion network for biomedical image segmentation (2021). https://doi.org/10.48550/ARXIV.2105.07451, https://arxiv.org/abs/2105.07451

  42. Tagaris, A., Kollias, D., Stafylopatis, A.: Assessment of Parkinson’s disease based on deep neural networks. In: Boracchi, G., Iliadis, L., Jayne, C., Likas, A. (eds.) EANN 2017. CCIS, vol. 744, pp. 391–403. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65172-9_33

    Chapter  Google Scholar 

  43. Tagaris, A., Kollias, D., Stafylopatis, A., Tagaris, G., Kollias, S.: Machine learning for neurodegenerative disorder diagnosis-survey of practices and launch of benchmark dataset. Int. J. Artif. Intell. Tools 27(03), 1850011 (2018)

    Article  Google Scholar 

  44. Takikawa, T., Acuna, D., Jampani, V., Fidler, S.: Gated-SCNN: gated shape CNNs for semantic segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, October 2019. https://doi.org/10.1109/iccv.2019.00533

  45. Wang, J., et al.: Deep high-resolution representation learning for visual recognition (2019). https://doi.org/10.48550/ARXIV.1908.07919, https://arxiv.org/abs/1908.07919

  46. Wingate, J., Kollia, I., Bidaut, L., Kollias, S.: Unified deep learning approach for prediction of Parkinson’s disease. IET Image Proc. 14(10), 1980–1989 (2020)

    Article  Google Scholar 

  47. Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2 (2019). https://github.com/facebookresearch/detectron2

  48. Xu, Q., Duan, W., He, N.: DCSAU-net: a deeper and more compact split-attention u-net for medical image segmentation (2022). https://doi.org/10.48550/ARXIV.2202.00972, https://arxiv.org/abs/2202.00972

  49. Zhang, Z., Liu, Q., Wang, Y.: Road extraction by deep residual u-net (2017). https://doi.org/10.48550/ARXIV.1711.10684, https://arxiv.org/abs/1711.10684

  50. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was conducted in the Artificial Intelligence and Learning Systems Laboratory of the School of Electrical and Computer Engineering of the National Technical University of Athens. The computations in this paper were performed on equipment provided by the Greek Research and Technology Network.

Author information

Authors and Affiliations

  1. National Technical University of Athens, Athens, Greece

    Natalia Salpea & Paraskevi Tzouveli

  2. Queen Mary University of London, London, UK

    Dimitrios Kollias

Authors
  1. Natalia Salpea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paraskevi Tzouveli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dimitrios Kollias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Natalia Salpea .

Editor information

Editors and Affiliations

  1. IBM Research - MIT-IBM Watson AI Lab, Massachusetts, USA

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Salpea, N., Tzouveli, P., Kollias, D. (2023). Medical Image Segmentation: A Review of Modern Architectures. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13807. Springer, Cham. https://doi.org/10.1007/978-3-031-25082-8_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25082-8_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25081-1

  • Online ISBN: 978-3-031-25082-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation