Skip to main content
SpringerLink
Log in

A state-of-the-art survey of U-Net in microscopic image analysis: from simple usage to structure mortification

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Microscopic image analysis technology helps solve the inadvertences of artificial traditional methods in disease, wastewater treatment, and environmental change monitoring analysis. Convolutional neural network (CNN) play an important role in microscopic image analysis. Image segmentation, in which U-Net is increasingly applied in microscopic image segmentation, is a crucial step in detection, tracking, monitoring, feature extraction, modelling, and analysis. This paper comprehensively reviews the development history of U-Net, analyses several research results of various segmentation methods since the emergence of U-Net, and conducts a comprehensive review of related papers. This paper summarised the improved methods of U-Net and then listed the existing significance of image segmentation techniques and their improvements introduced over the years. Finally, focusing on the different improvement strategies of U-Net in different papers, the related work of each application target is reviewed according to detailed technical categories to facilitate future research. Researchers can see the dynamics of the transmission of technological development and keep up with future trends in this interdisciplinary field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Data availability

This paper is a survey article, and there is no experimental work in it, so we did not use any data in this paper.

References

  1. Wu Q, Merchant F, Castleman K (2010) Microscope image processing. Elsevier, Amsterdam

    Google Scholar 

  2. Zenhausern F, Boyle M, Wickramasinghe H (1994) Apertureless near-field optical microscope. Appl Phys Lett 65(13):1623–1625

    ADS  CAS  Google Scholar 

  3. Toledo-Crow R, Yang P, Chen Y, Vaez-Iravani M (1992) Near-field differential scanning optical microscope with atomic force regulation. Appl Phys Lett 60(24):2957–2959

    ADS  CAS  Google Scholar 

  4. Inouye Y, Kawata S (1994) Near-field scanning optical microscope with a metallic probe tip. Opt Lett 19(3):159–161

    ADS  CAS  PubMed  Google Scholar 

  5. Williams D, Carter C (1996) The transmission electron microscope. Springer, Berlin, pp 3–17

    Google Scholar 

  6. Seiler H (1983) Secondary electron emission in the scanning electron microscope. J Appl Phys 54(11):R1–R18

    ADS  CAS  Google Scholar 

  7. Tersoff J, Hamann D (1985) Theory of the scanning tunneling microscope. Phys Rev B 31(2):805

    ADS  CAS  Google Scholar 

  8. Binnig G, Quate C, Gerber C (1986) Atomic force microscope. Phys Rev Lett 56(9):930

    ADS  CAS  PubMed  Google Scholar 

  9. Duncan M, Reintjes J, Manuccia T (1982) Scanning coherent anti-Stokes Raman microscope. Opt Lett 7(8):350–352

    ADS  CAS  PubMed  Google Scholar 

  10. Adrian M, Dubochet J, Lepault J, McDowall A (1984) Cryo-electron microscopy of viruses. Nature 308(5954):32–36

    ADS  CAS  PubMed  Google Scholar 

  11. Li C, Zhang J, Kulwa F, Qi S, Qi Z (2020) A SARS-CoV-2 microscopic image dataset with ground truth images and visual features. In Proc. of PRCV 2020, pp 244–255

  12. Li C, Wang K, Xu N (2019) A survey for the applications of content-based microscopic image analysis in microorganism classification domains. Artif Intell Rev 51(4):577–646

    Google Scholar 

  13. Zhang J, Li C, Kosov S, Grzegorzek M, Shirahama K, Jiang T, Sun C, Li Z, Li H (2021) LCU-Net: a novel low-cost U-Net for environmental microorganism image segmentation. Pattern Recogn 115:107885

    Google Scholar 

  14. Li X, Li C, Rahaman M, Li X, Sun H, Zhang H, Zhang Y, Li X, Wu J, Yao Y (2022) A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification, and detection approaches. Art Intell Rev 55(6):4809–4878

    Google Scholar 

  15. Zhouand X, Li C, Rahaman M, Yao Y, Ai S, Sun C, Wang Q, Zhang Y, Li M, Li X (2020) A comprehensive review for breast histopathology image analysis using classical and deep neural networks. IEEE Access 8:90931–90956

    Google Scholar 

  16. Rahaman M, Li C, Wu X, Yao Y, Hu Z, Jiang T, Li X, Qi S (2020) A survey for cervical cytopathology image analysis using deep learning. IEEE Access 8:61687–61710

    Google Scholar 

  17. Hore S, Chakroborty S, Ashour A, Dey N, Ashour A, Sifaki-Pistolla D, Bhattacharya T, Chaudhuri S (2015) Finding contours of hippocampus brain cell using microscopic image analysis. J Adv Microsc Res 10(2):93–103

    Google Scholar 

  18. Øien S, Wragg D, Reinsch H, Svelle S, Bordiga S, Lamberti C, Lillerud K (2014) Detailed structure analysis of atomic positions and defects in zirconium metal-organic frameworks. Crystal Growth Des 14(11):5370–5372

    Google Scholar 

  19. Clelland W, Fens T (1991) Automated rock characterization with SEM/image-analysis techniques. SPE Form Eval 6(04):437–443

    CAS  Google Scholar 

  20. Pagliai M, Vignozzi N (2002) Image analysis and microscopic techniques to characterize soil pore system. In: Blahovec J, Kutilek M (eds) Physical methods in agriculture. Springer, Berlin, pp 13–38

    Google Scholar 

  21. Abell A, Willis K, Lange D (1999) Mercury intrusion porosimetry and image analysis of cement-based materials. J Colloid Interface Sci 211(1):39–44

    ADS  CAS  PubMed  Google Scholar 

  22. Nilsson H (1995) Remote sensing and image analysis in plant pathology. Annu Rev Phytopathol 33(1):489–528

    CAS  PubMed  Google Scholar 

  23. Nilsson N (2014) Principles of artificial intelligence. Morgan Kaufmann, Burlington

    Google Scholar 

  24. Litjens G, Kooi T, Bejnordi B, Setio A, Ciompi F, Ghafoorian M, Laak J, Ginneken B, Sánchez C (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    PubMed  Google Scholar 

  25. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In Proc. of ICMICCA 2015, pp 234–241

  26. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proc. of CVPR 2015, pp 3431–3440

  27. Zhang J, Li C, Rahaman MM, Yao Y, Ma P, Zhang J, Zhao X, Jiang T, Grzegorzek M (2023) A comprehensive survey with quantitative comparison of image analysis methods for microorganism biovolume measurements. Arch Comput Methods Eng 30(1):639–673

    PubMed  Google Scholar 

  28. Weiming Hu, Li Xintong, Li Chen, Li Rui, Jiang Tao, Sun Hongzan, Huang Xinyu, Grzegorzek Marcin, Li Xiaoyan (2023) A state-of-the-art survey of artificial neural networks for whole-slide image analysis: from popular convolutional neural networks to potential visual transformers. Comput Biol Med 161:107034

    Google Scholar 

  29. Ma Pingli, Li Chen, Rahaman Md Mamunur, Yao Yudong, Zhang Jiawei, Zou Shuojia, Zhao Xin, Grzegorzek Marcin (2023) A state-of-the-art survey of object detection techniques in microorganism image analysis: from classical methods to deep learning approaches. Artif Intell Rev 56(2):1627–1698

    PubMed  Google Scholar 

  30. Zhang J, Li C, Rahaman MM, Yao Y, Ma P, Zhang J, Zhao X, Jiang T, Grzegorzek M (2022) A comprehensive review of image analysis methods for microorganism counting: from classical image processing to deep learning approaches. Art Intell Rev 55(4):2875–2944

    Google Scholar 

  31. Taghanaki S, Abhishek K, Cohen J, Cohen-Adad J, Hamarneh G (2020) Deep semantic segmentation of natural and medical images: a review. Art Intell Rev 54:137–178

    Google Scholar 

  32. Du G, Cao X, Liang J, Chen X, Zhan Y (2020) Medical image segmentation based on u-net: a review. J Imaging Sci Technol 64(2):20508

    Google Scholar 

  33. Colonna A, Scarpa F, Ruggeri A (2018) Segmentation of corneal nerves using a u-net-based convolutional neural network. In: Computational pathology and ophthalmic medical image analysis, pp 185–192

  34. Seong S, Park H (2019) Automated identification of neural cells in the multi-photon images using deep-neural networks. arXiv: 1909.11269

  35. Daniel M, Atzrodt L, Bucher F, Wacker K, Böhringer S, Reinhard T, Böhringer D (2019) Automated segmentation of the corneal endothelium in a large set of ‘real-world’specular microscopy images using the U-Net architecture. Sci Rep 9(1):1–7

    ADS  Google Scholar 

  36. Núñez-Fernández D, Ballan L, Jiménez-Avalos G, Coronel J, Zimic M(2020) Automatic semantic segmentation for prediction of tuberculosis using lens-free microscopy images. arXiv: 2007.02482

  37. Ojeda-Pat A, Martin-Gonzalez A, Soberanis-Mukul R (2020) Convolutional Neural Network U-Net for Trypanosoma cruzi Segmentation. In: Proc. of ISICS 2020, pp 118–131

  38. Chen Z, Liu X, Yang J, Little E, Zhou Y (2020) Deep learning-based method for SEM image segmentation in mineral characterization, an example from Duvernay Shale samples in Western Canada Sedimentary Basin. Comput. Geosci. 138:104450

    Google Scholar 

  39. Oktay A, Gurses A (2019) Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images. Micron 120:113–119

    CAS  PubMed  Google Scholar 

  40. Farley S, Hodgkinson J, Gordon O, Turner J, Soltoggio A, Moriarty P, Hunsicker E (2020) Improving the segmentation of scanning probe microscope images using convolutional neural networks. Mach Learn Sci Technol 2(1):015015

    Google Scholar 

  41. Jaworek-Korjakowska J (2018) A deep learning approach to vascular structure segmentation in dermoscopy colour images. BioMed Res. Int. 2018:5049390–5049390

    PubMed  PubMed Central  Google Scholar 

  42. Meyer M, Costa P, Galdran A, Mendonça A, Campilho A (2017) A deep neural network for vessel segmentation of scanning laser ophthalmoscopy images. In: Proc. of ICIAR 2017, pp 507–515

  43. Webb R, Hughes G, Pomerantzeff O (1980) Flying spot TV ophthalmoscope. Appl Opt 19(17):2991–2997

    ADS  CAS  PubMed  Google Scholar 

  44. Zhang J, Dashtbozorg B, Bekkers E, Pluim J, Duits R, Romeny B (2016) Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE Trans Med Imaging 35(12):2631–2644

    PubMed  Google Scholar 

  45. Swiderska-Chadaj Z, Markiewicz T, Gallego J, Bueno G, Grala B, Lorent M (2018) Deep learning for damaged tissue detection and segmentation in Ki-7 brain tumor specimens based on the U-net model. Bulletin of the polish academy of sciences. Tech Sci 66(6):849–856

    Google Scholar 

  46. learning dense volumetric segmentation from sparse annotation (2016) Özgün Çiçek, A. Abdulkadir, S. Lienkamp, T. Brox, and O. Ronneberger. 3D U-Net. In: Proceeding of ICMICCAI 2016, pp 424–432

  47. Fang Z, Yue W, Zhitao X, Lei G, Jun W, Yanbei L, Wen W (2019) Nanoparticle segmentation based on U-Net convolutional neural network. Laser Optoelectron. Progr 56(6):061005

    Google Scholar 

  48. Fu C, Lee S, Ho D, Han S, Salama P, Dunn K, Delp E (2018) Three dimensional fluorescence microscopy image synthesis and segmentation. In: Proceeding of CVPR 2018, pp 2221–2229

  49. Eschweiler D, Spina T, Choudhury R, Meyerowitz E, Cunha A, Stegmaier J (2019) CNN-based preprocessing to optimize watershed-based cell segmentation in 3D confocal microscopy images. In: Proceeding of ISBI 2019, pp 223–227

  50. Willis L, Refahi Y, Wightman R, Landrein B, José Teles, Huang K, Meyerowitz E, Henrik Jönsson (2016) Cell size and growth regulation in the Arabidopsis thaliana apical stem cell niche. Proc Natl Acad Sci 113(51):E8238–E8246

    ADS  CAS  PubMed  PubMed Central  Google Scholar 

  51. Fernandez R, Das P, Mirabet V, Moscardi E, Traas J, Verdeil J, Malandain G, Godin C (2010) Imaging plant growth in 4D: robust tissue reconstruction and lineaging at cell resolution. Nat Methods 7(7):547

    CAS  PubMed  Google Scholar 

  52. Mosaliganti K, Noche R, Xiong F, Swinburne I, Megason S (2012) ACME: automated cell morphology extractor for comprehensive reconstruction of cell membranes. PLoS Comput Biol 8(12):e1002780

    ADS  CAS  PubMed  PubMed Central  Google Scholar 

  53. Heinrich L, Funke J, Pape C, Nunez-Iglesias J, Saalfeld S (2018) Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete drosophila brain. In: Proceeding of ICMICCAI 2018, pp 317–325

  54. Wang H, Zhang D, Song Y, Liu S, Wang Y, Feng D, Peng H, Cai W (2019) Segmenting neuronal structure in 3D optical microscope images via knowledge distillation with teacher-student network. In: Proceeding of ISBI 2019, pp 228–231

  55. Zhang M, Li X, Xu M, Li Q (2017) Image segmentation and classification for sickle cell disease using deformable u-net. arXiv: 1710.08149

  56. Zhang M, Li X, Xu X, Li Q (2018) RBC semantic segmentation for sickle cell disease based on deformable U-Net. In: Proceeding of ICMICCAI 2018, pp 695–702

  57. Xu X, Papageorgiou D, Abidi S, Dao M, Zhao H, Karniadakis G (2017) A deep convolutional neural network for classification of red blood cells in sickle cell anemia. PLoS Comput Biol 13(10):e1005746

    ADS  PubMed  PubMed Central  Google Scholar 

  58. Qin X, Wu C, Chang H, Lu H, Zhang X (2020) Match feature U-Net: dynamic receptive field networks for biomedical image segmentation. Symmetry 12(8):1230

    ADS  Google Scholar 

  59. Rad R, Saeedi P, Au J, Havelock J (2018) Blastomere cell counting and centroid localization in microscopic images of human embryo. In: Proceeding of MMSP 2018, pp 1–6

  60. Rad R, Saeedi P, Au J, Havelock J (2020) Trophectoderm segmentation in human embryo images via inceptioned U-Net. Med Image Anal 62:101612

    PubMed  Google Scholar 

  61. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In Proceeding of CVPR 2015:1–9

    Google Scholar 

  62. Saeedi P, Yee D, Au J, Havelock J (2017) Automatic identification of human blastocyst components via texture. IEEE Trans Biomed Eng 64(12):2968–2978

    PubMed  Google Scholar 

  63. Matuszewski D, Sintorn I (2018) Minimal annotation training for segmentation of microscopy images. In: Proceeding of ISBI 2018, pp 387–390

  64. Kylberg G, Uppström M, HEDLUND K, Borgefors G, SINTORN I (2012) Segmentation of virus particle candidates in transmission electron microscopy images. J Microsc 245(2):140–147

    CAS  PubMed  Google Scholar 

  65. Mocan I, Itu R, Ciurte A, Danescu R, Buiga R (2018) Automatic Detection of Tumor Cells in Microscopic Images of Unstained Blood using Convolutional Neural Networks. In: Proceeding of ICCP 2018, pp 319–324

  66. Xu Z, Sobhani F, Moro C, Zhang Q (2019) Us-net for robust and efficient nuclei instance segmentation. In: Proceeding of ISBI 2019, pp 44–47

  67. Li W, Qian X, Ji J (2017) Noise-tolerant deep learning for histopathological image segmentation. In: Proc. of ICIP 2017, pp 3075–3079

  68. Fabijańska A (2018) Segmentation of corneal endothelium images using a U-Net-based convolutional neural network. Artif Intell Med 88:1–13

    PubMed  Google Scholar 

  69. Kumar C, TN M, Narasimhadhan A (2020) Cell Segmentation by Modified U-Net Architecture for Biomedical Images. In: Proceeding of CONECCT 2020, pp 1–6

  70. Bermúdez-Chacón R, Márquez-Neila P, Salzmann M, Fua P (2018) A domain-adaptive two-stream U-Net for electron microscopy image segmentation. In: Proceeding of ISBI 2018, pp 400–404

  71. Jha D, Riegler M, Johansen D, Halvorsen P, Johansen H (2020) Doubleu-net: A deep convolutional neural network for medical image segmentation. In: Proceeding of CBMS 2020, pp 558–564

  72. Zhuang J (2018) Laddernet: Multi-path networks based on u-net for medical image segmentation. arXiv: 1810.07810

  73. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv: 1409.1556

  74. Bernal J, Sánchez F, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111

    PubMed  Google Scholar 

  75. Torr A, Basaran D, Sero J, Rittscher J, Sailem H (2020) DeepSplit: Segmentation of Microscopy Images Using Multi-Task Convolutional Networks. In: Annual conference on medical image understanding and analysis, pp 155–167

  76. Bozkurt A, Kose K, Alessi-Fox C, Gill M, Dy J, Brooks D, Rajadhyaksha M (2018) A multiresolution convolutional neural network with partial label training for annotating reflectance confocal microscopy images of skin. In: Proceeding of ICMICCAI 2018, pp 292–299

  77. Zhao B, Chen X, Li Z, Yu Z, Yao S, Yan L, Wang Y, Liu Z, Liang C, Han C (2020) Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation. Med Image Anal 65:101786

    PubMed  Google Scholar 

  78. Huang G, Liu Z, Maaten L, Weinberger K (2017) Densely connected convolutional networks. In: Proceeding of CVPR 2017, pp 4700–4708

  79. Vahadane Abhishek, Atheeth B, Majumdar Shantanu (2021) Dual encoder attention u-net for nuclei segmentation. In: 2021 43rd annual international conference of the IEEE engineering in medicine & biology society (EMBC), IEEE, pp 3205–3208

  80. Dang Vu Quoc, Simon Graham, Tahsin Kurc, Nhat To Minh Nguyen, Muhammad Shaban, Talha Qaiser, Alemi Koohbanani Navid, Ali Khurram Syed, Jayashree Kalpathy-Cramer, Tianhao Zhao et al (2019) Methods for segmentation and classification of digital microscopy tissue images. Front Bioeng Biotechnol 7:53

    Google Scholar 

  81. Oktay O, Schlemper J, Folgoc L, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla N, Kainz B (2018) Attention u-net: Learning where to look for the pancreas. arXiv: 1804.03999

  82. Lian S, Luo Z, Zhong Z, Lin X, Su S, Li S (2018) Attention guided U-Net for accurate iris segmentation. J Vis Commun Image Represent 56:296–304

    Google Scholar 

  83. Proença H, Filipe S, Santos R, Oliveira J, Alexandre L (2009) The UBIRIS. v2: a database of visible wavelength iris images captured on-the-move and at-a-distance. IEEE Trans Pattern Anal Mach Intell 32(8):1529–1535

    Google Scholar 

  84. Lv Y, Ma H, Li J, Liu S (2020) Attention guided U-Net with atrous convolution for accurate retinal vessels segmentation. IEEE Access 8:32826–32839

    Google Scholar 

  85. Bansal N, Dutta M (2013) Retina vessels detection algorithm for biomedical symptoms diagnosis. Int J Comput Appl, 71(20)

  86. Guo Y, Budak Ü, Vespa L, Khorasani E, Şengür A (2018) A retinal vessel detection approach using convolution neural network with reinforcement sample learning strategy. Measurement 125:586–591

    ADS  Google Scholar 

  87. Thangaraj S, Periyasamy V, Balaji R (2018) Retinal vessel segmentation using neural network. IET Image Proc 12(5):669–678

    Google Scholar 

  88. Mou L, Zhao Y, Chen L, Cheng J, Gu Z, Hao H, Qi H, Zheng Y, Frangi A, Liu J (2019) CS-Net: channel and spatial attention network for curvilinear structure segmentation. In: Proceeding of ICMICCAI 2019, pp 721–730

  89. Li R, Li M, Li J, Zhou Y (2019) Connection sensitive attention u-net for accurate retinal vessel segmentation. arXiv:1903.05558

  90. Jiang Y, Wang F, Gao J, Cao S (2020) Multi-path recurrent u-net segmentation of retinal fundus image. Appl Sci 10(11):3777

    CAS  Google Scholar 

  91. Zhang H, Zhu H, Ling X (2020) Polar coordinate sampling-based segmentation of overlapping cervical cells using attention U-Net and random walk. Neurocomputing 383:212–223

    Google Scholar 

  92. Zhu N, Liu C, Singer Z, Danino T, Laine A, Guo J (2020) Segmentation with residual attention u-net and an edge-enhancement approach preserves cell shape features. arXiv:2001.05548

  93. Sivaswamy J, Krishnadas S, Chakravarty A, Joshi G, Tabish A (2015) A comprehensive retinal image dataset for the assessment of glaucoma from the optic nerve head analysis. JSM Biomed Imaging Data Papers 2(1):1004

    Google Scholar 

  94. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceeding CVPR 2016, pp 770–778

  95. Xiancheng W, Wei L, Bingyi M, He J, Jiang Z, Xu W, Ji Z, Hong G, Zhaomeng S (2018) Retina blood vessel segmentation using a U-net based convolutional neural network. In: Procedia Computer Science, pp 1–11

  96. Leng J, Liu Y, Zhang T, Quan P, Cui Z (2018) Context-aware u-net for biomedical image segmentation. In: Proceeding of BIBM 2018, pp 2535–2538

  97. Chidester B, Ton T, Tran M, Ma J, Do M (2019) Enhanced rotation-equivariant u-net for nuclear segmentation. In: Proceeding of CVPRW 2019

  98. Zhang J, Li C, Kosov S, Grzegorzek M, Shirahama K, Jiang T, Sun C, Li Zihan, Li H (2021) LCU-Net: A novel low-cost U-Net for environmental microorganism image segmentation. Pattern Recogn 115:107885

    Google Scholar 

  99. Petit Olivi, Thome N, Rambour C, Themyr L, Collins T, Soler L (2021) U-net transformer: Self and cross attention for medical image segmentation. In: Machine Learning in medical imaging: 12th international workshop, MLMI 2021, held in conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings vol 12, Springer, pp 267–276.

  100. Huang G, Chen D, Li T, Wu F, Maaten L, Weinberger K (2017) Multi-scale dense convolutional networks for efficient prediction. arXiv: 1703.09844, 2

  101. Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In: Proceeding of CVPR 2017, pp 11–19

  102. Cheng Y, Ma M, Zhang L, Jin C, Ma L, Zhou Y (2020) Retinal blood vessel segmentation based on Densely Connected U-Net. Math Biosci Eng 17:3088–3108

    PubMed  Google Scholar 

  103. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceeding of ICCV 2015, pp 1026–1034

  104. Owen C, Rudnicka A, Mullen R, Barman S, Monekosso D, Whincup P, Ng J, Paterson C (2009) Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (CAIAR) program. Investig Ophthalmol Visual Sci 50(5):2004–2010

    Google Scholar 

  105. Wang C, Zhao Z, Ren Q, Xu Y, Yu Y (2019) Dense U-net based on patch-based learning for retinal vessel segmentation. Entropy 21(2):168

    ADS  MathSciNet  PubMed  PubMed Central  Google Scholar 

  106. Samanta P, Raipuria G, Singhal N (2021) Context Aggregation Network For Semantic Labeling In Histopathology Images. In: Proceeding of ISBI 2021, pp 673–676

  107. Li J, Yang S, Huang X, Da Q, Yang X, Hu Z, Duan Q, Wang C, Li H (2019) Signet ring cell detection with a semi-supervised learning framework. In: Proceeding of ICIPMI 2019, pp 842–854

  108. Sirinukunwattana K, Pluim J, Chen H, Qi X, Heng P, Guo Y, Wang L, Matuszewski B, Bruni E, Sanchez U (2017) Gland segmentation in colon histology images: the glas challenge contest. Med Image Anal 35:489–502

    PubMed  Google Scholar 

  109. Zhang J, Jin Y, Xu J, Xu X, Zhang Y (2018) Mdu-net: Multi-scale densely connected u-net for biomedical image segmentation. arXiv:1812.00352

  110. Liu Y, Treible W, Kolagunda A, Nedo A, Saponaro P, Caplan J, Kambhamettu C (2018) Densely connected stacked u-network for filament segmentation in microscopy images. In: Proceeding of ECCV 2018

  111. Wu C, Xie Y, Shao L, Yang J, Ai D, Song H, Wang Y, Huang Y (2019) Automatic boundary segmentation of vascular doppler optical coherence tomography images based on cascaded U-net architecture. OSA Contin 2(3):677–689

    Google Scholar 

  112. Zhou Z, RSiddiquee M, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp 3–11

  113. Zhou Z, Siddiquee M, Tajbakhsh N, Liang J (2019) Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867

    PubMed  PubMed Central  Google Scholar 

  114. Wang H, Li Y, Luo Z (2020) An Improved Breast Cancer Nuclei Segmentation Method Based on UNet++. In: Proceeding of ICCAI 2020, pp 193–197

  115. Janowczyk A, Madabhushi A (2016) Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Inform 7:29

    PubMed  PubMed Central  Google Scholar 

  116. Tsunomura M, Shishikura M, Ishii T, Takahashi R, Tsumura N (2020) Segmentation of microscopic image of colorants using u-net based deep convolutional networks for material appearance design. In: International conference on image and signal processing, pp 197–204

  117. Xu X, Tan T, Xu F (2018) An improved U-net architecture for simultaneous arteriole and venule segmentation in fundus image. In: Proceeding of ACMIUA 2018, pp 333–340

  118. Liang X, Wang B (2020) Wheat powdery mildew spore images segmentation based on U-Net. J Phys Conf Series 1631:012074

    Google Scholar 

  119. Patel G, Tekchandani H, Verma S (2019) Cellular segmentation of bright-field absorbance images using residual u-net. In: Proceeding of ICAC 2019, pp 1–5

  120. Gómez de Mariscal E, Maška M, Kotrbová A, Pospíchalová V, Matula P, Muñoz-Barrutia A (2019) Deep-learning-based segmentation of small extracellular vesicles in transmission electron microscopy images. Sci Rep 9(1):1–10

    Google Scholar 

  121. Quan T, Hildebrand D, Jeong W (2016) Fusionnet: A deep fully residual convolutional neural network for image segmentation in connectomics. arXiv:1612.05360

  122. Arganda-Carreras I, Turaga S, Berger D, Cireşan D, Giusti A, Gambardella L, Schmidhuber J, Laptev D, Dwivedi S, Buhmann J (2015) Crowdsourcing the creation of image segmentation algorithms for connectomics. Front Neuroanat 9:142

    PubMed  PubMed Central  Google Scholar 

  123. Mehta S, Mercan E, Bartlett J, Weaver D, Elmore JG, Shapiro L (2018) Y-Net: joint segmentation and classification for diagnosis of breast biopsy images. In: Proceeding of ICMICCAI 2018, pp 893–901

  124. Ke R, Bugeau A, Papadakis N, Schuetz P, Schönlieb C (2019) A multi-task U-net for segmentation with lazy labels. arXiv preprint arXiv:1906.12177

  125. Wang Zekun, Zou Yanni, Liu Peter X (2021) Hybrid dilation and attention residual U-Net for medical image segmentation. Comput Biol Med 134:104449

    PubMed  Google Scholar 

  126. Zunair Hasib, Hamza A Ben (2021) Sharp U-Net: Depthwise convolutional network for biomedical image segmentation. Comput Biol Med 136:104699

    PubMed  Google Scholar 

  127. Yang Zhenzhen, Pengfei Xu, Yang Yongpeng, Bao Bing-Kun (2021) A densely connected network based on U-Net for medical image segmentation. ACM Trans Multimed Comput Commun Appl 17(3):1–14

    Google Scholar 

  128. Ibtehaz N, Rahman M (2020) MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    PubMed  Google Scholar 

  129. Coelho LP, Shariff A, Murphy RF (2009) Nuclear segmentation in microscope cell images: a hand-segmented dataset and comparison of algorithms. In: Proceeding of ISBI 2009, pp 518–521

  130. Lou A, Guan S, Loew M (2021) DC-UNet: rethinking the U-Net architecture with dual channel efficient CNN for medical image segmentation. In: medical imaging 2021: image processing, vol 11596, pp 115962T

  131. Gadosey P, Li Y, Adjei E, Zhang T, Liu Z, Yamak P, Essaf F (2020) Sd-unet: stripping down u-net for segmentation of biomedical images on platforms with low computational budgets. Diagnostics 10(2):110

    PubMed  PubMed Central  Google Scholar 

  132. Arbelle A, Raviv T (2019) Microscopy cell segmentation via convolutional LSTM networks. In: Proceedimg of ISBI 2019, pp 1008–1012

  133. Abdallah H, Formosa B, Liyanaarachchi A, Saigh M, Silvers S, Arslanturk S, Taatjes D, Larsson L, Jena B, Gatti D (2020) Res-CR-Net, a residual network with a novel architecture optimized for the semantic segmentation of microscopy images. Mach Learn Sci Technol 1(4):045004

    Google Scholar 

  134. Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A (2017) A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imaging 36(7):1550–1560

    PubMed  Google Scholar 

  135. Chakravarty A, Sivaswamy J (2018) RACE-net: a recurrent neural network for biomedical image segmentation. IEEE J Biomed Health Inform 23(3):1151–1162

    PubMed  Google Scholar 

  136. Alom M, Hasan M, Yakopcic C, Taha T, Asari V (2018) Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:1802.06955

  137. Alom M, Yakopcic C, Hasan M, Taha T, Asari V (2019) Recurrent residual U-Net for medical image segmentation. J Med Imaging 6(1):014006

    Google Scholar 

  138. Alom M, Yakopcic C, Taha T, Asari V (2018) Nuclei segmentation with recurrent residual convolutional neural networks based U-Net (R2U-Net). In: Proceeding of NAECON 2018, pp 228–233

  139. Zahangir A, Yakopcic C, Taha T, Asari V (2018) microscopic nuclei classification, segmentation and detection with improved deep convolutional neural network (DCNN) approaches. arXiv e-prints, pages arXiv–1811

  140. Yang Q, Xu Z, Liao C, Cai J, Huang Y, Chen H, Tao X, Huang Z, Chen J, Dong J (2020) Epithelium segmentation and automated Gleason grading of prostate cancer via deep learning in label-free multiphoton microscopic images. J Biophotonics 13(2):e201900203

    PubMed  Google Scholar 

  141. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  142. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceeding of AAAI, vol 31, pp 2017

  143. Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J (2019) Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging 38(10):2281–2292

    PubMed  Google Scholar 

  144. Zhang Z, Yin F, Liu J, Wong W, Tan N, Lee B, Cheng J, Wong T (2010) Origa-light: an online retinal fundus image database for glaucoma analysis and research. In: Proceeding of AICEMB 2010, pp 3065–3068

  145. Zhang Z, Wu C, Coleman S, Kerr D (2020) DENSE-INception U-net for medical image segmentation. Comput Methods Progr Biomed 192:105395

    Google Scholar 

  146. Huang Y, Li X, Yan C, Liu L, Dai H (2020) MIRD-Net for Medical Image Segmentation. In: Pacific-Asia conference on knowledge discovery and data mining, pp 207–219

  147. Trimeche I, Rossant F, Bloch I, Pâques M (2020) Fully automatic CNN-based segmentation of retinal bifurcations in 2D adaptive optics ophthalmoscopy images. In: Proceeding of IPTA 2020, pp 1–6

  148. Chen A, Li C, Zou S, Rahaman Md M, Yao Y, Chen H, Yang H, Zhao P, Weiming H, Liu W et al (2022) SVIA dataset: a new dataset of microscopic videos and images for computer-aided sperm analysis. Biocybern Biomed Eng 42(1):204–214

    CAS  Google Scholar 

  149. Zhao P, Li C, Rahaman Md M, Hao X, Ma P, Yang H, Sun H, Jiang T, Ning X, Grzegorzek M (2022) Emds-6: environmental microorganism image dataset sixth version for image denoising, segmentation, feature extraction, classification, and detection method evaluation. Front Microbiol 13:829027

    PubMed  PubMed Central  Google Scholar 

  150. Yang H, Li C, Zhao X, Cai B, Zhang J, Ma P, Zhao P, Chen A, Jiang T, Sun H et al (2023) EMDS-7: environmental microorganism image dataset seventh version for multiple object detection evaluation. Front Microbiol 14:1084312

    PubMed  PubMed Central  Google Scholar 

  151. Weiming H, Li C, Rahaman Md M, Chen H, Liu W, Yao Y, Sun H, Grzegorzek M, Li X, 2023) EBHI: a new enteroscope biopsy histopathological H &E image dataset for image classification evaluation. Phys Med 107:102534

    Google Scholar 

  152. Shi L, Li X, Weiming H, Chen H, Chen J, Fan Z, Gao M, Jing Y, Guotao L, Ma D et al (2023) EBHI-Seg: a novel enteroscope biopsy histopathological hematoxylin and eosin image dataset for image segmentation tasks. Front Med 10:1114673

    Google Scholar 

  153. Weiming H, Li C, Li X, Rahaman Md M, Ma J, Zhang Y, Chen H, Liu W, Sun C, Yao Y et al (2022) GasHisSDB: a new gastric histopathology image dataset for computer aided diagnosis of gastric cancer. Comput Biol Med 142:105207

    Google Scholar 

  154. Piantadosi G, Sansone M, Sansone C (2018) Breast segmentation in mri via u-net deep convolutional neural networks. In: Proceeding of ICPR 2018, pp 3917–3922

  155. Wang F, Jiang R, Zheng L, Meng C, Biswal B (2019) 3d u-net based brain tumor segmentation and survival days prediction. In: Proceeding of MICCAI-BW 2019, pp 131–141

  156. Lee B, Yamanakkanavar N, Choi J (2020) Automatic segmentation of brain MRI using a novel patch-wise U-net deep architecture. PLoS ONE 15(8):e0236493

    CAS  PubMed  PubMed Central  Google Scholar 

  157. Rundo L, Han C, Nagano Y, Zhang J, Hataya R, Militello C, Tangherloni A, Nobile M, Ferretti C, Besozzi D (2019) USE-Net: incorporating squeeze-and-excitation blocks into U-net for prostate zonal segmentation of multi-institutional MRI datasets. Neurocomputing 365:31–43

    Google Scholar 

  158. Kermi A, Mahmoudi I, Khadir M (2018) Deep convolutional neural networks using U-Net for automatic brain tumor segmentation in multimodal MRI volumes. In: Proceeding of MICCAI-BW 2018, pp 37–48

  159. Chen Y, Cao Z, Cao C, Yang J, Zhang J (2018) A modified U-Net for brain Mr image segmentation. In: Proceeding of ICCCS 2018, pp 233–242

  160. Song T, Meng F, Rodriguez-Paton A, Li P, Zheng P, Wang X (2019) U-next: a novel convolution neural network with an aggregation u-net architecture for gallstone segmentation in CT images. IEEE Access 7:166823–166832

    Google Scholar 

  161. Roth H, Oda H, Zhou X, Shimizu N, Yang Y, Hayashi Y, Oda M, Fujiwara M, Misawa K, Mori K (2018) An application of cascaded 3D fully convolutional networks for medical image segmentation. Comput Med Imaging Graph 66:90–99

    PubMed  Google Scholar 

  162. Roth H, Shen C, Oda H, Oda M, Hayashi Y, Misawa K, Mori K (2018) Deep learning and its application to medical image segmentation. Med Imaging Technol 36(2):63–71

    Google Scholar 

  163. Chen Y, Lin Y, Wang C, Lee C, Lee W, Wang T, Chen C (2019) Coronary artery segmentation in cardiac CT angiography using 3D multi-channel U-net. arXiv:1907.12246

  164. Saeedizadeh N, Minaee S, Kafieh R, Yazdani S, Sonka M (2021) COVID TV-Unet: Segmenting COVID-19 chest CT images using connectivity imposed Unet. Comput Methods Progr Biomed Update 1:100007

    Google Scholar 

  165. Weng Y, Zhou T, Li Y, Qiu X (2019) Nas-unet: neural architecture search for medical image segmentation. IEEE Access 7:44247–44257

    Google Scholar 

  166. Almajalid R, Shan J, Du Y, Zhang M (2018) Development of a deep-learning-based method for breast ultrasound image segmentation. In: Proceeding of ICMLA 2018, pp 1103–1108

  167. Wang Y, Wei C, Wang Z, Lu Q, Wang C (2018) A more streamlined u-net for nerve segmentation in ultrasound images. In: Proceeding of CAC 2018, pp 101–104

  168. Li X, Hong Y, Kong D, Zhang X (2019) Automatic segmentation of levator hiatus from ultrasound images using U-net with dense connections. Phys Med Biol 64(7):075015

    PubMed  Google Scholar 

  169. Yang J, Faraji M, Basu A (2019) Robust segmentation of arterial walls in intravascular ultrasound images using Dual Path U-Net. Ultrasonics 96:24–33

    PubMed  Google Scholar 

  170. Amiri M, Brooks R, Behboodi B, Rivaz H (2020) Two-stage ultrasound image segmentation using U-Net and test time augmentation. Int J Comput Assist Radiol Surg 15(6):981–988

    PubMed  Google Scholar 

  171. Amiri M, Brooks R, Rivaz H (2019) Fine tuning u-net for ultrasound image segmentation: Which layers? In: Domain adaptation and representation transfer and medical image learning with less labels and imperfect data, pp 235–242

  172. Wu J, Liu W, Li C, Jiang T, Shariful IM, Sun H, Li X, Li X, Huang X, Grzegorzek M (2022) A state-of-the-art survey of u-net in microscopic image analysis: from simple usage to structure mortification. arXiv:2202.06465

Download references

Acknowledgements

This work is supported by the “National Natural Science Foundation of China” (No. 82220108007). We thank Miss. Zixian Li and Mr. Guoxian Li for their important discussion in this work. We thank B.A. Qi Qiu and B.A. Yingying Hou for their professional English proofreading in this paper. Chen Li and Hongzan Sun have the same contributions as corresponding authors in this paper. A preprint has previously been published in arXiv. [172].

Author information

Authors and Affiliations

  1. Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China

    Jian Wu, Wanli Liu, Chen Li, Islam Mohammad Shariful, Xiaoqi Li & Xintong Li

  2. School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China

    Tao Jiang

  3. Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, 07030, USA

    Yudong Yao

  4. Shengjing Hospital of China Medical University, Shenyang, China

    Hongzan Sun

  5. Institute for Medical Informatics, University of Luebeck, Luebeck, Germany

    Xinyu Huang & Marcin Grzegorzek

Authors
  1. Jian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wanli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Islam Mohammad Shariful
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yudong Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hongzan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xiaoqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xintong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Xinyu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Marcin Grzegorzek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, J., Liu, W., Li, C. et al. A state-of-the-art survey of U-Net in microscopic image analysis: from simple usage to structure mortification. Neural Comput & Applic 36, 3317–3346 (2024). https://doi.org/10.1007/s00521-023-09284-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09284-4

Keywords

Search

Navigation