Advertisement

Medical Image Segmentation Using Deep Learning

  • Karen López-Linares RománEmail author
  • María Inmaculada García Ocaña
  • Nerea Lete Urzelai
  • Miguel Ángel González Ballester
  • Iván Macía Oliver
Chapter
Part of the Intelligent Systems Reference Library book series (ISRL, volume 171)

Abstract

This chapter aims at providing an introduction to deep learning-based medical image segmentation. First, the reader is guided through the inherent challenges of medical image segmentation, for which actual approaches to overcome those limitations are discussed. Secondly, supervised and semi-supervised architectures are described, where encoder-decoder type networks are the most widely employed ones. Nonetheless, generative adversarial network-based semi-supervised approaches have recently gained the attention of the scientific community. The shift from traditional 2D to 3D architectures is also discussed, as well as the most common loss functions to improve the performance of medical image segmentation approaches. Finally, some future trends and conclusion are described.

References

  1. 1.
    Albarqouni, Shadi, Baur, Christoph, Achilles, Felix, Belagiannis, Vasileios, Demirci, Stefanie, Navab, Nassir: AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imaging 35(5), 1313–1321 (2016)CrossRefGoogle Scholar
  2. 2.
    Alom, Z., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on u-net (R2U-Net) for medical image segmentation, p. 12Google Scholar
  3. 3.
    Anthimopoulos, M., Christodoulidis, S., Ebner, L., Geiser, T., Christe, A., Mougiakakou, S.: Semantic segmentation of pathological lung tissue with dilated fully convolutional networks. IEEE J. Biomed. Health Inform. pp. 1 (2018). arXiv:1803.06167
  4. 4.
    Apostolopoulos, S., De Zanet, S., Ciller, C., Wolf, S., Sznitman, R.: Pathological OCT retinal layer segmentation using branch residual u-shape networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017. Lecture Notes in Computer Science, pp. 294–301. Springer International Publishing (2017)Google Scholar
  5. 5.
    Bai, W., Sinclair, M., Tarroni, G., Oktay, O., Rajchl, M., Vaillant, G., Lee, A.M., Aung, N., Lukaschuk, E., Sanghvi, M.M., Zemrak, F., Fung, K., Paiva, J.M., Carapella, V., Kim, Y.J., Suzuki, H., Kainz, B., Matthews, P.M., Petersen, S.E., Piechnik, S.K., Neubauer, S., Glocker, B., Rueckert, D.: Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson. 20 (2018)Google Scholar
  6. 6.
    Ben-Cohen, A., Diamant, I., Klang, E., Amitai, M., Greenspan, H.: fully convolutional network for liver segmentation and lesions detection. In: Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S., Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, vol. 10008, pp. 77–85. Springer International Publishing, Cham (2016)Google Scholar
  7. 7.
    Bortsova, G., van Tulder, G., Dubost, F., Peng, T., Navab, N., van der Lugt, A., Bos, D., De Bruijne, M.: Segmentation of intracranial arterial calcification with deeply supervised residual dropout networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L. and Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 356–364. Springer International Publishing (2017)Google Scholar
  8. 8.
    Bui, T.D., Shin, J., Moon, T.: 3D densely convolutional networks for volumetric segmentation (2017)Google Scholar
  9. 9.
    Cai, J., Lu, L., Zhang, Z., Xing, F., Yang, L., Yin, Q.: Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G. and Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 442–450. Springer International Publishing (2016)Google Scholar
  10. 10.
    Carneiro, G., Zheng, Y., Xing, F., Yang, L.: Review of deep learning methods in mammography, cardiovascular, and microscopy image analysis. In: Lu, L., Zheng, Y., Carneiro, G., Yang, L. (eds.) Deep Learning and Convolutional Neural Networks for Medical Image Computing: Precision Medicine. High Performance and Large-Scale Datasets, Advances in Computer Vision and Pattern Recognition, pp. 11–32. Springer International Publishing, Cham (2017)CrossRefGoogle Scholar
  11. 11.
    Chudzik, P., Majumdar, S., Caliva, F., Al-Diri, B., Hunter, A.: Exudate segmentation using fully convolutional neural networks and inception modules. In: Medical Imaging 2018: Image Processing, vol. 10574, pp. 1057430. International Society for Optics and Photonics (2018)Google Scholar
  12. 12.
    Çiçed, Ö., Abdulkadir, A., Lienkamp, S.S. Brox, T., Ronneberger, O.: 3D U-net: learning dense volumetric segmentation from sparse annotation (2016). arXiv:1606.06650
  13. 13.
    Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 2843–2851. Curran Associates, Inc. (2012)Google Scholar
  14. 14.
    Dolz, J., Gopinath, K., Yuan, J., Lombaert, H., Desrosiers, C., Ayed, I.B.: HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation (2018)Google Scholar
  15. 15.
    Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., Heng, P.A.: 3D deeply supervised network for automatic liver segmentation from CT volumes. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.), Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 149–157. Springer International Publishing (2016)Google Scholar
  16. 16.
    Fritscher, K., Raudaschl, P., Zaffino, P., Spadea, M.F., Sharp, G.C., Schubert, R.: Deep neural networks for fast segmentation of 3D medical images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 158–165. Springer International Publishing (2016)Google Scholar
  17. 17.
    Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., Gurusamy, K., Davidson, B.R., Pereira, S.P., Clarkson, M.J. and Barratt, D.C.: towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 728–736. Springer International Publishing (2017)CrossRefGoogle Scholar
  18. 18.
    Guerrero, R., Qin, C., Oktay, O., Bowles, C., Chen, L., Joules, R., Wolz, R., Valds-Hernndez, M.C., Dickie, D.A., Wardlaw, J., Rueckert, D.: White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. NeuroImage: Clinical 17, 918–934 (2017)CrossRefGoogle Scholar
  19. 19.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017). arXiv:1505.03540CrossRefGoogle Scholar
  20. 20.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)Google Scholar
  21. 21.
    Heinrich, L., Funke, J., Pape, C., Nunez-Iglesias, J., Saalfeld, S.: Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete drosophila brain. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted InterventionMICCAI 2018, Lecture Notes in Computer Science, pp. 317–325. Springer International Publishing (2018)Google Scholar
  22. 22.
    Heinrich, M.P., Oktay, O.: BRIEFnet: deep pancreas segmentation using binary sparse convolutions. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 329–337. Springer International Publishing (2017)Google Scholar
  23. 23.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2016). arXiv:1608.06993
  24. 24.
    Hussain, S., Anwar, S.M., Majid, M.: Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 248–261 (2018). arXiv:1708.00377CrossRefGoogle Scholar
  25. 25.
    Iqbal, Talha, Ali, Hazrat: Generative adversarial network for medical images (MI-GAN). J. Med. Syst. 42(11), 231 (2018)CrossRefGoogle Scholar
  26. 26.
    Jog, A., Fischl, B. (2018) Pulse sequence resilient fast brain segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2018, Lecture Notes in Computer Science, pp. 654–662. Springer International Publishing (2018)Google Scholar
  27. 27.
    Keshwani, D., Kitamura, Y., Li, Y.: Computation of total kidney volume from CT images in autosomal dominant polycystic kidney disease using multi-task 3D convolutional neural networks, p. 8Google Scholar
  28. 28.
    Koziski, M., Mosinska, A., Salzmann, M., Fua, P.: Learning to segment 3D linear structures using only 2D annotations. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2018, Lecture Notes in Computer Science, pp. 283–291. Springer International Publishing (2018)Google Scholar
  29. 29.
    Kumar, A., Agarwala, S., Dhara, A.K., Nandi, D., Thakur, S.B., Bhadra, A.K., Sadhu, A.: Segmentation of lung field in HRCT images using u-net based fully convolutional networks, p. 10Google Scholar
  30. 30.
    Li, H., Chen, D., Nailon, W.H., Davies, M.E., Laurenson, D.: Improved breast mass segmentation in mammograms with conditional residual u-net (2018). arXiv:1808.08885
  31. 31.
    Li, J., Sarma, K.V., Ho, K.C., Gertych, A., Knudsen, B.S., Arnold, C.W.: A multi-scale u-net for semantic segmentation of histological images from radical prostatectomies. In: AMIA Annual Symposium Proceedings, 2017, pp. 1140–1148 (2018)Google Scholar
  32. 32.
    Li, Rongjian, Zeng, Tao, Peng, Hanchuan, Ji, Shuiwang: Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction. IEEE Trans. Med. Imaging 36(7), 1533–1541 (2017)CrossRefGoogle Scholar
  33. 33.
    Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sanchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). arXiv:1702.05747CrossRefGoogle Scholar
  34. 34.
    Liu, X., Deng, Z., Yang, Y.: Recent progress in semantic image segmentation. Artif. Intell. Rev. (2018). arXiv:1809.10198
  35. 35.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation, p. 10Google Scholar
  36. 36.
    Lopez-Linares, K., Aranjuelo, N., Kabongo, L., Maclair, G., Lete, N., Ceresa, M., Garcia-Familiar, A., Macia, I., Ballester, M.A.G.: Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks. Med. Image Anal. 46, 202–214 (2018)CrossRefGoogle Scholar
  37. 37.
    Luc, P., Couprie, C., Chintala, S. and Verbeek, J.: Semantic segmentation using adversarial networks (2016). arXiv:1611.08408
  38. 38.
    Meng, Q., Roth, H.R., Kitasaka, T., Oda, M., Ueno, J., Mori, K.: Tracking and segmentation of the airways in chest CT using a fully convolutional network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 198–207. Springer International Publishing (2017)Google Scholar
  39. 39.
    Milletari, F., Ahmadi, S.A., Kroll, C., Plate, A., Rozanski, V., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Botzel, K., Navab, N.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound (2016). arXiv:1601.07014
  40. 40.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: Fully convolutional neural networks for volumetric medical image segmentation (2016). arXiv:1606.04797
  41. 41.
    Moeskops, P., Wolterink, J.M., van der Velden, B.H., Gilhuijs, K.G., Leiner, T., Viergever, M.A., Isgum, I.: Deep learning for multi-task medical image segmentation in multiple modalities. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 478–486. Springer International Publishing (2016)Google Scholar
  42. 42.
    Novikov, A.A., Major, D., Wimmer, M., Lenis, D., Buhler, K.: Deep sequential segmentation of organs in volumetric medical scans (2018). arXiv:1807.02437
  43. 43.
    Novikov, A.A., Lenis, D., Major, D., Hladuvka, J., Wimmer, M. and Buhler, K.: Fully convolutional architectures for multi-class segmentation in chest radiographs (2017). arXiv:1701.08816
  44. 44.
    Onieva, J., Andresen, L., Holsting, J.Q., Rahaghi, F.N., Ballester, M.A.G., Estepar, R.S.J., Roman, K.L.L., de La Bruere, I.: 3D pulmonary artery segmentation from CTA scans using deep learning with realistic data augmentationGoogle Scholar
  45. 45.
    Perone, Christian S., Calabrese, Evan, Cohen-Adad, Julien: Spinal cord gray matter segmentation using deep dilated convolutions. Sci. Rep. 8(1), 5966 (2018)CrossRefGoogle Scholar
  46. 46.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol. 9351 of LNCS, pp. 234–241. Springer, Berlin (2015). arXiv:1505.04597Google Scholar
  47. 47.
    Roth, H.R., Shen, C., Oda, H., Oda, M., Hayashi, Y., Misawa, K., Mori, K.: Deep learning and its application to medical image segmentation (2018). arXiv:1803.08691
  48. 48.
    Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3D fully convolutional deep networks (2017). arXiv:1706.05721
  49. 49.
    Shen, C., Roth, H.R., Oda, H., Oda, M., Hayashi, Y., Misawa, K., Mori, K.: On the influence of Dice loss function in multi-class organ segmentation of abdominal CT using 3D fully convolutional networks (2018). arXiv:1801.05912
  50. 50.
    Shen, Dinggang, Guorong, Wu, Suk, Heung-Il: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017)CrossRefGoogle Scholar
  51. 51.
    Shin, H.C., Tenenholtz, N.A., Rogers, J.K., Schwarz, C.G., Senjem, M.L., Gunter, J.L., Andriole, K.P., Michalski, M.: Medical image synthesis for data augmentation and anonymization using generative adversarial networks (2018). arXiv:1807.10225
  52. 52.
    Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training (2016). arXiv:1612.07828
  53. 53.
    Stollenga, M.F., Byeon, W., Liwicki, M., Schmidhuber, J.: Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation (2015). arXiv:1506.07452
  54. 54.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going Deeper with Convolutions (2014). arXiv:1409.4842
  55. 55.
    Tai, L., Ye, H., Ye, Q., Liu, M.: PCA-aided fully convolutional networks for semantic segmentation of multi-channel fMRI (2016). arXiv:1610.01732
  56. 56.
    Vesal, S., Ravikumar, N., Maier, A.: Dilated convolutions in neural networks for left atrial segmentation in 3D gadolinium enhanced-MRI (2018). arXiv:1808.01673
  57. 57.
    Wolterink, J.M., Kamnitsas, K., Ledig, C. and Isgum, I.: Generative adversarial networks and adversarial methods in biomedical image analysis (2018). arXiv:1810.10352
  58. 58.
    Yang, D., Xu, D., Zhou, S.K., Georgescu, B., Chen, M., Grbic, S., Metaxas, D., Comaniciu, D.: Automatic liver segmentation using an adversarial image-to-image network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L. Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 507–515. Springer International Publishing (2017)Google Scholar
  59. 59.
    Yang, L., Zhang, Y., Guldner, I.H., Zhang, S., Chen, D.Z.: 3D segmentation of glial cells using fully convolutional networks and k-terminal cut. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 658–666. Springer International Publishing (2016)Google Scholar
  60. 60.
    Yang, X., Yu, L., Li, S., Wang, X., Wang, N., Qin, J., Ni, D., Heng, P.A.: Towards automatic semantic segmentation in volumetric ultrasound. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 711–719. Springer International Publishing (2017)Google Scholar
  61. 61.
    Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review (2018). arXiv:1809.07294
  62. 62.
    Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2015). arXiv:1511.07122
  63. 63.
    Yu, L., Cheng, J.Z., Dou, Q., Yang, X., Chen, H., Qin, J., Heng, P.A.: Automatic 3D cardiovascular MR segmentation with densely-connected volumetric convnets. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 287–295. Springer International Publishing (2017)Google Scholar
  64. 64.
    Zhou, X., Ito, T., Takayama, R., Wang, S., Hara, T., Fujita, H.: Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting. In Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S, Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, Lecture Notes in Computer Science, pp. 111–120. Springer International Publishing (2016)Google Scholar
  65. 65.
    Zhou, Y., Xie, L., Fishman, E.K., Yuille, A.L.: Deep supervision for pancreatic cyst segmentation in abdominal CT scans. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 222–230. Springer International Publishing (2017)Google Scholar
  66. 66.
    Zhou, Y., Xie, L., Shen, W., Wang, Y., Fishman, E.K., Yuille, A.L.: A fixed-point model for pancreas segmentation in abdominal CT scans. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 693–701. Springer International Publishing (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Karen López-Linares Román
    • 1
    Email author
  • María Inmaculada García Ocaña
    • 1
  • Nerea Lete Urzelai
    • 1
  • Miguel Ángel González Ballester
    • 2
  • Iván Macía Oliver
    • 1
  1. 1.VicomtechSan SebastiánSpain
  2. 2.Universitat Pompeu FabraBarcelonaSpain

Personalised recommendations