Skip to main content
SpringerLink
Log in

A comprehensive survey on convolutional neural network in medical image analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

CNN is inspired from Primary Visual (V1) neurons. It is a typical deep learning technique and can help teach machine how to see and identify objects. In the most recent decade, deep learning develops rapidly and has been well used in various fields of expertise such as computer vision and natural language processing. As the representative algorithm of deep learning, Convolution Neural Network (CNN) has been regarded as a breakthrough of historic significance in image processing and visual recognition tasks since the astonishing results achieved on ImageNet Large Scale Visual Recognition Competition (ILSVRC) Unlike methods based on handcrafted features, CNN models can build high-level features from low-level ones in a data-driven fashion and have displayed great potential in medical image analysis among the aspects of segmentation of histological images identification, lesion detection, tissue classification, etc. This paper provides a review on CNN from the perspectives of its basic mechanism introduction, structure, typical architecture and main application in medical image analysis through analyzing over 100 references from Google Scholar, PubMed, Web of Science and various sources published from 1958 to 2020.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36

Similar content being viewed by others

References

  1. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375

  2. Alakwaa W, Nassef M, Badr A (2017) Lung cancer detection and classification with 3D convolutional neural network (3D-CNN). Lung Cancer 8(8):409

    Google Scholar 

  3. Anaraki AK, Ayati M, Kazemi F (2019) Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms. Biocybernetics Biomed Eng 39(1):63–74

    Google Scholar 

  4. Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou S (2016) Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans Med Imaging 35(5):1207–1216

    Google Scholar 

  5. Azamjah N, Soltan-Zadeh Y, Zayeri F (2019) Global trend of breast Cancer mortality rate: a 25-year study. Asian Pacific J Cancer Prevent: APJCP 20(7):2015–2020

    Google Scholar 

  6. Baldassi C, Malatesta EM, Zecchina R (2019) Properties of the geometry of solutions and capacity of multilayer neural networks with rectified linear unit activations. Phys Rev Lett 123(17):6

    Google Scholar 

  7. Bardou D, Zhang K, Ahmad SM (2018) Classification of breast cancer based on histology images using convolutional neural networks. IEEE Access 6:24680–24693

    Google Scholar 

  8. Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404

    Google Scholar 

  9. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166

    Google Scholar 

  10. Bottou L (2010) Large-scale machine learning with stochastic gradient descent, in Proceedings of COMPSTAT'2010. Springer, p 177–186

  11. Bottou L (2012) Stochastic gradient descent tricks, in Neural networks: Tricks of the trade. Springer, p 421–436

  12. Bray F et al Erratum: Global cancer statistics (2018): GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries

  13. Che Z-G, Chiang T-A, Che Z-H (2011) Feed-forward neural networks training: a comparison between genetic algorithm and back-propagation learning algorithm. Int J Innovativ Comput Inf Control 7(10):5839–5850

    Google Scholar 

  14. Chen L-C et al (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV)

  15. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. in Proceedings of the IEEE conference on computer vision and pattern recognition

  16. Chung H, Lee SJ, Park JG (2016) Deep neural network using trainable activation functions. In 2016 International Joint Conference on Neural Networks (IJCNN). IEEE

  17. Dasgupta A, Singh S (2017) A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE

  18. de Koning HJ, van der Aalst CM, de Jong PA, Scholten ET, Nackaerts K, Heuvelmans MA, Lammers JWJ, Weenink C, Yousaf-Khan U, Horeweg N, van ’t Westeinde S, Prokop M, Mali WP, Mohamed Hoesein FAA, van Ooijen PMA, Aerts JGJV, den Bakker MA, Thunnissen E, Verschakelen J, Vliegenthart R, Walter JE, ten Haaf K, Groen HJM, Oudkerk M (2020) Reduced lung-cancer mortality with volume CT screening in a randomized trial. N Engl J Med 382(6):503–513

    Google Scholar 

  19. Du J (2018) Understanding of object detection based on CNN family and YOLO. In Journal of Physics: Conference Series. IOP Publishing

  20. Dumoulin V, Visin F (2016) A guide to convolution arithmetic for deep learning. arXiv preprint arXiv:1603.07285

  21. Gao F, Wu T, Li J, Zheng B, Ruan L, Shang D, Patel B (2018) SD-CNN: a shallow-deep CNN for improved breast cancer diagnosis. Comput Med Imaging Graph 70:53–62

    Google Scholar 

  22. Gauen K et al (2017) Low-power image recognition challenge. In 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE.

  23. Girshick R (2015) Fast r-cnn. in Proceedings of the IEEE international conference on computer vision

  24. Gkioxari G, Girshick R, Malik J (2015) Contextual action recognition with r* cnn. in Proceedings of the IEEE international conference on computer vision

  25. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning.[Sl]. MIT press: 1 Rogers street in Cambridge, Ma 02142

    MATH  Google Scholar 

  26. Hamidian S et al (2017) 3D convolutional neural network for automatic detection of lung nodules in chest CT. In: Medical Imaging 2017: Computer-Aided Diagnosis. Int Soc Optics Photonics

  27. He K et al (2016) Identity mappings in deep residual networks. In European conference on computer vision. Springer

  28. Hinton G, Srivastava N, Swersky K (2012) Overview of mini-batch gradient descent. Neural Netw Machine Learning 575

  29. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Google Scholar 

  30. Howard AG (2013) Some improvements on deep convolutional neural network based image classification. arXiv preprint arXiv:1312.5402

  31. Huang J-T, Li J, Gong Y (2015) An analysis of convolutional neural networks for speech recognition. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE

  32. Hussain S, Anwar SM, Majid M (2018) Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 282:248–261

    Google Scholar 

  33. Hussein S et al (2017) Tumornet: Lung nodule characterization using multi-view convolutional neural network with gaussian process. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE

  34. Iandola FN et al (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360

  35. Jiang YZ et al (2015) Collaborative fuzzy clustering from multiple weighted views. IEEE Trans Cybernetics 45(4):688–701

    Google Scholar 

  36. Jiang YZ et al (2017) Seizure classification from EEG signals using transfer Learning, semi-supervised Learning and TSK fuzzy system. IEEE Trans Neural Syst Rehab Eng 25(12):2270–2284

    Google Scholar 

  37. Jiang YZ et al (2017) Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system. IEEE Trans Fuzzy Syst 25(1):3–20

    Google Scholar 

  38. Jiang Y, Zhao K, Xia K, Xue J, Zhou L, Ding Y, Qian P (2019) A novel distributed multitask fuzzy clustering algorithm for automatic MR brain image segmentation. J Med Syst 43(5):118

    Google Scholar 

  39. Jiang YZ et al (2019) Seizure recognition using a novel multitask radial basis function neural network. J Med Imaging Health Informatics 9(9):1865–1870

    Google Scholar 

  40. Jiang YZ et al (2019) A novel synthetic CT generation method using multitask maximum entropy clustering. Ieee Access 7:119644–119653

    Google Scholar 

  41. Jiang Y, Gu X, Wu D, Hang W, Xue J, Qiu S, Chin-Teng L (2020) A novel negative-transfer-resistant fuzzy clustering model with a shared cross-domain transfer latent space and its application to brain CT image segmentation. IEEE/ACM Trans Comput Biol Bioinform 1. doi: https://doi.org/10.1109/TCBB.2019.2963873

  42. Jiang Y, Zhang Y, Lin C, Wu D, Lin CT (2020) EEG-based driver drowsiness estimation using an online multi-view and transfer TSK fuzzy system. IEEE Trans Intell Transp Syst: 1–13 https://doi.org/10.1109/TITS.2020.2973673

  43. Juefei-Xu F, Naresh Boddeti V, Savvides M (2017) Local binary convolutional neural networks. in Proceedings of the IEEE conference on computer vision and pattern recognition

  44. Kamnitsas K et al (2015) Multi-scale 3D convolutional neural networks for lesion segmentation in brain MRI. Ischemic Stroke Lesion Segment 13:46

    Google Scholar 

  45. Kang G et al (2017) 3D multi-view convolutional neural networks for lung nodule classification. PloS one 12(11):1–21

    Google Scholar 

  46. Khan H, Shah PM, Shah MA, Islam S, Rodrigues JJPC (2020) Cascading handcrafted features and convolutional neural network for IoT-enabled brain tumor segmentation. Comput Commun 153:196–207

    Google Scholar 

  47. Khirirat S, Feyzmahdavian HR, Johansson M (2017) Mini-batch gradient descent: Faster convergence under data sparsity. in 2017 IEEE 56th Annual Conference on Decision and Control (CDC). IEEE

  48. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. in Advances in neural information processing systems

  49. Lawrence S, Giles CL, Ah Chung Tsoi, Back AD (1997) Face recognition: a convolutional neural-network approach. IEEE Trans Neural Netw 8(1):98–113

    Google Scholar 

  50. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Google Scholar 

  51. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Google Scholar 

  52. Lemaréchal C (2012) Cauchy and the gradient method. Doc Math Extra 251:254

    MATH  Google Scholar 

  53. Lenc K, Vedaldi A (2015) R-cnn minus r. arXiv preprint arXiv:1506.06981

  54. Lévy D, Jain A (2016) Breast mass classification from mammograms using deep convolutional neural networks. arXiv preprint arXiv:1612.00542

  55. Li J et al (2017) A comparison of deep learning methods for environmental sound detection. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE

  56. Li Y, Zhang X, Chen D 2018 Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  57. Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400

  58. Liu W et al (2016) Ssd: Single shot multibox detector. In: European conference on computer vision. Springer

  59. Marsalli M (2006) Mcculloch-pitts neurons. in The 2008 Annual Meeting of the consortium on cognitive science instruction (ccsi)

  60. Mateen M et al (2019) Fundus image classification using VGG-19 architecture with PCA and SVD. Symmetry 11(1):1

    Google Scholar 

  61. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Mathematical Biophys 5(4):115–133

    MathSciNet  MATH  Google Scholar 

  62. Mehta R, Sivaswamy J (2017) M-net: A convolutional neural network for deep brain structure segmentation. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE.

  63. Mikolov T, et al (2010) Recurrent neural network based language model. in Eleventh annual conference of the international speech communication association

  64. Minsky M, Papert SA (2017) Perceptrons: An introduction to computational geometry. MIT press

  65. Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MJNL, Isgum I (2016) Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans Med Imaging 35(5):1252–1261

    Google Scholar 

  66. Mordang J-J et al (2016) Automatic microcalcification detection in multi-vendor mammography using convolutional neural networks. In: International Workshop on Breast Imaging. Springer

  67. Nielsen MA (2015) Neural networks and deep learning, vol 2018. Determination press San Francisco, CA, USA

  68. Pascanu R, Mikolov T, Bengio Y (2012) Understanding the exploding gradient problem. CoRR, abs/1211.5063. 2:417

  69. Pedamonti D (2018) Comparison of non-linear activation functions for deep neural networks on MNIST classification task. arXiv preprint arXiv:1804.02763

  70. Purkait P, Zhao C, Zach C (2017) SPP-Net: Deep absolute pose regression with synthetic views. arXiv preprint arXiv:1712.03452

  71. Rastegari M, et al (2016) Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision. Springer

  72. Reddy S (2018) Use of artificial intelligence in healthcare delivery, in eHealth-Making Health Care Smarter. IntechOpen:81–97

  73. Ren S et al (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. in Advances in neural information processing systems

  74. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer

  75. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386–408

    Google Scholar 

  76. Rouhi R, Jafari M, Kasaei S, Keshavarzian P (2015) Benign and malignant breast tumors classification based on region growing and CNN segmentation. Expert Syst Appl 42(3):990–1002

    Google Scholar 

  77. Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747

  78. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536

    MATH  Google Scholar 

  79. Sajjad M, Khan S, Muhammad K, Wu W, Ullah A, Baik SW (2019) Multi-grade brain tumor classification using deep CNN with extensive data augmentation. J Comput Sci 30:174–182

    Google Scholar 

  80. Salehi SSM, Erdogmus D, Gholipour A (2017) Auto-context convolutional neural network (auto-net) for brain extraction in magnetic resonance imaging. IEEE Trans Med Imaging 36(11):2319–2330

    Google Scholar 

  81. Salekin MS, Jelodar AB, Kushol R (2019) Cooking state recognition from images using inception architecture. In 2019 International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST). IEEE

  82. Scardapane S, van Vaerenbergh S, Totaro S, Uncini A (2019) Kafnets: kernel-based non-parametric activation functions for neural networks. Neural Netw 110:19–32

    Google Scholar 

  83. Schairer C, Mink PJ, Carroll L, Devesa SS (2004) Probabilities of death from breast cancer and other causes among female breast cancer patients. J Natl Cancer Inst 96(17):1311–1321

    Google Scholar 

  84. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117

    Google Scholar 

  85. Schmidt WF et al (1993) Initializations, back-propagation and generalization of feed-forward classifiers. In IEEE International Conference on Neural Networks. IEEE

  86. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Google Scholar 

  87. Shen W et al (2015) Multi-scale convolutional neural networks for lung nodule classification. In: International Conference on Information Processing in Medical Imaging. Springer

  88. Sheng T et al (2018) A quantization-friendly separable convolution for mobilenets. In 2018 1st Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2). IEEE

  89. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  90. Su H et al (2015) Region segmentation in histopathological breast cancer images using deep convolutional neural network. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). IEEE

  91. Sun M, Song Z, Jiang X, Pan J, Pang Y (2017) Learning pooling for convolutional neural network. Neurocomputing 224:96–104

    Google Scholar 

  92. Sun W, Tseng TL(B), Zhang J, Qian W (2017) Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data. Comput Med Imaging Graph 57:4–9

    Google Scholar 

  93. Szegedy C et al (2015) Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition

  94. Szegedy C et al (2016) Rethinking the inception architecture for computer vision. in Proceedings of the IEEE conference on computer vision and pattern recognition

  95. Szegedy C et al (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. in Thirty-first AAAI conference on artificial intelligence

  96. Ting FF, Tan YJ, Sim KS (2019) Convolutional neural network improvement for breast cancer classification. Expert Syst Appl 120:103–115

    Google Scholar 

  97. Toğaçar M, Ergen B, Cömert Z (2020) BrainMRNet: brain tumor detection using magnetic resonance images with a novel convolutional neural network model. Med Hypotheses 134:109531

    Google Scholar 

  98. Waks AG, Winer EP (2019) Breast cancer treatment: a review. Jama 321(3):288–300

    Google Scholar 

  99. Xue Y, Farhat FG, Boukrina O, Barrett AM, Binder JR, Roshan UW, Graves WW (2020) A multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI images. NeuroImage: Clin 25:102118

    Google Scholar 

  100. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  101. Yu D et al (2014) Mixed pooling for convolutional neural networks. In International conference on rough sets and knowledge technology. Springer

  102. Yuan Z-W, Zhang J (2016) Feature extraction and image retrieval based on AlexNet. in Eighth International Conference on Digital Image Processing (ICDIP 2016). International Society for Optics and Photonics

  103. Zarándy Á et al (2015) Overview of CNN research: 25 years history and the current trends. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE

  104. Zeiler MD, Fergus R (2013) Stochastic pooling for regularization of deep convolutional neural networks. arXiv preprint arXiv:1301.3557

  105. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In European conference on computer vision. Springer

  106. Zeiler MD, Taylor GW, Fergus R (2011) Adaptive deconvolutional networks for mid and high level feature learning. In 2011 International Conference on Computer Vision. IEEE

  107. Zhang Z, Sarhadi M (1993) A modified neuron activation function which enables single layer perceptrons to solve some linearly inseparable problems. In Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan). IEEE

  108. Zubair S, Yan F, Wang W (2013) Dictionary learning based sparse coefficients for audio classification with max and average pooling. Digital Signal Process 23(3):960–970

    MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was partially supported by Royal Society International Exchanges Cost Share Award, UK (RP202G0230); Medical Research Council Confidence in Concept Award, UK (MC_PC_17171); Hope Foundation for Cancer Research, UK (RM60G0680); British Heart Foundation Accelerator Award, UK; Guangxi Key Laboratory of Trusted Software (kx201901); Fundamental Research Funds for the Central Universities (CDLS-2020-03); Key Laboratory of Child Development and Learning Science (Southeast University), Ministry of Education.

Author information

Authors and Affiliations

  1. School of Informatics, University of Leicester, Leicester, LE1 7RH, UK

    Xujing Yao, Xinyue Wang, Shui-Hua Wang & Yu-Dong Zhang

  2. Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, 21589, Saudi Arabia

    Shui-Hua Wang & Yu-Dong Zhang

  3. School of Architecture Building and Civil Engineering, Loughborough University, Loughborough, LE11 3TU, UK

    Shui-Hua Wang

Authors
  1. Xujing Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinyue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shui-Hua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu-Dong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shui-Hua Wang or Yu-Dong Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yao, X., Wang, X., Wang, SH. et al. A comprehensive survey on convolutional neural network in medical image analysis. Multimed Tools Appl 81, 41361–41405 (2022). https://doi.org/10.1007/s11042-020-09634-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09634-7

Keywords

Search

Navigation