Skip to main content

Advertisement

Log in

Deep feature extraction and classification of breast ultrasound images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Controlled despeckling (structure/edges/feature preservation with smoothing the homogeneous areas) is a desired pre-processing step for the design of computer-aided diagnostic (CAD) systems using ultrasound images as the presence of speckle noise masks diagnostically important information making interpretation difficult even for experienced radiologist. For efficiently classifying the breast tumors, the conventional CAD system designs use hand-crafted features. However, these features are not robust to the variations in size, shape and orientation of the tumors resulting in lower sensitivity. Thus deep feature extraction and classification of breast ultrasound images have recently gained attention from research community. The deep networks come with an advantage of directly learning the representative features from the images. However, these networks are difficult to train from scratch if the representative training data is small in size. Therefore transfer learning approach for deep feature extraction and classification of medical images has been widely used. In the present work the performance of four pre-trained convolutional neural networks VGG-19, SqueezeNet, ResNet-18 and GoogLeNet has been evaluated for differentiating between benign and malignant tumor types. From the results of the experiments, it is noted that CAD system design using GoogLeNet architecture for deep feature extraction followed by correlation based feature selection and fuzzy feature selection using ANFC-LH yields highest accuracy of 98.0% with individual class accuracy value of 100% and 96% for benign and malignant classes respectively. For differentiating between the breast tumors, the proposed CAD system design can be utilized in routine clinical environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Ahmed SS, Dey N, Ashour AS, Pistolla DS, Timar DB, Balas VE, Tavares JMRS (2017) Effect of fuzzy partitioning in Crohn’s disease classification: a neuro-fuzzy-based approach. Med Biol Eng Comput 55(1):101–115

    Google Scholar 

  2. Ai H, Hu X, Zhang J (2019) Medical tumor image classification algorithm and its application in breast cancer. Investig Clin 60(5):1169–1179

    Google Scholar 

  3. Aja-Fernández S, AlberolaLópez C (2006) On the estimation of the coefficient of variation for anisotropic diffusion speckle filtering. IEEE Trans Image Process 15(9):2694–2701

    Google Scholar 

  4. Al-Dhabyani W, Fahmy A, Gomma M, Khaled H (2019) Deep learning approaches for data augmentation and classification of breast masses using ultrasound images. Int J Adv Comput Sci Appl 10(5):618–627

    Google Scholar 

  5. Alvarenga AV, Infantosi AFC, Pereira WCA, Azevedo CM (2012) Assessing the combined performance of texture and morphological parameters in distinguishing breast tumors in ultrasound images. Medical Analysis 39(12):7350–7358

    Google Scholar 

  6. Antropova N, Huynh BQ, Giger ML (2017) A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys 44(10):5162–5171

    Google Scholar 

  7. Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK (2018) Medical image analysis using convolutional neural networks: a review. J Med Syst 42:226

    Google Scholar 

  8. Bakkouri I, Afdel K (2019) Computer-aided diagnosis (CAD) system based on multi-layer feature fusion network for skin lesion recognition in dermoscopy images. Multimed Tools Appl. https://doi.org/10.1007/s11042-019-07988-1

  9. Balagourouchetty L, Pragatheeswaran JK, Pottakat B, Ramkumar G (2019) GoogLeNet based ensemble FCNet classifier for focal liver lesion diagnosis. IEEE J Biomed Health Informatics 24:1686–1694. https://doi.org/10.1109/JBHI.2019.2942774

    Article  Google Scholar 

  10. Byra M (2018) Discriminant analysis of neural style representations for breast lesion classification in ultrasound. Biocybernetics Biomed Eng 38(3):684–690

    Google Scholar 

  11. Byra M, Galperin M, Olson L, O’Boyle M, Comstock C, Andre M (2019) Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion. Med Phys 46(2):746–755. https://doi.org/10.1002/mp.13361

    Article  Google Scholar 

  12. Cao Z, Duan L, Yang G, Yue T, Chen Q (2019). An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Med Imaging, 19, doi: https://doi.org/10.1186/s12880-019-0349-x.

  13. Cetisli B (2010) Development of an adaptive neuro-fuzzy classifier using linguistic hedges: part 1. Expert Syst Appl 37:6093–6101

    Google Scholar 

  14. Cetisli B (2010) Development of an adaptive neuro-fuzzy classifier using linguistic hedges: part 2. Expert Syst Appl 37:6102–6108

    Google Scholar 

  15. Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277

    MATH  Google Scholar 

  16. Chen CY, Chiou HJ, Chou SY, Chiou SY, Wang HK, Chou YH, Chiang HK (2009) Computer-aided diagnosis of soft-tissue tumors using sonographic morphologic and texture features. Acad Radiol 16(2):1531–1538

    Google Scholar 

  17. Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM (2016). Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT Scientific Reports, 6, doi: https://doi.org/10.1038/srep24454

  18. Chi J, Walia E, Babyn P, Wang J, Groot G, Eramian M (2017) Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J Digit Imaging 30(4):477–486

    Google Scholar 

  19. Chiao JY, Chen KY, Liao KYK, Hsieh PH, Zhang G, Huang TC (2019) Detection and classification the breast tumors using mask R-CNN on sonograms. Medicine 98(19):e15200

    Google Scholar 

  20. Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network. Eur Radiol 29:5458–5468. https://doi.org/10.1007/s00330-019-06118-7

    Article  Google Scholar 

  21. Consensus document for management of breast cancer. Indian Council of Medical Research, [online], http://www.icmr.nic.in/guide/cancer/Breast_Cancer.pdf (Accessed: April 2018).

  22. Crytsal P, Strano SD, Shcharynski S, Koretz MJ (2003) Using sonography to screen women with mammographically dense breasts. Am J Roentgenol 181:177–182

    Google Scholar 

  23. Fletcher SW, Elmore JG (2003) Mammographic screening for breast cancer. N Engl J Med 348(17):1672–1680

    Google Scholar 

  24. Fujioka T, Kubota K, Mori M, Kikuchi Y, Katsuta L, Kasahara M, Oda G, Ishiba T, Nakagawa T, Tateishi U (2019) Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network. Jpn J Radiol 37:466–472

    Google Scholar 

  25. Garcia AH, Konig P (2018) Further advantages of data augmentation on convolutional neural networks. In: Kurkova V et al (eds) ICANN, vol 11139. Springer, Switzerland, pp 95–103

    Google Scholar 

  26. Gardezi SJS, Elazab A, Lei B, Wang T (2019) Breast cancer detection and diagnosis using mammographic data: systematic review. J Med Internet Res 21(7):e14464

    Google Scholar 

  27. Gokhale S (2009) Ultrasound characterization of breast masses. Int J Radiol Imaging 19(3):242–247

    Google Scholar 

  28. Gordon PB, Goldenberg SL (1995) Malignant breast masses detected only by ultrasound. Cancer 76(4):626–630

    Google Scholar 

  29. Han HK, Kang JY, Jeong MH, Park W, Kim WC, Bang WC, Seong YK (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714–7728

    Google Scholar 

  30. Han S, Lee S, Lee JR (2019) A practical implementation of deep learning method for supporting the classification of breast lesions in ultrasound images. Int J Advanc Smart Convergence 8(1):24–34

    Google Scholar 

  31. He K, Zhang X, Ren S, Sun J (2016). Deep residual learning for image recognition. In Proc. of IEEE conference on computer vision and Pattern Recognition, pp. 770–778.

  32. Hooda R, Mittal A, Sofat S (2019). Automated TB classification using ensemble of deep architectures. Multimedia tools and applications, doi: https://doi.org/10.1007/s11042-019-07984-5

  33. http://ultrasoundcases.info/category.aspx?cat=67, Accessed December 2016.

  34. Hu Z, Tang J, Wang Z, Zhang K, Zhang L, Sun Q (2018) Deep learning for image-based cancer detection and diagnosis- a survey. Pattern Recogn 83:134–149

    Google Scholar 

  35. Huang Y, Han L, Dou H, Luo H, Yuan Z, Zhang J, Fin G (2019). Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images. BioMedical Eng OnLine, 18, doi: https://doi.org/10.1186/s12938-019-0626-5.

  36. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016). SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and < 0.5 MB model size, arXiv:1602.07360.

  37. Jain I, Jain VK, Jain R (2018) Correlation feature selection based improved-binary particle swarm optimization for gene selection and cancer classification. Appl Soft Comput 62:203–215

    Google Scholar 

  38. Kadah YM, Farag AA, Zurada JM, Badawi AM, Youssef ABM (1996) Classification algorithms for quantitative tissue characterization of diffuse liver disease from ultrasound. IEEE Trans Med Imaging 14(4):466–478

    Google Scholar 

  39. Kameswari SSD, Vijayakumar V (2019) A CNN based breast tumor classifier using Mendeley BUS dataset. Int J Innov Technol Explor Eng 8(6):1340–1343

    Google Scholar 

  40. Kar S, Das S, Ghosh PK (2015) Applications of neuro fuzzy systems: a brief review and future outline. Appl Soft Comput 15:243–259

    Google Scholar 

  41. Karimi B, Krzyzak A (2014) Computer-aided system for automatic classification of suspicious lesions in breast ultrasound images. In: Rutkowski I et al (eds) ICAISC 2014, vol 8468. Springer, Cham, pp 131–142

    Google Scholar 

  42. Key TJ, Verkasalo PK, Banks E (2001) Epidemiology of breast cancer. The Lancet Oncology 2(3):133–140

    Google Scholar 

  43. Kriti, Virmani J, Agarwal R (2019) Effect of despeckle filtering on classification of breast tumors using ultrasound images. Biocybernetics Biomed Eng 39:536–560

  44. Kriti, Virmani J, Agarwal R (2019) Assessment of despeckle filtering algorithms for segmentation of breast tumors from ultrasound images. Biocybernetics Biomed Eng 39:100–121

  45. Kumar A, Kim J, Lyndon D, Fulham M, Feng D (2016) An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE J Biomed Health Informatics 21(1):31–40

    Google Scholar 

  46. Kumar I, Bhadauria HS, Virmani J, Thakur S (2017) A hybrid hierarchical framework for classification of breast density using digitized film screen mammograms. Multimed Tools Appl 76(18):18789–18813

    Google Scholar 

  47. Lakhani P, Sundaram B (2017) Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284(2):574–582

    Google Scholar 

  48. Lee CY, Chen GL, Zhang ZX, Chou YH, Hsu CC (2018) Is intensity inhomogeneity correction useful for classification of breast cancer in sonograms using deep neural network ? J Healthcare Eng 2018:1–10

    Google Scholar 

  49. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van der Laak JA, Ginneken BV, Sanchez CI (2017) A survey of deep, learning in medical image analysis. Med Image Anal 42:60–88

    Google Scholar 

  50. Manogaran G, Varatharajan R, Priyan MK (2018) Hybrid recommendation system for heart disease diagnosis based on kernel learning with adaptive neuro-fuzzy inference system. Multimed Tools Appl 77(4):4379–4399

    Google Scholar 

  51. Mateen M, Wen J, Song S, Huang Z (2019). Fundus image classification using VGG-19 architecture with PCA and SVD. Symmetry, 11 (1): doi: 20.3390/sym11010001.

  52. Meng D, Zhang L, Cao G, Cao W, Zhang G, Hu B (2017) Liver fibrosis classification based on transfer learning and FCNet for ultrasound images. IEEE Access 5:5804–5810

    Google Scholar 

  53. Michalak K, Kwasnicka H (2006) Correlation-based feature selection strategy in classification problems. Int J Appl Math Comput Sci 16:503–511

    MathSciNet  MATH  Google Scholar 

  54. Moon WK, Lo CM, Huang CS, Chen JH, Chang RF (2012) Computer-aided diagnosis based on speckle patterns in ultrasound images. Ultrasound Med Biol 38(7):1251–1261

    Google Scholar 

  55. Nibali A, He Z, Wollersheim D (2017) Pulmonary nodule classification with deep residual networks. Int J Comput Assist Radiol Surg 12(10):1799–1808

    Google Scholar 

  56. Prabusankarlal KM, Thirumoorthy MR (2015) Assessment of combined textural and morphological features for diagnosis of breast masses in ultrasound. Human-Centric Comput Inform Sci 5:12–28

    Google Scholar 

  57. Priego-Torres BM, Morillo DS, Granero MAF, Rojo MG (2020) Automatic segmentation of whole-slide H&E stained breast histopathology images using a deep convolutional neural network architecture. Expert Syst Appl 151:113387

    Google Scholar 

  58. Qi X, Zhang L, Chen Y, Pi Y, Chen Y, Lv Q, Yi Z (2019) Automated diagnosis of breast ultrasonography images using deep learning neural networks. Med Image Anal 52:185–198

    Google Scholar 

  59. Qian X, Patton EW, Swaney J, Xing Q, Zeng TH (2018). Machine learning on cataracts classification using SqueezeNet. In Proc. of 4th international conference on Universal Village, doi: https://doi.org/10.1109/UV.2018.8642133.

  60. R Core Team. A language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria, 2013, http://www.R-project.org/.

  61. Rani VMK, Dhanekaran SS (2019) Classification of ultrasound breast cancer tumor images using neural learning and predicting the tumor growth rate. Multimed Tools Appl 79:16967–16985. https://doi.org/10.1007/s11042-019-7487-6

    Article  Google Scholar 

  62. Ravishankar H, Sudhakar P, Venkataramani R, Thiruvenkadam S, Annangi P, Babu N, Vaidya V (2016) Understanding the mechanisms of deep transfer learning for medical images. In: Carneiro G et al (eds) Deep learning and data labeling for medical applications, vol 10008. Springer, Cham, pp 188–196

    Google Scholar 

  63. Rawat J, Singh A, Bhadauria HS, Virmani J, Devgun JS (2017) Classification of acute lymphoblastic leukaemia using hybrid hierarchical classifiers. Multimed Tools Appl 76(18):19057–19085

    Google Scholar 

  64. Rawat J, Singh A, Bhadauria HS, Virmani J, Devgun JS (2018) Leukocyte classification using adaptive neuro-fuzzy inference system in microscopic blood images. Arab J Sci Eng 43(12):7041–7058

    Google Scholar 

  65. Rehman M, Khan SH, Abbas Z, Rizvi SMD (2019). Classification of diabetic retinopathy images based on customized CNN architecture. In Proc. of Amity International conference on artificial intelligence, pp. 244–248.

  66. Sharma A, Kumar R (2019) A constrained framework for context-aware remote E-healthcare (CARE) services. Trans Emerg Telecommun Technol. https://doi.org/10.1002/ett.3649

  67. Sharma A, Ansari MD, Kumar R (2017). A comparative study of edge detectors in digital image processing. In Proc. of 4th international conference on signal processing, computing and control, doi: https://doi.org/10.1109/ISPCC.2017.8269683.

  68. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298

    Google Scholar 

  69. Simonyan K, Zisserman A (2014). Very deep convolutional networks for large-scale image recognition, arXiv: 1409.1556.

  70. Singh BK (2019) Determining relevant biomarkers for prediction of breast cancer using anthropometric and clinical features: a comparative investigation in machine learning paradigm. Biocybernetics and Biomedical Engineering 39(2):393–409

    Google Scholar 

  71. Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R (2017) Deep learning for automated skeletal bone age assessment in X-ray images. Med Image Anal 36:41–51

    Google Scholar 

  72. Su Y, Wang Y, Jiao J, Guo Y (2011) Automatic detection and classification of breast tumors in ultrasonic images using texture and morphological features. Open Med Inform J 5(Suppl1-M3):26–37

    Google Scholar 

  73. Sudharshan PJ, Petitjean C, Spanhol F, Oliviera LE, Huette L, Honeine P (2019) Multiple instance learning for histopathological breast cancer image classification. Expert Syst Appl 117:103–111

    Google Scholar 

  74. Sun J, Wan C, Vheng J, Yu F, Liu J (2017) Retinal image quality classification using fine-tuned CNN. In: Cardoso M et al (eds) Fetal, infant and ophthalmic medical image analysis, vol 10554. Springer, Cham, pp 126–133

    Google Scholar 

  75. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015). Going deeper with convolutions. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9.

  76. Tajbaksh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312

    Google Scholar 

  77. Tohno E, Ueno E, Watanabe H (2009) Ultrasound screening of breast cancer. Breast Cancer 16:18–22

    Google Scholar 

  78. Ubeyli ED (2009) Adaptive neuro-fuzzy inference systems for automatic detection of breast cancer. J Med Syst 33:353–358

    Google Scholar 

  79. Vesal S, Ravikumar N, Davari AA, Ellmann S, Maier A (2018) Classification of breast cancer histology images using transfer learning. In: Campilho A et al (eds) Image analysis and recognition, vol 10882. Springer, Cham, pp 812–819

    Google Scholar 

  80. Vianna VP (2018). Study and development of a computer-aided diagnosis system for classification of chest X-ray images using convolutional neural networks pre-trained for ImageNet and data augmentation, arXiv preprint arXiv: 1806.00839.

  81. Virmani J, Kumar V, Kalra N, Khandelwal N (2013) SVM-based characterization of liver ultrasound images using wavelet packet texture descriptors. J Digit Imaging 26(3):530–543

    Google Scholar 

  82. Wan S, Liang Y, Zhang Y (2018) Deep convolutional neural networks for diabetic retinopathy detection by image classification. Comput Electr Eng 72:274–282

    Google Scholar 

  83. Wang J, Perez L (2017). The effectiveness of data augmentation in image classification using deep learning. arXiv: 1712.04621v1.

  84. Weiss N, Kost H, Homeyer A (2018). Towards interactive breast tumor classification using transfer learning. In: Campilho a. et al. (eds.), ICIAR 2018, vol. 10882, pp. 727-736, springer.

  85. Wu WJ, Moon WK (2008) Ultrasound breast tumor image computer-aided diagnosis with texture and morphological features. Acad Radiol 15(7):873–880

    Google Scholar 

  86. Wu WJ, Lin SW, Moon WK (2015) An artificial immune system-based support vector machine approach for classifying ultrasound breast tumor images. J Digit Imaging 28(5):576–585

    Google Scholar 

  87. Xiao T, Liu L, Li K, Qin W, Yu S, Li Z (2018) Comparison of transferred deep neural networks in ultrasonic breast masses discrimination. BioMed Res Int. https://doi.org/10.1155/2018/4605191

  88. Xie X, Shi F, Niu J, Tang X (2018) Breast ultrasound image classification and segmentation using convolutional neural networks. In: Hong R et al (eds) Pacific rim conference on multimedia, vol 11166. Springer, Cham, pp 200–211

    Google Scholar 

  89. Yoshida H, Casalino DD, Keserci B, Coskun A, Ozturk O, Savranlar A (2003) Wavelet-packet based texture analysis for differentiation between benign and malignant liver tumors in ultrasound images. Phys Med Biol 48(22):3735–3753

    Google Scholar 

  90. Yu L, Chen H, Dou Q, Heng PA (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004

    Google Scholar 

  91. Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning-I. Inf Sci 8(3):199–249

    MathSciNet  MATH  Google Scholar 

  92. Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning-II. Inf Sci 8(4):301–357

    MathSciNet  MATH  Google Scholar 

  93. Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning-III. Inf Sci 9(1):43–80

    MathSciNet  MATH  Google Scholar 

  94. Zakeri FS, Behnam H, Ahmadinejad N (2012) Classification of benign and malignant breast masses based on shape and texture features in sonography images. J Med Syst 36(3):1621–1627

    Google Scholar 

  95. Zeimarani B, Costa MGF, Nyrani NZ, Filho CFFC (2019) A novel breast tumor classification in ultrasound images, using deep convolutional neural network. In: Costa RF, Machado J, Alvarenga A (eds) XXVI Brazilian congress on biomedical engineering, vol 70/2. Springer, Singapore, pp 89–94

    Google Scholar 

  96. Zhang E, Seiler S, Chen M, Lu W, Gu X (2019). BIRADS features oriented semi-supervised deep learning for breast ultrasound computer-aided diagnosis. arXiv:1904.01076.

  97. Zhou Z, Shin J, Zhang L, Gurudu S, Gotway M, Liang J (2017). Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In Proc. of IEEE conference on computer vision and Pattern Recognition, pp. 7340–7351.

Download references

Acknowledgments

The authors would like to thank Dr. Shruti Thakur, Kamla Nehru Hospital, Shimla for grading the filtered images and marking the tumor contours in the original ultrasound images. The authors would also like to thank Director, Thapar Institute of Engineering and Technology, Patiala and Director, CSIR-CSIO, Chandigarh for constant patronage and support in carrying out the present research.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jitendra Virmani.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kriti, Virmani, J. & Agarwal, R. Deep feature extraction and classification of breast ultrasound images. Multimed Tools Appl 79, 27257–27292 (2020). https://doi.org/10.1007/s11042-020-09337-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09337-z

Keywords

Navigation