Improving Generalization of ENAS-Based CNN Models for Breast Lesion Classification from Ultrasound Images

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12722)


Neural Architecture Search (NAS) is one of the most recent developments in automating the design process for deep convolutional neural network (DCNN) architectures. NAS and later Efficient NAS (ENAS) based models have been adopted successfully for various applications including ultrasound image classification for breast lesions. Such a data driven approach leads to creation of DCNN models that are more applicable to the data set at hand but with a risk for model overfitting. In this paper, we first investigate the extent of the ENAS model generalization error problem by using different test data sets of ultrasound images of breast lesions. We have observed a significant reduction of overall average accuracy by nearly 10% and even more severe reduction of specificity rate by more than 20%, indicating that model generalization error is a serious issue with ENAS models for breast lesion classification in ultrasound images. To overcome the generalization error, we examined the effectiveness of a range of techniques including reducing model complexity, use of data augmentation, and use of unbalanced training sets. Experimental results show that different methods for the tuned ENAS models achieved different levels of accuracy when they are tested on internal and two external test data sets. The paper demonstrates that ENAS models trained on an unbalanced dataset with more benign cases tend to generalize well on unseen images achieving average accuracies of 85.8%, 82.7%, and 88.1% respectively for the internal and the two external test data sets not only on specificity alone, but also sensitivity. In particular, the generalization of the refined models across internal and external test data is maintained.


ENAS Ultrasound image Breast lesion classification Deep learning Reduce generalization error Imbalanced dataset 



This research is sponsored by TenD Innovations.


  1. 1.
    The International Agency for Research on Cancer (IARC) report. World Cancer Day 2021: Spotlight on IARC research related to breast cancer. International Agency for Research on Cancer. Accessed 12 May 2021
  2. 2.
    Stavros, T.A., Thickman, D., Rapp, L., Dennis, M.A., Parker, S.H., Sisney, G.: Solid breast nodules : use of sonography to distinguish lesions. Radiology 196, 123–134 (1995)CrossRefGoogle Scholar
  3. 3.
    Xie, X., Niu, J., Liu, X., Chen, Z., Tang, S., Yu, S.: A Survey on incorporating domain knowledge into deep learning for medical image analysis. Med. Image Anal. 69, 101985 (2021)CrossRefGoogle Scholar
  4. 4.
    Zhu, Y.C., et al.: A generic deep learning framework to classify thyroid and breast lesions in ultrasound images. Ultrasonics 110, 106300 (2021)CrossRefGoogle Scholar
  5. 5.
    Goodfellow, Y., Bengio, Y., Courville, A.: Deep Learning. MIT Press, USA (2016)zbMATHGoogle Scholar
  6. 6.
    Wistuba, M., Rawat, A., Pedapati, T.: A survey on neural architecture search, vol. 20, pp. 1–21 (2019). [Online]: Accessed 4 Mar 2021
  7. 7.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012)Google Scholar
  9. 9.
    Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)CrossRefGoogle Scholar
  10. 10.
    Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35(5), 1299–1312 (2016)CrossRefGoogle Scholar
  11. 11.
    Han, S., et al.: A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys. Med. Biol. 62(19), 7714–7728 (2017)CrossRefGoogle Scholar
  12. 12.
    Xiao, T., Liu, L., Li, K., Qin, W., Yu, S., Li, Z.: Comparison of transferred deep neural networks in ultrasonic breast masses discrimination. Biomed. Res. Int. 2018, 1–9 (2018)Google Scholar
  13. 13.
    Masud, M., Eldin Rashed, A.E., Hossain, M.S.: Convolutional neural network-based models for diagnosis of breast cancer. Neural Comput. Appl. 5, 1–12 (2020). Scholar
  14. 14.
    Hijab, A., Rushdi, M.A., Gomaa, M.M.: Breast cancer classification in ultrasound images using transfer learning. In: Proceedings of Fifth International Conference on Advances in Biomedical Engineering (2019)Google Scholar
  15. 15.
    Zhang, H., Han, L., Chen, K., Peng, Y., Lin, J.: Diagnostic efficiency of the breast ultrasound computer-aided prediction model based on convolutional neural network in breast cancer. J. Digit. Imaging 33(5), 1218–1223 (2020). Scholar
  16. 16.
    Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning, pp.1–16 (2017). [Online]: Accessed 12 May 2021
  17. 17.
    Zoph, B., Le, Q.: Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8697–8710 (2018)Google Scholar
  18. 18.
    Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameters sharing. In: Proceedings of International Conference on M.L, pp. 4095–4104 (2018)Google Scholar
  19. 19.
    Liu, C., et al.: Progressive neural architecture search. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 19–35. Springer, Cham (2018). Scholar
  20. 20.
    Gessert, N., Schlaefer, A.: Efficient neural architecture search on low-dimensional data for OCT image segmentation. arXiv preprint arXiv:1905.02590 (2019)
  21. 21.
    Dong, N., Xu, M., Liang, X., Jiang, Y., Dai, W., Xing, E.: Neural architecture search for adversarial medical image segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 828–836. Springer, Cham (2019). Scholar
  22. 22.
    Mohammed, A., Du, H., AlZoubi, A.: An ENAS based approach for constructing deep learning models for breast cancer recognition from ultrasound images. In: Proceedings of MIDL Conference. arXiv preprint arXiv:2005.13695 (2020)
  23. 23.
    Recht, B., Roelofs, R., Schmidt, L., Shankar, V.: Do ImageNet classifiers generalize to ImageNet? In: Proceedings of International Conference on Machine Learning, pp. 5389–5400 (2019)Google Scholar
  24. 24.
    Rice, L., Wong, E., Kolter, Z.: Overfitting an adversarially robust deep learning. In: Proceedings of International Conference on Machine Learning, pp. 8093–8104. PMLR (2020)Google Scholar
  25. 25.
    Samala, R.K., Chan, H.P., Hadjiiski, L.M., Helvie, M.A., Richter, C.D.: Generalization error analysis for deep convolutional neural network with transfer learning in breast cancer diagnosis. Phys. Med. Biol. 65(10), 105002 (2020)CrossRefGoogle Scholar
  26. 26.
    Zeimarani, B., Costa, M.G.F., Nurani, N.Z., Bianco, S.R., De Albuquerque Pereira, W.C., Filho, C.F.F.C.: Breast lesion classification in ultrasound images using deep convolutional neural network. IEEE Access. 8, 133349–133359 (2020)Google Scholar
  27. 27.
    Jiang, Y., Zhao, C., Dou, Z., Pang, L.: Neural architecture refinement: a practical way for avoiding overfitting in NAS. arXiv preprint arXiv:1905.02341 (2019)
  28. 28.
    Al-Dhabyani, W., Gomaa, M., Khaled, H., Fahmy, A.: Dataset of breast ultrasound images. Data Brief 28, 104863 (2020)CrossRefGoogle Scholar
  29. 29.
    Johnson, J.M., Khoshgoftaar, T.M.: Survey on deep learning with class imbalance. J. Big Data 6(1), 1–54 (2019). Scholar
  30. 30.
    Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 106, 249–259 (2018)CrossRefGoogle Scholar
  31. 31.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  32. 32.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2016, pp. 770–778 (2015)Google Scholar
  33. 33.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2016, pp. 2818–2826 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.The University of BuckinghamBuckinghamUK

Personalised recommendations