Using Transfer Learning with Convolutional Neural Networks to Diagnose Breast Cancer from Histopathological Images

  • Weiming Zhi
  • Henry Wing Fung Yueng
  • Zhenghao Chen
  • Seid Miad Zandavi
  • Zhicheng Lu
  • Yuk Ying Chung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10637)


Diagnosis from histopathological images is the gold standard in diagnosing breast cancer. This paper investigates using transfer learning with convolutional neural networks to automatically diagnose breast cancer from patches of histopathological images. We compare the performance of using transfer learning with an off-the-shelf deep convolutional neural network architecture, VGGNet, and a shallower custom architecture. Our proposed final ensemble model, which contains three custom convolutional neural network classifiers trained using transfer learning, achieves a significantly higher image classification accuracy on the large public benchmark dataset than the current best results, for all image resolution levels.


Histopathological image analysis Convolutional neural networks Transfer learning Deep learning 


  1. 1.
    Ferlay, J., Soerjomataram, I., Ervik, M., Dikshit, M., Eser, S., Mathers, C., et al.: GLOBOCAN 2012 v1.0, cancer incidence and mortality worldwide: IARC CancerBase no. 11 (2013)Google Scholar
  2. 2.
    Rubin, R., Strayer, D., Rubin, E., McDonald, J.: Rubin’s pathology: Clinicopathologic Foundations of Medicine. Lippincott Williams and Wilkins (2007).Google Scholar
  3. 3.
    Pan, S., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 13451359 (2010)CrossRefGoogle Scholar
  4. 4.
    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Computer Vision and Pattern Recognition, CVPR (2009)Google Scholar
  5. 5.
    Spanhol, F.A., Oliveira, L.S., Petitjean, C., Heutte, L.: A Dataset for Breast Cancer Histopathological Image Classification. IEEE Trans. Biomed. Eng. 63(7), 1455–1462 (2016)CrossRefGoogle Scholar
  6. 6.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large scale image recognition. In: ICLR 2015 (2015)Google Scholar
  7. 7.
    Doyle, S., Agner, S., Madabhushi, A., Feldman, M., Tomaszewski, J.: Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE International Symposium on Biomedical Imaging, vol. 29, pp. 496–499 (2008)Google Scholar
  8. 8.
    van Diest, P.J., Fleege, J.C., Baak, J.P.A.: Syntactic structure analysis in invasive breast cancer: analysis of reproducibility, biologic background, and prognostic value. Human Pathol. 23(8), 876–883 (1991)CrossRefGoogle Scholar
  9. 9.
    Sewak, M., Vaidya, P., Chan, C.C., Duan, Z.: SVM approach to breast cancer classification. In: 2nd International Multi-Symposiums on Computer and Computational Sciences, IMSCCS 2007, Iowa City, IA, pp. 32–37 (2007)Google Scholar
  10. 10.
    George, Y.M., Zayed, H.H., Roushdy, M.I., Elbagoury, B.M.: Remote computer-aided breast cancer detection and diagnosis system based on cytological images. IEEE Syst. J. 8(3), 949–964 (2014)CrossRefGoogle Scholar
  11. 11.
    Spanhol, F.A., Oliveira, L.S., Petitjean, C., Heutte, L.: Breast cancer histopathological image classification using convolutional neural networks. In: 2016 International Joint Conference on Neural Networks, IJCNN, Vancouver, BC, pp. 2560–2567 (2016)Google Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  13. 13.
    Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K.M., Kim, L., Summers, R.M.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 11701181 (2016)CrossRefGoogle Scholar
  14. 14.
    Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016)CrossRefGoogle Scholar
  15. 15.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)Google Scholar
  16. 16.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks?. In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar
  17. 17.
    Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2016)Google Scholar
  18. 18.
    Kittler, J., Hatef, M., Duin, R.P., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Weiming Zhi
    • 1
  • Henry Wing Fung Yueng
    • 2
  • Zhenghao Chen
    • 2
  • Seid Miad Zandavi
    • 2
  • Zhicheng Lu
    • 2
  • Yuk Ying Chung
    • 2
  1. 1.Department of Engineering ScienceUniversity of AucklandAucklandNew Zealand
  2. 2.School of Information TechnologiesUniversity of SydneySydneyAustralia

Personalised recommendations