Skip to main content
SpringerLink
Log in

Fusion of whole and part features for the classification of histopathological image of breast tissue

  • Research
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

Purpose

Nowadays Computer-Aided Diagnosis (CAD) models, particularly those based on deep learning, have been widely used to analyze histopathological images in breast cancer diagnosis. However, due to the limited availability of such images, it is always tedious to train deep learning models that require a huge amount of training data. In this paper, we propose a new deep learning-based CAD framework that can work with less amount of training data.

Methods

We use pre-trained models to extract image features that can then be used with any classifier. Our proposed features are extracted by the fusion of two different types of features (foreground and background) at two levels (whole-level and part-level). Foreground and background feature to capture information about different structures and their layout in microscopic images of breast tissues. Similarly, part-level and whole-level features capture are useful in detecting interesting regions scattered in high-resolution histopathological images at local and whole image levels. At each level, we use VGG16 models pre-trained on ImageNet and Places datasets to extract foreground and background features, respectively. All features are extracted from mid-level pooling layers of such models.

Results

We show that proposed fused features with a Support Vector Classifier (SVM) produce better classification accuracy than recent methods on BACH dataset and our approach is orders of magnitude faster than the best performing recent method (EMS-Net).

Conclusion

We believe that our method would be another alternative in the diagnosis of breast cancer because of performance and prediction time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets are publicly available.

Code availability

Source codes will be made publicly available upon acceptance of the paper.

Notes

  1. https://www.aihw.gov.au/reports/cancer/cancer-in-Australia-2019 (Accessed date: 15/02/2020).

  2. https://scikit-learn.org/stable/about.html.

References

  1. Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A. Classification of breast cancer histology images using convolutional neural networks. PLoS ONE. 2017;12(6):e0177544.

    Article  Google Scholar 

  2. Aresta G, Araújo T, Kwok S, Chennamsetty SS, Safwan M, Alex V, Marami B, Prastawa M, Chan M, Donovan M, et al. Bach: Grand challenge on breast cancer histology images. Med Image Anal. 2019;56:122–39.

    Article  Google Scholar 

  3. Barker J, Hoogi A, Depeursinge A, Rubin DL. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med Image Anal. 2016;30:60–71.

    Article  Google Scholar 

  4. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

    Article  Google Scholar 

  5. Campanella G, Silva VWK, Fuchs TJ. Terabyte-scale deep multiple instance learning for classification and localization in pathology. arXiv preprint. arXiv:180506983 (2018).

  6. Chen T, Guestrin C. Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 785–794 (2016).

  7. Chennamsetty SS, Safwan M, Alex V. Classification of breast cancer histology image using ensemble of pre-trained neural networks. In: Proceedings of the international conference on image analysis and recognition (ICIAR), pp 804–811 (2018).

  8. Chollet F, et al. Keras. https://github.com/fchollet/keras (2015).

  9. Cristianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods. Cambridge: Cambridge University Press; 2000.

    Book  Google Scholar 

  10. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision pattern and recognition (CVPR) (2009).

  11. Falk T, Mai D, Bensch R, Çiçek O, Abdulkadir A, Marrakchi Y, Böhm A, Deubner J, Jäckel Z, Seiwald K, et al. U-net: deep learning for cell counting, detection, and morphometry. Nat Methods. 2019;16(1):67–70.

    Article  Google Scholar 

  12. Gandomkar Z, Brennan PC, Mello-Thoms C. Mudern: multi-category classification of breast histopathological image using deep residual networks. Artif Intell Med. 2018;88:14–24.

    Article  Google Scholar 

  13. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778 (2016).

  14. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 4700–4708 (2017).

  15. Jiang Y, Chen L, Zhang H, Xiao X. Breast cancer histopathological image classification using convolutional neural networks with small se-resnet module. PLoS ONE. 2019;14(3):e0214587.

    Article  Google Scholar 

  16. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. Lightgbm: a highly efficient gradient boosting decision tree. In: Proceedings of the advances in neural information processing systems, pp 3146–3154 (2017).

  17. Khan AM, Sirinukunwattana K, Rajpoot N. Geodesic geometric mean of regional covariance descriptors as an image-level descriptor for nuclear atypia grading in breast histology images. In: Proceedings of the international workshop on machine learning in medical imaging (MLMI), pp 101–108 (2014).

  18. Kumar A, Singh SK, Saxena S, Lakshmanan K, Sangaiah AK, Chauhan H, Shrivastava S, Singh RK. Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer. Inf Sci. 2020;508:405–21.

    Article  Google Scholar 

  19. Kwok S. Multiclass classification of breast cancer in whole-slide images. In: Proceedings of the international conference on image analysis and recognition (ICIAR), pp 931–940 (2018).

  20. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.

    Article  Google Scholar 

  21. Mehta S, Mercan E, Bartlett J, Weaver D, Elmore JG, Shapiro L. Y-net: joint segmentation and classification for diagnosis of breast biopsy images. In: Proceedings of the international conference on medical image computing and computer-assisted intervention (MICCAI), pp 893–901 (2018).

  22. Murtagh F. Multilayer perceptrons for classification and regression. Neurocomputing. 1991;2(5–6):183–97.

    Article  MathSciNet  Google Scholar 

  23. Nazeri K, Aminpour A, Ebrahimi M. Two-stage convolutional neural network for breast cancer histology image classification. In: Proceedings of the international conference image analysis and recognition (ICIAR). Springer, pp 717–726 (2018).

  24. Pimkin A, Makarchuk G, Kondratenko V, Pisov M, Krivov E, Belyaev M. Ensembling neural networks for digital pathology images classification and segmentation. In: Proceedings of the international conference on image analysis recognition (ICIAR), pp 877–886 (2018).

  25. Qi X, Xing F, Foran DJ, Yang L. Robust segmentation of overlapping cells in histopathology specimens using parallel seed detection and repulsive level set. IEEE Trans Biomed Eng. 2011;59(3):754–65.

    Google Scholar 

  26. Rakhlin A, Shvets A, Iglovikov V, Kalinin AA. Deep convolutional neural networks for breast cancer histology image analysis. In: Proceedings of the international conference on image analysis and recognition (ICIAR), pp 737–744 (2018).

  27. Rossum G. Python reference manual. Technical report. Amsterdam: Stichting Mathematisch Centrum; 1995.

    Google Scholar 

  28. Roy K, Banik D, Bhattacharjee D, Nasipuri M. Patch-based system for classification of breast histology images using deep learning. Comput Med Imaging Graph. 2019;71:90–103.

    Article  Google Scholar 

  29. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv:14091556 (2014).

  30. Sitaula C, Xiang Y, Basnet A, Aryal S, Lu X. Hdf: Hybrid deep features for scene image representation. In: Proceedings of the international joint conference on neural networks (IJCNN), pp 1–8 (2020).

  31. Sudharshan P, Petitjean C, Spanhol F, Oliveira LE, Heutte L, Honeine P. Multiple instance learning for histopathological breast cancer image classification. Expert Syst Appl. 2019;117:103–11.

    Article  Google Scholar 

  32. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–9 (2015).

  33. Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv preprint. arXiv:160207261 (2016a).

  34. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826 (2016b).

  35. Vapnik V. The nature of statistical learning theory. Berlin: Springer; 2013.

    MATH  Google Scholar 

  36. Vink JP, Van Leeuwen M, Van Deurzen C, de Haan G. Efficient nucleus detector in histopathology images. J Microsc. 2013;249(2):124–35.

    Article  Google Scholar 

  37. Yan R, Ren F, Wang Z, Wang L, Zhang T, Liu Y, Rao X, Zheng C, Zhang F. Breast cancer histopathological image classification using a hybrid deep neural network. Methods. 2020;173:52–60.

    Article  Google Scholar 

  38. Yang Z, Ran L, Zhang S, Xia Y, Zhang Y. Ems-net: ensemble of multiscale convolutional neural networks for classification of breast cancer histology images. Neurocomputing. 2019;366:46–53.

    Article  Google Scholar 

  39. Zhou B, Khosla A, Lapedriza A, Torralba A, Oliva A. Places: an image database for deep scene understanding. arXiv preprint. arXiv:161002055 (2016).

Download references

Acknowledgements

We would like to thank ICIAR2018 grand challenge for providing a histopathological image dataset to use in our research. Dr Sunil Aryal is supported by the Air Force Office of Scientific Research grant under Award Number FA2386-20-1-4005.

Funding

There are no financial supports to complete this work.

Author information

Authors and Affiliations

  1. School of Information Technology, Deakin University, 75 Pigdons Rd, Waurn Ponds, VIC, 3216, Australia

    Chiranjibi Sitaula & Sunil Aryal

Authors
  1. Chiranjibi Sitaula
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sunil Aryal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chiranjibi Sitaula.

Ethics declarations

Conflict of interest

We would like to confirm that there are no known conflict of interests exist.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sitaula, C., Aryal, S. Fusion of whole and part features for the classification of histopathological image of breast tissue. Health Inf Sci Syst 8, 38 (2020). https://doi.org/10.1007/s13755-020-00131-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-020-00131-7

Keywords

Search

Navigation