Nowadays Computer-Aided Diagnosis (CAD) models, particularly those based on deep learning, have been widely used to analyze histopathological images in breast cancer diagnosis. However, due to the limited availability of such images, it is always tedious to train deep learning models that require a huge amount of training data. In this paper, we propose a new deep learning-based CAD framework that can work with less amount of training data.
We use pre-trained models to extract image features that can then be used with any classifier. Our proposed features are extracted by the fusion of two different types of features (foreground and background) at two levels (whole-level and part-level). Foreground and background feature to capture information about different structures and their layout in microscopic images of breast tissues. Similarly, part-level and whole-level features capture are useful in detecting interesting regions scattered in high-resolution histopathological images at local and whole image levels. At each level, we use VGG16 models pre-trained on ImageNet and Places datasets to extract foreground and background features, respectively. All features are extracted from mid-level pooling layers of such models.
We show that proposed fused features with a Support Vector Classifier (SVM) produce better classification accuracy than recent methods on BACH dataset and our approach is orders of magnitude faster than the best performing recent method (EMS-Net).
We believe that our method would be another alternative in the diagnosis of breast cancer because of performance and prediction time.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
The datasets are publicly available.
Source codes will be made publicly available upon acceptance of the paper.
https://www.aihw.gov.au/reports/cancer/cancer-in-Australia-2019 (Accessed date: 15/02/2020).
Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A. Classification of breast cancer histology images using convolutional neural networks. PLoS ONE. 2017;12(6):e0177544.
Aresta G, Araújo T, Kwok S, Chennamsetty SS, Safwan M, Alex V, Marami B, Prastawa M, Chan M, Donovan M, et al. Bach: Grand challenge on breast cancer histology images. Med Image Anal. 2019;56:122–39.
Barker J, Hoogi A, Depeursinge A, Rubin DL. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med Image Anal. 2016;30:60–71.
Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.
Campanella G, Silva VWK, Fuchs TJ. Terabyte-scale deep multiple instance learning for classification and localization in pathology. arXiv preprint. arXiv:180506983 (2018).
Chen T, Guestrin C. Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 785–794 (2016).
Chennamsetty SS, Safwan M, Alex V. Classification of breast cancer histology image using ensemble of pre-trained neural networks. In: Proceedings of the international conference on image analysis and recognition (ICIAR), pp 804–811 (2018).
Chollet F, et al. Keras. https://github.com/fchollet/keras (2015).
Cristianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods. Cambridge: Cambridge University Press; 2000.
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision pattern and recognition (CVPR) (2009).
Falk T, Mai D, Bensch R, Çiçek O, Abdulkadir A, Marrakchi Y, Böhm A, Deubner J, Jäckel Z, Seiwald K, et al. U-net: deep learning for cell counting, detection, and morphometry. Nat Methods. 2019;16(1):67–70.
Gandomkar Z, Brennan PC, Mello-Thoms C. Mudern: multi-category classification of breast histopathological image using deep residual networks. Artif Intell Med. 2018;88:14–24.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778 (2016).
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 4700–4708 (2017).
Jiang Y, Chen L, Zhang H, Xiao X. Breast cancer histopathological image classification using convolutional neural networks with small se-resnet module. PLoS ONE. 2019;14(3):e0214587.
Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. Lightgbm: a highly efficient gradient boosting decision tree. In: Proceedings of the advances in neural information processing systems, pp 3146–3154 (2017).
Khan AM, Sirinukunwattana K, Rajpoot N. Geodesic geometric mean of regional covariance descriptors as an image-level descriptor for nuclear atypia grading in breast histology images. In: Proceedings of the international workshop on machine learning in medical imaging (MLMI), pp 101–108 (2014).
Kumar A, Singh SK, Saxena S, Lakshmanan K, Sangaiah AK, Chauhan H, Shrivastava S, Singh RK. Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer. Inf Sci. 2020;508:405–21.
Kwok S. Multiclass classification of breast cancer in whole-slide images. In: Proceedings of the international conference on image analysis and recognition (ICIAR), pp 931–940 (2018).
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.
Mehta S, Mercan E, Bartlett J, Weaver D, Elmore JG, Shapiro L. Y-net: joint segmentation and classification for diagnosis of breast biopsy images. In: Proceedings of the international conference on medical image computing and computer-assisted intervention (MICCAI), pp 893–901 (2018).
Murtagh F. Multilayer perceptrons for classification and regression. Neurocomputing. 1991;2(5–6):183–97.
Nazeri K, Aminpour A, Ebrahimi M. Two-stage convolutional neural network for breast cancer histology image classification. In: Proceedings of the international conference image analysis and recognition (ICIAR). Springer, pp 717–726 (2018).
Pimkin A, Makarchuk G, Kondratenko V, Pisov M, Krivov E, Belyaev M. Ensembling neural networks for digital pathology images classification and segmentation. In: Proceedings of the international conference on image analysis recognition (ICIAR), pp 877–886 (2018).
Qi X, Xing F, Foran DJ, Yang L. Robust segmentation of overlapping cells in histopathology specimens using parallel seed detection and repulsive level set. IEEE Trans Biomed Eng. 2011;59(3):754–65.
Rakhlin A, Shvets A, Iglovikov V, Kalinin AA. Deep convolutional neural networks for breast cancer histology image analysis. In: Proceedings of the international conference on image analysis and recognition (ICIAR), pp 737–744 (2018).
Rossum G. Python reference manual. Technical report. Amsterdam: Stichting Mathematisch Centrum; 1995.
Roy K, Banik D, Bhattacharjee D, Nasipuri M. Patch-based system for classification of breast histology images using deep learning. Comput Med Imaging Graph. 2019;71:90–103.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv:14091556 (2014).
Sitaula C, Xiang Y, Basnet A, Aryal S, Lu X. Hdf: Hybrid deep features for scene image representation. In: Proceedings of the international joint conference on neural networks (IJCNN), pp 1–8 (2020).
Sudharshan P, Petitjean C, Spanhol F, Oliveira LE, Heutte L, Honeine P. Multiple instance learning for histopathological breast cancer image classification. Expert Syst Appl. 2019;117:103–11.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–9 (2015).
Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv preprint. arXiv:160207261 (2016a).
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826 (2016b).
Vapnik V. The nature of statistical learning theory. Berlin: Springer; 2013.
Vink JP, Van Leeuwen M, Van Deurzen C, de Haan G. Efficient nucleus detector in histopathology images. J Microsc. 2013;249(2):124–35.
Yan R, Ren F, Wang Z, Wang L, Zhang T, Liu Y, Rao X, Zheng C, Zhang F. Breast cancer histopathological image classification using a hybrid deep neural network. Methods. 2020;173:52–60.
Yang Z, Ran L, Zhang S, Xia Y, Zhang Y. Ems-net: ensemble of multiscale convolutional neural networks for classification of breast cancer histology images. Neurocomputing. 2019;366:46–53.
Zhou B, Khosla A, Lapedriza A, Torralba A, Oliva A. Places: an image database for deep scene understanding. arXiv preprint. arXiv:161002055 (2016).
We would like to thank ICIAR2018 grand challenge for providing a histopathological image dataset to use in our research. Dr Sunil Aryal is supported by the Air Force Office of Scientific Research grant under Award Number FA2386-20-1-4005.
There are no financial supports to complete this work.
Conflict of interest
We would like to confirm that there are no known conflict of interests exist.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sitaula, C., Aryal, S. Fusion of whole and part features for the classification of histopathological image of breast tissue. Health Inf Sci Syst 8, 38 (2020). https://doi.org/10.1007/s13755-020-00131-7
- Histopathological images
- Breast cancer
- Image classification
- Deep learning
- Computer-aided diagnosis