Skip to main content
SpringerLink
Log in

Ensembling handcrafted features with deep features: an analytical study for classification of routine colon cancer histopathological nuclei images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The use of Deep Learning (DL) based methods in medical histopathology images have been one of the most sought after solutions to classify, segment, and detect diseased biopsy samples. However, given the complex nature of medical datasets due to the presence of intra-class variability and heterogeneity, the use of complex DL models might not give the optimal performance up to the level which is suitable for assisting pathologists. Therefore, ensemble DL methods with the scope of including domain agnostic handcrafted Features (HC-F) inspired this work. We have, through experiments, tried to highlight that a single DL network (domain-specific or state of the art pre-trained models) cannot be directly used as the base model without proper analysis with the relevant dataset. We have used F1-measure, Precision, Recall, AUC, and Cross-Entropy Loss to analyse the performance of our approaches. We observed from the results that the DL features ensemble bring a marked improvement in the overall performance of the model, whereas, domain agnostic HC-F remains dormant on the performance of the DL models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (surf). Comput Vis Image Understand 110(3):346–359

    Article  Google Scholar 

  2. Bejnordi BE, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens G, Van Der Laak JA, Hermsen M, Manson QF, Balkenhol M et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama 318(22):2199–2210

    Article  Google Scholar 

  3. Carneiro G, Peng T, Bayer C, Navab N (2015) Weakly-supervised structured output learning with flexible and latent graphs using high-order loss functions. In: Proceedings of the IEEE international conference on computer vision, pp 648–656

  4. Chen CL, Mahjoubfar A, Tai LC, Blaby IK, Huang A, Niazi KR, Jalali B (2016) Deep learning in label-free cell classification. Scientific Reports 6:21471

    Article  Google Scholar 

  5. Demir C, Yener B (2005) Automated cancer diagnosis based on histopathological images: a systematic survey. Rensselaer Polytechnic Institute, Tech. Rep

  6. Dhandra B, Hegadi R, Hangarge M, Malemath V (2006) Endoscopic image classification based on active contours without edges. In: 2006 1st international conference on digital information management. IEEE, pp 167–172

  7. Diamond J, Anderson NH, Bartels PH, Montironi R, Hamilton PW (2004) The use of morphological characteristics and texture analysis in the identification of tissue composition in prostatic neoplasia. Human Pathology 35(9):1121–1131

    Article  Google Scholar 

  8. Doyle S, Hwang M, Shah K, Madabhushi A, Feldman M, Tomaszeweski J (2007) Automated grading of prostate cancer using architectural and textural image features. In: 2007 4th IEEE international symposium on biomedical imaging: from nano to macro. IEEE, pp 1284–1287

  9. Dubey SR, Singh S, Singh RK (2015) Local bit-plane decoded pattern: a novel feature descriptor for biomedical image retrieval. IEEE J Biomed Health Inform 20(4):1139–1147

    Article  Google Scholar 

  10. Dubey SR, Singh S, Singh RK (2015) Local diagonal extrema pattern: a new and efficient feature descriptor for ct image retrieval. IEEE Signal Processing Letters 22(9):1215–1219

    Article  Google Scholar 

  11. Dubey SR, Singh S, Singh RK (2015) Local neighbourhood-based robust colour occurrence descriptor for colour image retrieval. IET Image Process 9(7):578–586

    Article  Google Scholar 

  12. Dubey SR, Singh S, Singh RK (2015) Local wavelet pattern: a new feature descriptor for image retrieval in medical ct databases. IEEE Trans Image Process 24 (12):5892–5903

    Article  MathSciNet  Google Scholar 

  13. Dubey SR, Singh S, Singh RK (2015) Rotation and scale invariant hybrid image descriptor and retrieval. Comput Elect Eng 46:288–302

    Article  Google Scholar 

  14. Fei-Fei L, Perona P (2005) A bayesian hierarchical model for learning natural scene categories. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 2. IEEE, pp 524–531

  15. Gao Z, Wang L, Zhou L, Zhang J (2016) Hep-2 cell image classification with deep convolutional neural networks. IEEE J Biomed Health Inform 21(2):416–428

    Article  Google Scholar 

  16. Genest C, Zidek JV, et al. (1986) Combining probability distributions: a critique and an annotated bibliography. Stat Sci 1(1):114–135

    Article  MathSciNet  Google Scholar 

  17. Gil J, Wu H, Wang BY (2002) Image analysis and morphometry in the diagnosis of breast cancer. Microscopy Research and Technique 59(2):109–118

    Article  Google Scholar 

  18. Graham S, Vu QD, Raza SEA, Azam A, Tsang YW, Kwak JT, Rajpoot N (2019) Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal 101563:58

    Google Scholar 

  19. Gurcan MN, Boucheron L, Can A, Madabhushi A, Rajpoot N, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147

    Article  Google Scholar 

  20. He H, Bai Y, Garcia EA, Li S (2008) Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE World Congress on Computational Intelligence). IEEE, pp 1322–1328

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  22. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  23. Jafari-Khouzani K, Soltanian-Zadeh H (2003) Multiwavelet grading of pathological images of prostate. IEEE Trans Biomed Eng 50(6):697–704

    Article  Google Scholar 

  24. Keenan SJ, Diamond J, Glenn McCluggage W, Bharucha H, Thompson D, Bartels PH, Hamilton PW (2000) An automated machine vision system for the histological grading of cervical intraepithelial neoplasia (cin). J Pathology 192 (3):351–362

    Article  Google Scholar 

  25. Litjens G, Bandi P, Ehteshami Bejnordi B, Geessink O, Balkenhol M, Bult P, Halilovic A, Hermsen M, van de Loo R, Vogels R et al (2018) 1399 h&e-stained sentinel lymph node sections of breast cancer patients: the camelyon dataset. Gigascience 7(6):giy065

    Article  Google Scholar 

  26. Marshall WW, McWhortor WF (1989) Method and apparatus for pattern recognition. US Patent 4,817,176

  27. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59

    Article  Google Scholar 

  28. Pearson K (1901) Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2 (11):559–572

    Article  Google Scholar 

  29. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  30. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  31. Sims A, Bennett M, Murray A (2003) Image analysis can be used to detect spatial changes in the histopathology of pancreatic tumours. Phys Med Biols 48(13):N183

    Article  Google Scholar 

  32. Sirinukunwattana K, e Ahmed Raza S, Tsang YW, Snead DR, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35 (5):1196–1206

    Article  Google Scholar 

  33. Sirinukunwattana K, Snead DR, Rajpoot NM (2015) A novel texture descriptor for detection of glandular structures in colon histology images. In: Medical imaging 2015: digital pathology. International Society for Optics and Photonics, vol 9420, p 94200s

  34. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  35. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, pp 2818–2826

  36. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Medical Imaging 35(5):1299–1312

    Article  Google Scholar 

  37. Torrey L, Shavlik J (2009) Transfer learning. Handbook of research on machine learning applications, vol 3

  38. Tripathi S, Singh S (2018) Histopathological image classification: defying deep architectures on complex data. In: International conference on recent trends in image processing and pattern recognition. Springer, pp 361–370

  39. Wang H, Roa AC, Basavanhally AN, Gilmore HL, Shih N, Feldman M, Tomaszewski J, Gonzalez F, Madabhushi A (2014) Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. Journal of Medical Imaging 1(3):034003

    Article  Google Scholar 

  40. Weyn B, Van De Wouwer G, Van Daele A, Scheunders P, Van Dyck D, Van Marck E, Jacob W (1998) Automated breast tumor diagnosis and grading based on wavelet chromatin texture description. Cytometry: The Journal of the International Society for Analytical Cytology 33(1):32–40

    Article  Google Scholar 

  41. Xu Y, Jia Z, Ai Y, Zhang F, Lai M, Eric I, Chang C (2015) Deep convolutional activation features for large scale brain tumor histopathology image classification and segmentation. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 947–951

  42. Yuan Y, Failmezger H, Rueda OM, Ali HR, Gräf S, Chin SF, Schwarz RF, Curtis C, Dunning MJ, Bardwell H et al (2012) Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Science Translational Medicine 4(157):157ra143–157ra143

    Article  Google Scholar 

  43. Zhang J, Xia Y, Xie Y, Fulham M, Feng DD (2018) Classification of medical images in the biomedical literature by jointly using deep and handcrafted visual features. IEEE J Biomed Health Inform 22(5):1521–1530

    Article  Google Scholar 

Download references

Acknowledgements

This research was carried out in the Indian Institute of Information Technology, Allahabad and supported by the Ministry of Human Resource and Development, Government of India. We are also grateful to the NVIDIA corporation for supporting our research in this area by granting us TitanX (PASCAL) GPU.

Author information

Authors and Affiliations

  1. Indian Institute of Information Technology Allahabad, Prayagraj, Uttar Pradesh, 211015, India

    Suvidha Tripathi & Satish Kumar Singh

Authors
  1. Suvidha Tripathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satish Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Suvidha Tripathi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tripathi, S., Singh, S.K. Ensembling handcrafted features with deep features: an analytical study for classification of routine colon cancer histopathological nuclei images. Multimed Tools Appl 79, 34931–34954 (2020). https://doi.org/10.1007/s11042-020-08891-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08891-w

Keywords

Search

Navigation