Skip to main content

Advertisement

SpringerLink
Log in

Nuclei detection in breast histopathology images with iterative correction

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

This work presents a deep network architecture to improve nuclei detection performance and achieve the high localization accuracy of nuclei in breast cancer histopathology images. The proposed model consists of two parts, generating nuclear candidate module and refining nuclear localization module. We first design a novel patch learning method to obtain high-quality nuclear candidates, where in addition to categories, location representations are also added to the patch information to implement the multi-task learning process of nuclear classification and localization; meanwhile, the deep supervision mechanism is introduced to obtain the coherent contributions from each scale layer. In order to refine nuclear localization, we propose an iterative correction strategy to make the prediction progressively closer to the ground truth, which significantly improves the accuracy of nuclear localization and facilitates neighbor size selection in the nonmaximum suppression step. Experimental results demonstrate the superior performance of our method for nuclei detection on the H&E stained histopathological image dataset as compared to previous state-of-the-art methods, especially in multiple cluttered nuclei detection, can achieve better results than existing techniques.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References 

  1. Al-Janabi S, Huisman A, Willems SM, van Diest PJ (2012) Digital slide images for primary diagnostics in breast pathology: a feasibility study. Hum Pathol 43:2318–2325

    Article  CAS  PubMed  Google Scholar 

  2. Elmore JG, Longton GM, Carney PA et al (2015) Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA - J Am Med Assoc 313(11):1122–1132

    Article  CAS  Google Scholar 

  3. H. Viray, K. Li, T.A. Long., P. Vasalos, et al., A prospective, multi-institutional diagnostic trial to determine pathologist accuracy in estimation of percentage of malignant cells, Arch. Pathol. Lab. Med., vol. 137, no. 11, pp. 1545–1549, 2013.

  4. Elston CW, Ellis IO (1991) Pathological prognostic factors in breast cancer I The value of histological grade in breast cancer: experience from a large study with long-term follow-up. Histopathology 19(5):403–410

    Article  CAS  PubMed  Google Scholar 

  5. Zhang X, Liu W, Dundar M, Badve S, Zhang S (2015) Towards largescale histopathological image analysis: hashing-based image retrieval. IEEE Trans Med Imaging 34(2):496–506

    Article  PubMed  Google Scholar 

  6. W. Lu, S. Graham, M. Bilal, N. Rajpoot, and F. Minhas, Capturing cellular topology in multi-gigapixel pathology images, in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, 2020, pp. 1049–1058.

  7. Irshad H, Veillard A, Roux L, Racoceanu D (2014) Methods for nuclei detection, segmentation, and classification in digital histopathology: a review—current status and future potential. IEEE Rev Biomed Eng 7:97–114

    Article  PubMed  Google Scholar 

  8. Nawaz S, Yuan Y (2016) Computational pathology: exploring the spatial dimension of tumor ecology. Cancer Lett 380(1):296–303

    Article  CAS  PubMed  Google Scholar 

  9. S Ren, K He, R Girshick, J Sun (2015) Faster R-CNN: towards realtime object detection with region proposal networks, in Proc. Adv Neural Inf Process Syst (NIPS) 1: 91-99

  10. S. Yousefi and Y. Nie, Transfer learning from nucleus detection to classification in histopathology images, in Proc. IEEE Int. Symp. Biomed. Imaging (ISBI), 2019, pp. 957–960.

  11. T. Mahmood, M. Arsalan, M. Owais, M.B. Lee, and K.R. Park (2020) Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster R-CNN and Deep CNNs. J. Clin. Med 9 3.

  12. Xu J, Xiang L, Liu Q, Gilmore H, Wu J, Tang J, Madabhushi A (2016) Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging 35(1):119–130

    Article  PubMed  Google Scholar 

  13. Höfener H, Homeyer A, Weiss N, Molin J, Lundström CF, Hahn HK (2018) Deep learning nuclei detection: a simple approach can deliver state-of-the-art results. Comput Med Imag Grap 70:43–52

    Article  Google Scholar 

  14. Brieu N, Schmidt G (2017) Learning size adaptive local maxima selection for robust nuclei detection in histopathology images. In: Proc IEEE Intl Symp Biomed Imaging, pp 937–941

  15. Chen T, Chefdhotel C (2014) Deep learning based automatic immune cell detection for immunohistochemistry images. In: Proc Intl Workshop Mach Learn Med Imaging, pp 17–24

  16. Bodla N, Singh B, Chellappa R, Davis LS (2017) Soft-NMS improving object detection with one line of code. In: Proc IEEE Intl Conf Computer Vision (ICCV), pp 5562–5570

  17. Tychsen-Smith L, Petersson L (2018) Improving object localization with fitness NMS and bounded IoU loss. In: Proc. IEEE Conf Comput Vis Pattern Recog (CVPR), pp 6877–6885

  18. Oberweger M, Wohlhart P, Lepetit V (2015) Training a feedback loop for hand pose estimation. In: Proc IEEE Intl Conf Computer Vision (ICCV), pp 3316–3324

  19. Carreira J, Agrawal P, Fragkiadaki K, Malik J (2016) Human pose estimation with iterative error feedback. In: Proc IEEE Conf Comput Vis Pattern Recog (CVPR), pp 4733–4742

  20. Xie S, Tu Z (2017) Holistically-nested edge detection. Int J Comput Vis 125(1):3–18

    Article  Google Scholar 

  21. Liu Y, Cheng M-M, Hu X, Bian J-W, Zhang L, Bai X, Tang J (2019) Richer convolutional features for edge detection. IEEE Trans Pattern Anal Mach Intell 41(8):1939–1946

    Article  PubMed  Google Scholar 

  22. Xie Y, Xing F, Kong X, Su H, Yang L (2015) Beyond classification: structured regression for robust cell detection using convolutional neural network. In: Proc Medical Image Computing and Computer-assisted Intervention (MICCAI), pp 358–365

  23. Lu C, Mandal M (2014) Toward automatic mitotic cell detection and segmentation in multispectral histopathological images. IEEE J Biomed Health Inform 18(2):594–605

    Article  PubMed  Google Scholar 

  24. Paul A, Mukherjee DP (2015) Mitosis detection for invasive breast cancer grading in histopathological images. IEEE Trans Image Process 24(11):4041–4054

    Article  PubMed  Google Scholar 

  25. Sirinukunwattana K, Raza SEA, Tsang Y-W, Snead D, Cree I, Rajpoot N (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35(5):1196–1206

    Article  PubMed  Google Scholar 

  26. Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: Proc Medical Image Computing and Computer-assisted Intervention (MICCAI), pp 411–418

  27. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  PubMed  Google Scholar 

  28. Graham S, Vu QD, Raza SEA, Azam A, Tsang YW, Kwak JT, Rajpoot N (2019) Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal 58:1–15

    Article  Google Scholar 

  29. Chen H, Dou Q, Wang X, Qin J, Heng PA (2016) Mitosis detection in breast cancer histology images via deep cascaded networks. In: Proc AAAI Conf Artificial Intelligence, pp 1160–1166

  30. Zhang J, Hu H, Chen S, Huang Y, Guan Q (2016) Cancer cells detection in phase-contrast microscopy images based on faster R-CNN. In: Proc Intl Symp Computational Intelligence and Design, pp 363–367

  31. Yi J, Wu P, Hoeppner DJ, Metaxas D (2017) Fast neural cell detection using light-weight SSD neural network. In: Proc IEEE Conf Comput Vis Pattern Recog Workshops, pp 108–112

  32. Sun Y, Huang X, Zhou H, Zhang Q (2021) SRPN: similarity-based region proposal networks for nuclei and cells detection in histology images. Med Image Anal 72:1–11

    Article  Google Scholar 

  33. Vargas V, Koller J (2018) Semi-supervised generative adversarial network (SGAN) for nuclei detection on breast cancer histopathology images. [Online]. Available: https://github.com/vmvargas/GAN-for-Nuclei-Detection. Accessed 7 May 2021

  34. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Proc Adv Neural Inf Process Syst (NIPS), pp 2843–2851

  35. Woo S, Park J, Lee JY, Kweon IS (2018) CBAM: convolutional block attention module. In: Proc European Conf Computer Vision (ECCV), pp 3–19

  36. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023

    Article  PubMed  Google Scholar 

  37. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proc IEEE Intl Conf Computer Vision (ICCV), pp 1026–1034

  38. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proc Intl Conf Machine Learning (ICML), pp 448–456

  39. Song G, Liu Y, Wang X (2020) Revisiting the sibling head in object detector. In: Proc IEEE Conf Comput Vis Pattern Recog (CVPR), pp 11560–11569

  40. Wu Y, Chen Y, Yuan L, Liu Z, Wang L, Li H, Fu Y (2020) Rethinking classification and localization for object detection. In: Proc IEEE Conf Comput Vis Pattern Recog (CVPR), pp 10183–10192

  41. Wang CY, Bochkovskiy A, Liao HYM (2022) YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. https://doi.org/10.48550/arXiv.2207.02696

Download references

Funding

This work was supported by Special Fund for Basic Scientific Research Business Expenses of Central Universities (Grant numbers CZY22014).

Author information

Author notes

  1. Ziyi Liu and Yu Cai have equal contribution.

Authors and Affiliations

  1. School of Biomedical Engineering, South Central Minzu University, Wuhan, 430074, People’s Republic of China

    Ziyi Liu, Yu Cai & Qiling Tang

  2. Affiliated Yantai Yuhuangding Hospital of Qingdao University, Yantai, 264001, People’s Republic of China

    Ziyi Liu

Authors
  1. Ziyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiling Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Ziyi Liu, Yu Cai, and Qiling Tang. The first draft of the manuscript was written by Qiling Tang and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Qiling Tang.

Ethics declarations

Ethics approval and consent to participate

Not applicable. All analyses were based on public dataset; thus no ethical approval and patient consent are required.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Research involving human participants and/or animals

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work described has not been submitted elsewhere for publication, in whole or in part, and all the authors listed have approved the manuscript that is enclosed. 

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Cai, Y. & Tang, Q. Nuclei detection in breast histopathology images with iterative correction. Med Biol Eng Comput 62, 465–478 (2024). https://doi.org/10.1007/s11517-023-02947-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02947-3

Keywords

Search

Navigation