Skip to main content
SpringerLink
Log in

Nuclei probability and centroid map network for nuclei instance segmentation in histology images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Nuclei instance segmentation is an integral step in digital pathology workflow as it is a prerequisite for most downstream tasks such as patient survival analysis, precision medicine, and cancer prognosis. There exist many challenges such as quality of labeled data, staining variation among tissue slides, high variation among multi-organ & multi-center digital slides and overlapping nuclei that are difficult to separate. Therefore, it is important to have an automatic and robust nuclei instance segmentation model that saves the time of pathologists by delineating accurate nuclei instances. To this end, we develop a nuclei instance segmentation pipeline that estimates distance transform and nuclear masks using an encoder–decoder-based CNN model. These estimated distance transform and nuclear masks are then utilized to delineate accurate nuclei boundaries from overlapping nuclei. We demonstrate that our proposed NC-Net model is lightweight and produces state-of-the-art results on the three recently published largest nuclei instance segmentation datasets to date. Additionally, our proposed NC-Net model is faster and utilizes a fewer number of parameters for learning as compared to other top-performing nuclei instance segmentation models. The purpose of developing a lightweight and state-of-the-art model is to provide capacity building to digital pathology workflows by reducing inference times and delineating accurate nuclear instances. The implementation details and the trained models are made available at this https://github.com/nauyan/NC-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Data sharing is not applicable, as no new data are generated.

Notes

  1. https://github.com/nauyan/NC-Net.

References

  1. Cancer. https://www.who.int/news-room/fact-sheets/detail/cancer. Accessed 2020-11-08

  2. Chow AY (2010) Cell cycle control by oncogenes and tumor suppressors: driving the transformation of normal cells into cancerous cells. Nat Educ 3(9):7

    Google Scholar 

  3. Fischer AH, Jacobson KA, Rose J, Zeller R (2008) Hematoxylin and eosin staining of tissue and cell sections. Cold Spring Harbor Protocols 2008(5):4986

    Article  Google Scholar 

  4. Elmore JG, Longton GM, Carney PA, Geller BM, Onega T, Tosteson AN, Nelson HD, Pepe MS, Allison KH, Schnitt SJ et al (2015) Diagnostic concordance among pathologists interpreting breast biopsy specimens. Jama 313(11):1122–1132

    Article  Google Scholar 

  5. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A (2019) Artificial intelligence in digital pathology-new tools for diagnosis and precision oncology. Nat Rev Clin Oncol 16(11):703–715

    Article  Google Scholar 

  6. Madabhushi A, Lee G (2016) Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal 33:170–175

    Article  Google Scholar 

  7. Alsubaie N, Sirinukunwattana K, Raza SEA, Snead D, Rajpoot N ( 2018) A bottom-up approach for tumour differentiation in whole slide images of lung adenocarcinoma. In: Medical imaging 2018: digital pathology, vol 10581, p 105810. International Society for Optics and Photonics

  8. Lu C, Romo-Bucheli D, Wang X, Janowczyk A, Ganesan S, Gilmore H, Rimm D, Madabhushi A (2018) Nuclear shape and orientation features from h &e images predict survival in early-stage estrogen receptor-positive breast cancers. Lab Invest 98(11):1438–1448

    Article  Google Scholar 

  9. Lu C, Romo-Bucheli D, Wang X, Janowczyk A, Ganesan S, Gilmore H, Rimm D, Madabhushi A (2018) Nuclear shape and orientation features from h &e images predict survival in early-stage estrogen receptor-positive breast cancers. Lab Invest 98(11):1438–1448

    Article  Google Scholar 

  10. Colling R, Pitman H, Oien K, Rajpoot N, Macklin P, Histopathology Working Group CPA, Bachtiar V, Booth R, Bryant A, Bull J, et al (2019) Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice. J Pathol 249( 2), 143– 150

  11. Jimenez-del-Toro O, Otálora S, Andersson M, Eurén K, Hedlund M, Rousson M, Müller H, Atzori M ( 2017) Analysis of histopathology images: from traditional machine learning to deep learning. In: Biomedical texture analysis, pp. 281– 314. Academic Press

  12. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A (2019) Artificial intelligence in digital pathology-new tools for diagnosis and precision oncology. Nat Rev Clin Oncol 16(11):703–715

    Article  Google Scholar 

  13. Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147–171

    Article  Google Scholar 

  14. Yang X, Li H, Zhou X (2006) Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy. IEEE Trans Circuits Syst I Regular Pap 53(11):2405–2414

    Article  Google Scholar 

  15. Veta M, Van Diest PJ, Kornegoor R, Huisman A, Viergever MA, Pluim JP (2013) Automatic nuclei segmentation in h &e stained breast cancer histopathology images. PloS One 8(7):70221

    Article  Google Scholar 

  16. Ali S, Madabhushi A (2012) An integrated region-, boundary-, shape-based active contour for multiple object overlap resolution in histological imagery. IEEE Trans Med Imag 31(7):1448–1460

    Article  Google Scholar 

  17. Wienert S, Heim D, Saeger K, Stenzinger A, Beil M, Hufnagl P, Dietel M, Denkert C, Klauschen F (2012) Detection and segmentation of cell nuclei in virtual microscopy images: a minimum-model approach. Sci Rep 2(1):1–7

    Article  Google Scholar 

  18. LaTorre A, Alonso-Nanclares L, Muelas S, Peña J, DeFelipe J (2013) Segmentation of neuronal nuclei based on clump splitting and a two-step binarization of images. Exp Syst Appl 40(16):6521–6530

    Article  Google Scholar 

  19. Kwak JT, Hewitt SM, Xu S, Pinto PA, Wood BJ ( 2015) Nucleus detection using gradient orientation information and linear least squares regression. In: Medical imaging 2015: digital pathology, vol 9420, pp 152– 159. SPIE

  20. Liao M, Zhao Y-q, Li X-h, Dai P-s, Xu X-w, Zhang J-k, Zou B-j ( 2016) Automatic segmentation for cell images based on bottleneck detection and ellipse fitting. Neurocomputing 173:615–622

  21. LeCun Y, Bengio Y, Hinton G et al (2015) Deep learning. Nature 521(7553):436–444 (Google Scholar Google Scholar Cross Ref Cross Ref)

    Article  Google Scholar 

  22. Wu Y, Ji X, Ji W, Tian Y, Zhou H (2020) Casr: a context-aware residual network for single-image super-resolution. Neural Comput Appl 32(18):14533–14548

    Article  Google Scholar 

  23. Dogar GM, Shahzad M, Fraz MM (2023) Attention augmented distance regression and classification network for nuclei instance segmentation and type classification in histology images. Biomed Signal Process Control 79:104199

    Article  Google Scholar 

  24. Nasir ES, Perviaz A, Fraz MM (2022) Nuclei and glands instance segmentation in histology images: a narrative review. arXiv preprint arXiv:2208.12460

  25. Fraz M, Khurram SA, Graham S, Shaban M, Hassan M, Loya A, Rajpoot NM (2020) Fabnet: feature attention-based network for simultaneous segmentation of microvessels and nerves in routine histology images of oral cancer. Neural Comput Appl 32(14):9915–9928

    Article  Google Scholar 

  26. Shaban M, Khurram SA, Fraz MM, Alsubaie N, Masood I, Mushtaq S, Hassan M, Loya A, Rajpoot NM (2019) A novel digital score for abundance of tumour infiltrating lymphocytes predicts disease free survival in oral squamous cell carcinoma. Sci Rep 9(1):1–13

    Article  Google Scholar 

  27. Rasool A, Fraz MM, Javed S( 2021) Multiscale unified network for simultaneous segmentation of nerves and micro-vessels in histology images. In: 2021 International conference on digital futures and transformative technologies (ICoDT2), pp. 1– 6. IEEE

  28. Bashir RS, Mahmood H, Shaban M, Raza SEA, Fraz MM, Khurram SA, Rajpoot NM ( 2020) Automated grade classification of oral epithelial dysplasia using morphometric analysis of histology images. In: Medical imaging 2020: digital pathology, vol 11320, pp 245–250. SPIE

  29. Javed S, Mahmood A, Fraz MM, Koohbanani NA, Benes K, Tsang Y-W, Hewitt K, Epstein D, Snead D, Rajpoot N (2020) Cellular community detection for tissue phenotyping in colorectal cancer histology images. Med Image Anal 63:101696

    Article  Google Scholar 

  30. Dogar GM, Fraz MM, Javed S( 2021) Feature attention network for simultaneous nuclei instance segmentation and classification in histology images. In: 2021 International conference on digital futures and transformative technologies (ICoDT2), pp 1–6. IEEE

  31. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28

  32. Lin T-Y, Goyal P, Girshick R, He K, Dollár P ( 2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  33. Long J, Shelhamer E, Darrell T ( 2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  34. Ronneberger O, Fischer P, Brox T ( 2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234– 241. Springer

  35. Raza SEA, Cheung L, Shaban M, Graham S, Epstein D, Pelengaris S, Khan M, Rajpoot NM (2019) Micro-net: A unified model for segmentation of various objects in microscopy images. Med Image Anal 52:160–173

    Article  Google Scholar 

  36. Graham S, Rajpoot NM ( 2018) Sams-net: Stain-aware multi-scale network for instance-based nuclei segmentation in histology images. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 590– 594. IEEE

  37. Chen H, Qi X, Yu L, Heng P-A ( 2016) Dcan: deep contour-aware networks for accurate gland segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2487–2496

  38. Cui Y, Zhang G, Liu Z, Xiong Z, Hu J (2019) A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images. Med Biol Eng Comput 57(9):2027–2043

    Article  Google Scholar 

  39. Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A (2017) A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imag 36(7):1550–1560

    Article  Google Scholar 

  40. Khoshdeli M, Parvin B (2018) Deep learning models delineates multiple nuclear phenotypes in h & e stained histology sections. arXiv preprint arXiv:1802.04427

  41. Zhou Y, Onder OF, Dou Q, Tsougenis E, Chen H, Heng P-A ( 2019) Cia-net: Robust nuclei instance segmentation with contour-aware information aggregation. In: International conference on information processing in medical imaging, pp 682– 693. Springer

  42. Vu QD, Graham S, Kurc T, To MNN, Shaban M, Qaiser T, Koohbanani NA, Khurram SA, Kalpathy-Cramer J, Zhao T et al (2019) Methods for segmentation and classification of digital microscopy tissue images. Front Bioeng Biotechnol, 53

  43. Graham S, Vu QD, Raza SEA, Azam A, Tsang YW, Kwak JT, Rajpoot N (2019) Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal 58:101563

    Article  Google Scholar 

  44. Gamper J, Koohbanani NA, Benes K, Graham S, Jahanifar M, Khurram SA, Azam A, Hewitt K, Rajpoot N (2020) Pannuke dataset extension, insights and baselines. arXiv preprint arXiv:2003.10778

  45. Graham S, Jahanifar M, Azam A, Nimir M, Tsang Y-W, Dodd K, Hero E, Sahota H, Tank A, Benes K, et al (2021) Lizard: A large-scale dataset for colonic nuclear instance segmentation and classification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 684– 693

  46. Alemi Koohbanani N, Jahanifar M, Gooya A, Rajpoot N ( 2019) Nuclear instance segmentation using a proposal-free spatially aware deep learning framework. In: International conference on medical image computing and computer-assisted intervention, pp 622–630. Springer

  47. Chen S, Ding C, Tao D (2020) Boundary-assisted region proposal networks for nucleus segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 279– 288. Springer

  48. Zhao B, Chen X, Li Z, Yu Z, Yao S, Yan L, Wang Y, Liu Z, Liang C, Han C (2020) Triple u-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation. Med Image Anal 65:101786

    Article  Google Scholar 

  49. Schmidt U, Weigert M, Broaddus C, Myers G ( 2018) Cell detection with star-convex polygons. In: International conference on medical image computing and computer-assisted intervention, pp 265– 273 . Springer

  50. Chen, S., Ding, C., Liu, M., Tao, D.: Cpp-net: Context-aware polygon proposal network for nucleus segmentation. arXiv preprint arXiv:2102.06867 (2021)

  51. Tan M, Le Q( 2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning, pp 6105–6114. PMLR

  52. Roy AG, Navab N, Wachinger C (2018) Recalibrating fully convolutional networks with spatial and channel squeeze and excitation blocks. IEEE Trans Med Imag 38(2):540–549

    Article  Google Scholar 

  53. Vincent L, Soille P (1991) Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 13(06):583–598

    Article  Google Scholar 

  54. Kirillov A, He K, Girshick R, Rother C, Dollár P ( 2019) Panoptic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9404– 9413

  55. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32

  56. Bradski G, Kaehler A (2000) Opencv. Dr. Dobb’s journal of software tools 3:2

  57. Van der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, Gouillart E, Yu T (2014) scikit-image: image processing in python. Peer J 2:453

    Article  Google Scholar 

  58. Buslaev A, Iglovikov VI, Khvedchenya E, Parinov A, Druzhinin M, Kalinin AA (2020) Albumentations: fast and flexible image augmentations. Information 11(2):125

    Article  Google Scholar 

  59. Yakubovskiy, P(2020) Segmentation Models Pytorch. GitHub https://github.com/qubvel

  60. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L( 2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255 . IEEE

  61. Bock S, Goppold J, Weiß M(2018) An improvement of the convergence proof of the adam-optimizer. arXiv preprint arXiv:1804.10587

  62. Carpenter AE, Jones TR, Lamprecht MR, Clarke C, Kang IH, Friman O, Guertin DA, Chang JH, Lindquist RA, Moffat J et al (2006) Cellprofiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol 7(10):1–11

    Article  Google Scholar 

  63. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  64. Chen RJ, Lu MY, Wang J, Williamson DF, Rodig SJ, Lindeman NI, Mahmood F (2020) Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis. IEEE Trans Medi Imag

Download references

Author information

Authors and Affiliations

  1. National University of Sciences and Technology (NUST), Islamabad, 44000, Pakistan

    Syed Nauyan Rashid & Muhammad Moazam Fraz

Authors
  1. Syed Nauyan Rashid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Moazam Fraz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Moazam Fraz.

Ethics declarations

Conflit of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rashid, S.N., Fraz, M.M. Nuclei probability and centroid map network for nuclei instance segmentation in histology images. Neural Comput & Applic 35, 15447–15460 (2023). https://doi.org/10.1007/s00521-023-08503-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08503-2

Keywords

Search

Navigation