Abstract
Accurate segmentation and analysis of membranes from immunohistochemical (IHC) images are crucial for cancer diagnosis and prognosis. Although several fully-supervised deep learning methods for membrane segmentation from IHC images have been proposed recently, the high demand for pixel-level annotations makes this process time-consuming and labor-intensive. To overcome this issue, we propose a novel deep framework for membrane segmentation that utilizes nuclei point-level supervision. Our framework consists of two networks: a Seg-Net that generates segmentation results for membranes and nuclei, and a Tran-Net that transforms the segmentation into semantic points. In this way, the accuracy of the semantic points is closely related to the segmentation quality. Thus, the inconsistency between the semantic points and the point annotations can be used as effective supervision for cell segmentation. We evaluated the proposed method on two IHC membrane-stained datasets and achieved an 81.36% IoU and 85.51% \(F_1\) score of the fully supervised method. All source codes are available here.
L. Cui, J. Feng, W. Yang and L. Yang–Equally contribution.
H. Li and Z. Xu—Equally first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aurenhammer, F., Klein, R.: Voronoi diagrams. Handb. Comput. Geom. 5(10), 201–290 (2000)
Di Cataldo, S., Ficarra, E., Macii, E.: Selection of tumor areas and segmentation of nuclear membranes in tissue confocal images: a fully automated approach. In: 2007 IEEE International Conference on Bioinformatics and Biomedicine (BIBM 2007), pp. 390–398. IEEE (2007)
Elmoataz, A., Schüpp, S., Clouard, R., Herlin, P., Bloyet, D.: Using active contours and mathematical morphology tools for quantification of immunohistochemical images. Signal Process. 71(2), 215–226 (1998)
Ficarra, E., Di Cataldo, S., Acquaviva, A., Macii, E.: Automated segmentation of cells with ihc membrane staining. IEEE Trans. Biomed. Eng. 58(5), 1421–1429 (2011)
Graham, S., et al.: Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)
Han, L., Yin, Z.: Unsupervised network learning for cell segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 282–292. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_27
Hou, L., Agarwal, A., Samaras, D., Kurc, T.M., Gupta, R.R., Saltz, J.H.: Robust histopathology image analysis: to label or to synthesize? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8533–8542 (2019)
Ji, Y., et al.: Multi-compound transformer for accurate biomedical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 326–336. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_31
Khameneh, F.D., Razavi, S., Kamasak, M.: Automated segmentation of cell membranes to evaluate her2 status in whole slide images using a modified deep learning network. Comput. Biol. Med. 110, 164–174 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lin, A., Chen, B., Xu, J., Zhang, Z., Lu, G., Zhang, D.: Ds-transunet: dual swin transformer u-net for medical image segmentation. IEEE Trans. Instrument. Meas. 71, 1–15 (2022)
Luna, M., Kwon, M., Park, S.H.: Precise separation of adjacent nuclei using a siamese neural network. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 577–585. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_64
Bueno-de Mesquita, J.M., Nuyten, D., Wesseling, J., van Tinteren, H., Linn, S., van De Vijver, M.: The impact of inter-observer variation in pathological assessment of node-negative breast cancer on clinical risk assessment and patient selection for adjuvant systemic treatment. Ann. Oncol. 21(1), 40–47 (2010)
Mi, H., et al.: A quantitative analysis platform for pd-l1 immunohistochemistry based on point-level supervision model. In: IJCAI, pp. 6554–6556 (2019)
Qaiser, T., Rajpoot, N.M.: Learning where to see: a novel attention model for automated immunohistochemical scoring. IEEE Trans. Med. Imaging 38(11), 2620–2631 (2019)
Qu, H., et al.: Weakly supervised deep nuclei segmentation using points annotation in histopathology images. In: International Conference on Medical Imaging with Deep Learning, pp. 390–400. PMLR (2019)
Roerdink, J.B., Meijster, A.: The watershed transform: definitions, algorithms and parallelization strategies. Fundamenta informaticae 41(1–2), 187–228 (2000)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Saha, M., Chakraborty, C.: Her2net: a deep framework for semantic segmentation and classification of cell membranes and nuclei in breast cancer evaluation. IEEE Trans. Image Process. 27(5), 2189–2200 (2018)
Swiderska-Chadaj, Z., et al.: Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, 101547 (2019)
Tian, K., et al.: Weakly-supervised nucleus segmentation based on point annotations: a coarse-to-fine self-stimulated learning strategy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 299–308. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_29
Vogel, C., et al.: P1–07-02: discordance between central and local laboratory her2 testing from a large her2- negative population in virgo, a metastatic breast cancer registry (2011)
Wolff, A.C., et al.: Human epidermal growth factor receptor 2 testing in breast cancer: American society of clinical oncology/college of american pathologists clinical practice guideline focused update. Arch. Pathol. Lab. Med. 142(11), 1364–1382 (2018)
Xu, J., et al.: Stacked sparse autoencoder (ssae) for nuclei detection on breast cancer histopathology images. IEEE Trans. Med. Imaging 35(1), 119–130 (2015)
Yang, L., Meer, P., Foran, D.J.: Unsupervised segmentation based on robust estimation and color active contour models. IEEE Trans. Inf. Technol. Biomed. 9(3), 475–486 (2005)
Yang, X., Li, H., Zhou, X.: Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy. IEEE Trans. Circ. Syst. I: Regul. Papers 53(11), 2405–2414 (2006)
Yoo, I., Yoo, D., Paeng, K.: PseudoEdgeNet: nuclei segmentation only with point annotations. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 731–739. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_81
Acknowledgements
This work is supported by the National Natural Science Foundation of China (NSFC Grant No. 62073260, No.62106198 and No.62276052), and the Natural Science Foundation of Shaanxi Province of China (2021JQ-461), the General Project of Education Department of Shaanxi Provincial Government under Grant 21JK0927. Medical writing support is provided by AstraZeneca China. The technical and equipment support is provided by HangZhou DiYingJia Technology Co., Ltd (DeepInformatics++). The authors would like to thank the medical team at AstraZeneca China and techinical team at DeepInformatics++ for their scientific comments on this study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, H. et al. (2023). Segment Membranes and Nuclei from Histopathological Images via Nuclei Point-Level Supervision. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14225. Springer, Cham. https://doi.org/10.1007/978-3-031-43987-2_52
Download citation
DOI: https://doi.org/10.1007/978-3-031-43987-2_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43986-5
Online ISBN: 978-3-031-43987-2
eBook Packages: Computer ScienceComputer Science (R0)
