Abstract
Segmentation of whole slide images (WSIs) is an important step for computer-aided cancer diagnosis. However, due to the gigapixel dimension, WSIs are usually cropped into patches for analysis. Processing high-resolution patches independently may leave out the global geographical relationships and suffer slow inference speed while using low-resolution patches can enlarge receptive fields but lose local details. Here, we propose a Hierarchical Attention Guided (HAG) framework to address above problems. Particularly, our framework contains a global branch and several local branches to perform prediction at different scales. Additive hierarchical attention maps are generated by the global branch with sparse constraints to fuse multi-resolution predictions for better segmentation. During the inference, the sparse attention maps are used as the certainty guidance to select important local areas with a quadtree strategy for acceleration. Experimental results on two WSI datasets highlight two merits of our framework: 1) effectively aggregate multi-resolution information to achieve better results, 2) significantly reduce the computational cost to accelerate the prediction without decreasing accuracy.
Keywords
- Segmentation
- Whole slide image
- Deep Learning
- Acceleration
J. Yan, H. Chen and K. Wang—contributed equally
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bejnordi, B.E., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Chaurasia, A., Culurciello, E.: Linknet: exploiting encoder representations for efficient semantic segmentation. In: Proceedings of the IEEE Visual Communications and Image Processing, pp. 1–4 (2017)
Tosun, A.B., et al.: Histological detection of high-risk benign breast lesions from whole slide images. In: Descoteaux, M., et al. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 144–152. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_17
Chen, H., Qi, X., Yu, L., Dou, Q., Qin, J., Heng, P.A.: Dcan: deep contour-aware networks for object instance segmentation from histology images. Med. Image Anal. 36, 135–146 (2017)
Cong, W.M., et al.: Practice guidelines for the pathological diagnosis of primary liver cancer: 2015 update. World J. Gastroenterol. 22(42), 9279 (2016)
Dong, N., et al.: Reinforced auto-zoom net: towards accurate and fast breast cancer segmentation in whole-slide images. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 317–325. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_36
Finkel, R.A., Bentley, J.L.: Quad trees a data structure for retrieval on composite keys. Acta Informatica 4(1), 1–9 (1974)
Gu, F., Burlutskiy, N., Andersson, M., Wilén, L.K.: Multi-resolution networks for semantic segmentation in whole slide images. In: Stoyanov, D., et al. (eds.) OMIA/COMPAY -2018. LNCS, vol. 11039, pp. 11–18. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00949-6_2
Harrold, I.M., Bean, S.M., Williams, N.C.: Emerging from the basement: the visible pathologist. Arch. Pathol. Laboratory Med. 143(8), 917–918 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ioannou, N., et al.: Accelerated ML-assisted tumor detection in high-resolution histopathology images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 406–414. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_45
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Mehta, S., Mercan, E., Bartlett, J., Weaver, D., Elmore, J.G., Shapiro, L.: Y-Net: joint segmentation and classification for diagnosis of breast biopsy images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 893–901. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_99
Paszke, A., et al.: Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8026–8037 (2019)
Qu, H., Yan, Z., Riedlinger, G.M., De, S., Metaxas, D.N.: Improving nuclei/gland instance segmentation in histopathology images by full resolution neural network and spatial constrained loss. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 378–386. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_42
van Rijthoven, M., Balkenhol, M., Siliņa, K., van der Laak, J., Ciompi, F.: Hooknet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images. Med. Image Anal. 68, 101890 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schmitz, R., et al.: Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture. Med. Image Anal. 70, 101996 (2021)
Tokunaga, H., Teramoto, Y., Yoshizawa, A., Bise, R.: Adaptive weighting multi-field-of-view CNN for semantic segmentation in pathology. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12597–12606 (2019)
Wang, Y., et al.: Double-uncertainty weighted method for semi-supervised learning. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 542–551. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_53
Yan, J., Chen, S., Zhang, Y., Li, X.: Neural architecture search for compressed sensing magnetic resonance image reconstruction. Comput. Med. Imaging Graph. 85, 101784 (2020)
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., Huang, J.: Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65, 101789 (2020)
Acknowledgement
This research was partly supported by the National Natural Science Foundation of China (Grant No. 41876098), the National Key R&D Program of China (Grant No. 2020AAA0108303), and Shenzhen Science and Technology Project (Grant No. JCYJ20200109143041798).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yan, J. et al. (2021). Hierarchical Attention Guided Framework for Multi-resolution Collaborative Whole Slide Image Segmentation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12908. Springer, Cham. https://doi.org/10.1007/978-3-030-87237-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-87237-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87236-6
Online ISBN: 978-3-030-87237-3
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
