Abstract
The research on content-based histopathological image retrieval (CBHIR) has become popular in recent years. CBHIR systems provide auxiliary diagnosis information for pathologists by searching for and returning regions that are contently similar to the region of interest (ROI) from a pre-established database. To retrieve diagnostically relevant regions from a database that consists of histopathological whole slide images (WSIs) for query ROIs is challenging and yet significant for clinical applications. In this paper, we propose a novel CBHIR framework for regions retrieval from WSI-database based on hierarchical graph neural networks (GNNs). Compared to the present CBHIR framework, the structural information of WSI is preserved by the proposed model, which makes the retrieval framework more sensitive to regions that are similar in tissue distribution. Moreover, benefited from the hierarchical GNN structures, the proposed framework is scalable for both the size and shape variation of ROIs. It allows the pathologist defining the query region using free curves. Thirdly, the retrieval is achieved by binary codes and hashing methods, which makes it very efficient and thereby adequate for practical large-scale WSI-database. The proposed method is validated on a lung cancer dataset and compared to the state-of-the-art methods. The proposed method achieved precisions above 82.4% in the irregular region retrieval task, which are superior to the state-of-the-art methods. The average time of retrieval is 0.514 ms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The dataset is accessible at https://acdc-lunghp.grand-challenge.org/. Since the annotations of testing part of the data set are not yet published, only the 150 training WSIs of the data were used in this paper.
References
Li, Z., Zhang, X., Müller, H., et al.: Large-scale retrieval for medical image analytics: a comprehensive review. Med. Image Anal. 43, 66–84 (2018)
Bejnordi, B.E., Veta, M., Van Diest, P.J., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Shi, X., Sapkota, M., Xing, F., et al.: Pairwise based deep ranking hashing for histopathology image classification and retrieval. Pattern Recogn. 81, 14–22 (2018)
Ma, Y., Jiang, Z., Zhang, H., et al.: Generating region proposals for histopathological whole slide image retrieval. Comput. Methods Programs Biomed. 159, 1–10 (2018)
Zheng, Y., Jiang, Z., Zhang, H., et al.: Histopathological whole slide image analysis using context-based CBIR. IEEE Trans. Med. Imaging 37(7), 1641–1652 (2018)
Jimenez-del-Toro, O., Otálora, S., Atzori, M., Müller, H.: Deep multimodal case-based retrieval for large histopathology datasets. In: Wu, G., Munsell, B., Zhan, Y., Bai, W., Sanroma, G., Coupé, P. (eds.) Patch-MI 2017. LNCS, vol. 10530, pp. 149–157. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67434-6_17
Day, W.H.E., Edelsbrunner, H.: Efficient algorithms for agglomerative hierarchical clustering methods. J. Classif. 1(1), 7–24 (1984)
Wu, Z., Pan, S., Chen, F., et al.: A comprehensive survey on graph neural networks. arXiv:1901.00596 (2019)
Li, R., Yao, J., Zhu, X., Li, Y., Huang, J.: Graph CNN for survival analysis on whole slide pathological images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_20
Gilmer, J., Schoenholz, S.S., Riley, P.F., et al.: Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning (ICML), pp. 1263–1272 (2017)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS), pp. 1024–1034 (2017)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of International Conference on Learning Representations (ICLR) (2017)
Ying, Z., You, J., Morris, C., et al.: Hierarchical graph representation learning with differentiable pooling. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS), pp. 4805–4815 (2018)
Li, Z., Hu, Z., et al.: Computer-aided diagnosis of lung carcinoma using deep learning - a pilot study. arXiv:1803.05471v1 (2018)
Huang, G., Liu, Z., Van Der Maaten, L., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4700–4708 (2017)
Zheng, Y., Jiang, Z., Zhang, H., et al.: Size-scalable content-based histopathological image retrieval from database that consists of WSIs. IEEE J. Biomed. Health Inf. 22(4), 1278–1287 (2018)
Acknowledgment
This work was supported by the National Natural Science Foundation of China (No. 61771031, 61371134, 61471016, and 61501009), China Postdoctoral Science Foundation (No. 2019M650446) and Motic-BUAA Image Technology Research Center.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, Y., Jiang, B., Shi, J., Zhang, H., Xie, F. (2019). Encoding Histopathological WSIs Using GNN for Scalable Diagnostically Relevant Regions Retrieval. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_61
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)
