Abstract
High-throughput 3D nuclei instance segmentation (NIS) is critical to understanding the complex structure and function of individual cells and their interactions within the larger tissue environment in the brain. Despite the significant progress in achieving accurate NIS within small image stacks using cutting-edge machine learning techniques, there has been a lack of effort to extend this approach towards whole-brain NIS from light-sheet microscopy. This critical area of research has been largely overlooked, despite its importance in the neuroscience field. To address this challenge, we propose an efficient deep stitching neural network built upon a knowledge graph model characterizing 3D contextual relationships between nuclei. Our deep stitching model is designed to be agnostic, enabling existing limited methods (optimized for image stack only) to overcome the challenges of whole-brain NIS, particularly in addressing the issue of inter- and intra-slice gaps. We have evaluated the NIS accuracy on top of three state-of-the-art deep models with \(128\times 128\times 64\) image stacks, and visualized results in both inter- and intra-slice gaps of whole brain. With resolved gap issues, our deep stitching model enables the whole-brain NIS (gigapixel-level) on entry-level GPU servers within 27 h.
Supported by NIH R01NS110791, NIH R01MH121433, NIH P50HD103573, and Foundation of Hope.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alahmari, S.S., Goldgof, D., Hall, L.O., Mouton, P.R.: A review of nuclei detection and segmentation on microscopy images using deep learning with applications to unbiased stereology counting. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Banerjee, A., Poddar, R.: Enhanced visualization of tissue microstructures using swept-source optical coherence tomography and edible oil as optical clearing agent. Optik 267, 169693 (2022)
Bennett, H.C., Kim, Y.: Advances in studying whole mouse brain vasculature using high-resolution 3D light microscopy imaging. Neurophotonics 9(2), 021902 (2022)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016, Part II. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Funke, J., Andres, B., Hamprecht, F.A., Cardona, A., Cook, M.: Efficient automatic 3D-reconstruction of branching neurons from EM data. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1004–1011. IEEE (2012)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Iqbal, A., Sheikh, A., Karayannis, T.: DeNerD: high-throughput detection of neurons for brain-wide analysis with deep learning. Sci. Rep. 9(1), 13828 (2019)
Lin, Z., et al.: NucMM dataset: 3D neuronal nuclei instance segmentation at sub-cubic millimeter scale. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 164–174. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_16
Pachitariu, M., Stringer, C.: Cellpose 2.0: how to train your own model. Nat. Methods 1–8 (2022)
Pape, C., Beier, T., Li, P., Jain, V., Bock, D.D., Kreshuk, A.: Solving large multicut problems for connectomics via domain decomposition. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–10 (2017)
Stringer, C., Wang, T., Michaelos, M., Pachitariu, M.: Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18(1), 100–106 (2021)
Vu, Q.D., Rajpoot, K., Raza, S.E.A., Rajpoot, N.: Handcrafted histological transformer (H2T): unsupervised representation of whole slide images. Med. Image Anal. 85, 102743 (2023)
Wang, H., Ren, H., Leskovec, J.: Relational message passing for knowledge graph completion. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1697–1707 (2021)
Yang, B., et al.: DaXi-high-resolution, large imaging volume and multi-view single-objective light-sheet microscopy. Nat. Methods 19(4), 461–469 (2022)
Yang, H., et al.: Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study. BMC Med. 19, 1–14 (2021)
You, S., et al.: High cell density and high-resolution 3D bioprinting for fabricating vascularized tissues. Sci. Adv. 9(8), eade7923 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wei, Z., Dan, T., Ding, J., Dere, M., Wu, G. (2023). A General Stitching Solution for Whole-Brain 3D Nuclei Instance Segmentation from Microscopy Images. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14223. Springer, Cham. https://doi.org/10.1007/978-3-031-43901-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-43901-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43900-1
Online ISBN: 978-3-031-43901-8
eBook Packages: Computer ScienceComputer Science (R0)
