Abstract
Segmenting 3D cell nuclei from microscopy image volumes is critical for biological and clinical analysis, enabling the study of cellular expression patterns and cell lineages. However, current datasets for neuronal nuclei usually contain volumes smaller than 10\(^{-3}\) mm\(^3\) with fewer than 500 instances per volume, unable to reveal the complexity in large brain regions and restrict the investigation of neuronal structures. In this paper, we have pushed the task forward to the sub-cubic millimeter scale and curated the NucMM dataset with two fully annotated volumes: one 0.1 mm\(^3\) electron microscopy (EM) volume containing nearly the entire zebrafish brain with around 170,000 nuclei; and one 0.25 mm\(^3\) micro-CT (uCT) volume containing part of a mouse visual cortex with about 7,000 nuclei. With two imaging modalities and significantly increased volume size and instance numbers, we discover a great diversity of neuronal nuclei in appearance and density, introducing new challenges to the field. We also perform a statistical analysis to illustrate those challenges quantitatively. To tackle the challenges, we propose a novel hybrid-representation learning model that combines the merits of foreground mask, contour map, and signed distance transform to produce high-quality 3D masks. The benchmark comparisons on the NucMM dataset show that our proposed method significantly outperforms state-of-the-art nuclei segmentation approaches. Code and data are available at https://connectomics-bazaar.github.io/proj/nucMM/index.html.
Z. Lin and D. Wei—Equally contributed.
Krishna Swaroop K and N. Wendt—Works were done during internship at Harvard University.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alwes, F., Enjolras, C., Averof, M.: Live imaging reveals the progenitors and cell dynamics of limb regeneration. Elife 5, e19766 (2016)
Berger, D.R., Seung, H.S., Lichtman, J.W.: Vast (volume annotation and segmentation tool): efficient manual and semi-automatic labeling of large 3D image stacks. Front. Neural Circ. 12, 88 (2018)
Bottou, L.: Stochastic gradient learning in neural networks. In: Proceedings of Neuro-Nımes (1991)
Caicedo, J.C., et al.: Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat. Methods 16, 1247–1253 (2019)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: NeurIPS (2012)
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)
Cousty, J., Bertrand, G., Najman, L., Couprie, M.: Watershed cuts: minimum spanning forests and the drop of water principle. TPAMI 31, 1362–1374 (2008)
Dyer, E.L., et al.: Quantifying mesoscale neuroanatomy using x-ray microtomography. Eneuro (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of tricks for image classification with convolutional neural networks. In: CVPR (2019)
Heinrich, L., Funke, J., Pape, C., Nunez-Iglesias, J., Saalfeld, S.: Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete drosophila brain. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 317–325. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_36
Januszewski, M., et al.: High-precision automated reconstruction of neurons with flood-filling networks. Nat. Methods 15, 605–610 (2018)
Kasthuri, N., et al.: Saturated reconstruction of a volume of neocortex. Cell 162, 648–661 (2015)
Krasowski, N., Beier, T., Knott, G., Köthe, U., Hamprecht, F.A., Kreshuk, A.: Neuron segmentation with high-level biological priors. TMI 37, 829–839 (2017)
Lee, K., Zung, J., Li, P., Jain, V., Seung, H.S.: Superhuman accuracy on the SNEMI3D connectomics challenge. arXiv:1706.00120 (2017)
Lin, T.Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Lou, X., Kang, M., Xenopoulos, P., Munoz-Descalzo, S., Hadjantonakis, A.K.: A rapid and efficient 2d/3d nuclear segmentation method for analysis of early mouse embryo and stem cell image data. Stem Cell Rep. 2, 382–397 (2014)
Meijering, E.: Cell segmentation: 50 years down the road. Signal Process. Mag. 29, 140–145 (2012)
Nhu, H.T.T., Drigo, R.A.E., Berggren, P.O., Boudier, T.: A novel toolbox to investigate tissue spatial organization applied to the study of the islets of langerhans. Sci. Rep. 7, 1–12 (2017)
Petkova, M.: Correlative Light and Electron Microscopy in an Intact Larval Zebrafish. Ph.D. thesis (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Ruszczycki, B., et al.: Three-dimensional segmentation and reconstruction of neuronal nuclei in confocal microscopic images. Front. Neuroanatomy 13, 81 (2019)
Shapson-Coe, A., et al.: A connectomic study of a petascale fragment of human cerebral cortex. bioRxiv (2021)
Stegmaier, J., et al.: Real-time three-dimensional cell segmentation in large-scale microscopy data of developing embryos. Dev. Cell 36, 225–240 (2016)
Stringer, C., Wang, T., Michaelos, M., Pachitariu, M.: Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106 (2021)
Tokuoka, Y., et al.: 3D convolutional neural networks-based segmentation to acquire quantitative criteria of the nucleus during mouse embryogenesis. NPJ Syst. Biol. Appl. 6, 1–12 (2020)
Toyoshima, Y., et al.: Accurate automatic detection of densely distributed cell nuclei in 3D space. PLoS Comput. Biol. 12, e1004970 (2016)
Turaga, S.C., Briggman, K.L., Helmstaedter, M., Denk, W., Seung, H.S.: Maximin affinity learning of image segmentation. In: NeurIPS (2009)
Ulman, V., et al.: An objective comparison of cell-tracking algorithms. Nat. Methods 14, 1141–1152 (2017)
van der Walt, S., et al.: The scikit-image contributors: scikit-image: image processing in Python. PeerJ (2014)
Wei, D., et al.: MitoEM dataset: large-scale 3D mitochondria instance segmentation from EM images. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 66–76. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_7
Weigert, M., Schmidt, U., Haase, R., Sugawara, K., Myers, G.: Star-convex polyhedra for 3d object detection and segmentation in microscopy. In: WACV (2020)
Zhou, P., Feng, J., Ma, C., Xiong, C., HOI, S., et al.: Towards theoretically understanding why sgd generalizes better than adam in deep learning. arXiv preprint arXiv:2010.05627 (2020)
Zlateski, A., Seung, H.S.: Image segmentation by size-dependent single linkage clustering of a watershed basin graph. arXiv:1505.00249 (2015)
Acknowledgments
This work has been partially supported by NSF award IIS-1835231 and NIH award U19NS104653. We thank Daniel Franco-Barranco for setting up the challenge using NucMM. M.D.P. would like to acknowledge the support of Howard Hughes Medical Institute International Predoctoral Student Research Fellowship. I.A-C would like to acknowledge the support of the Beca Leonardo a Investigadores y Creadores Culturales 2020 de la Fundación BBVA.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, Z. et al. (2021). NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-87193-2_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87192-5
Online ISBN: 978-3-030-87193-2
eBook Packages: Computer ScienceComputer Science (R0)
