Skip to main content
SpringerLink
Log in

Super-resolution Segmentation Network for Reconstruction of Packed Neurites

  • Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript

Abstract

Neuron reconstruction can provide the quantitative data required for measuring the neuronal morphology and is crucial in brain research. However, the difficulty in reconstructing dense neurites, wherein massive labor is required for accurate reconstruction in most cases, has not been well resolved. In this work, we provide a new pathway for solving this challenge by proposing the super-resolution segmentation network (SRSNet), which builds the mapping of the neurites in the original neuronal images and their segmentation in a higher-resolution (HR) space. During the segmentation process, the distances between the boundaries of the packed neurites are enlarged, and only the central parts of the neurites are segmented. Owing to this strategy, the super-resolution segmented images are produced for subsequent reconstruction. We carried out experiments on neuronal images with a voxel size of 0.2 μm × 0.2 μm × 1 μm produced by fMOST. SRSNet achieves an average F1 score of 0.88 for automatic packed neurites reconstruction, which takes both the precision and recall values into account, while the average F1 scores of other state-of-the-art automatic tracing methods are less than 0.70.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Barbastathis, G., Ozcan, A., & Situ, G. (2019). On the use of deep learning for computational imaging. Optica, 6(8), 921–943. https://doi.org/10.1364/OPTICA.6.000921

    Article  Google Scholar 

  • Bohland, J. W., Wu, C., Barbas, H., Bokil, H., Bota, M., Breiter, H. C., et al. (2009). A proposal for a coordinated effort for the determination of brainwide neuroanatomical connectivity in model organisms at a mesoscopic scale. PLoS Computational Biology, 5(3), e1000334.

    Article  Google Scholar 

  • Cai, R., Pan, C., Ghasemigharagoz, A., Todorov, M. I., Forstera, B., Zhao, S., et al. (2019). Panoptic imaging of transparent mice reveals whole-body neuronal projections and skull-meninges connections. Nature Neuroscience, 22(2), 317–327. https://doi.org/10.1038/s41593-018-0301-3

    Article  CAS  PubMed  Google Scholar 

  • Chen, H., Dou, Q., Yu, L., Qin, J., & Heng, P.-A. (2018). VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage, 170, 446–455.

    Article  Google Scholar 

  • Cuntz, H., Forstner, F., Borst, A., & Häusser, M. (2010). One rule to grow them all: A general theory of neuronal branching and its practical application. PLoS Computational Biology, 6(8), e1000877.

    Article  Google Scholar 

  • De, J., Cheng, L., Zhang, X., Lin, F., Li, H., Ong, K. H., et al. (2016). A Graph-Theoretical Approach for Tracing Filamentary Structures in Neuronal and Retinal Images. IEEE Transactions on Medical Imaging, 35(1), 257–272. https://doi.org/10.1109/TMI.2015.2465962

    Article  PubMed  Google Scholar 

  • DeFelipe, J. (2010). From the connectome to the synaptome: An epic love story. Science, 330(6008), 1198–1201.

    Article  CAS  Google Scholar 

  • Donohue, D. E., & Ascoli, G. A. (2011). Automated reconstruction of neuronal morphology: An overview. Brain Research Reviews, 67(1–2), 94–102. https://doi.org/10.1016/j.brainresrev.2010.11.003

    Article  PubMed  Google Scholar 

  • Economo, M. N., Clack, N. G., Lavis, L. D., Gerfen, C. R., Svoboda, K., Myers, E. W., et al. (2016). A platform for brain-wide imaging and reconstruction of individual neurons. eLife, 5, e10566. https://doi.org/10.7554/eLife.10566

    Article  PubMed  PubMed Central  Google Scholar 

  • Feng, L., Zhao, T., & Kim, J. (2015). neuTube 1.0: A New Design for Efficient Neuron Reconstruction Software Based on the SWC Format. eneuro, 2(1), ENEURO.0049–0014.2014. https://doi.org/10.1523/eneuro.0049-14.2014

  • Friedmann, D., Pun, A., Adams, E. L., Lui, J. H., Kebschull, J. M., Grutzner, S. M., et al. (2020). Mapping mesoscale axonal projections in the mouse brain using a 3D convolutional network. Proceedings of the National Academy of Sciences, 117(20), 11068–11075.

    Article  CAS  Google Scholar 

  • Gillette, T. A., Brown, K. M., & Ascoli, G. A. (2011). The DIADEM Metric: Comparing Multiple Reconstructions of the Same Neuron. Neuroinformatics, 9(2–3), 233–245.

    Article  Google Scholar 

  • Gong, H., Xu, D., Yuan, J., Li, X., Guo, C., Peng, J., et al. (2016). High-throughput dual-colour precision imaging for brain-wide connectome with cytoarchitectonic landmarks at the cellular level. Nature Communications, 7, 12142.

    Article  CAS  Google Scholar 

  • Gong, H., Zeng, S., Yan, C., Lv, X., Yang, Z., Xu, T., et al. (2013). Continuously tracing brain-wide long-distance axonal projections in mice at a one-micron voxel resolution. NeuroImage, 74(7), 87–98.

    Article  Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In IEEE International Conference on Computer Vision.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

  • Helmstaedter, M. (2013). Cellular-resolution connectomics: Challenges of dense neural circuit reconstruction. Nature Methods, 10(6), 501–507. https://doi.org/10.1038/nmeth.2476

    Article  CAS  PubMed  Google Scholar 

  • Huang, Q., Chen, Y., Liu, S., Xu, C., Cao, T., Xu, Y., et al. (2020). Weakly supervised learning of 3D deep network for neuron reconstruction. Frontiers in Neuroanatomy. https://doi.org/10.3389/fnana.2020.00038

    Article  PubMed  PubMed Central  Google Scholar 

  • Jiang, Y., Chen, W., Liu, M., Wang, Y., & Meijering, E. (2020). 3D Neuron Microscopy Image Segmentation via the Ray-Shooting Model and a DC-BLSTM Network. IEEE Transactions on Medical Imaging, 40, 26–37.

    Article  Google Scholar 

  • Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint. http://arxiv.org/abs/1412.6980

  • Li, A., Gong, H., Zhang, B., Wang, Q., Yan, C., Wu, J., et al. (2010). Micro-optical sectioning tomography to obtain a high-resolution atlas of the mouse brain. Science, 330(6009), 1404–1408. https://doi.org/10.1126/science.1191776

    Article  CAS  PubMed  Google Scholar 

  • Li, Q., & Shen, L. (2020). 3D Neuron Reconstruction in Tangled Neuronal Image With Deep Networks. IEEE Transactions on Medical Imaging, 39(2), 425–435. https://doi.org/10.1109/TMI.2019.2926568

    Article  PubMed  Google Scholar 

  • Li, R., Zeng, T., Peng, H., & Ji, S. (2017). Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction. IEEE Transactions on Medical Imaging, 36(7), 1533–1541.

    Article  Google Scholar 

  • Li, R., Zhu, M., Li, J., Bienkowski, M. S., Foster, N. N., Xu, H., et al. (2019a). Precise segmentation of densely interweaving neuron clusters using G-Cut. Nature Communications, 10(1), 1–12.

    Google Scholar 

  • Li, S., Quan, T., Xu, C., Huang, Q., Kang, H., Chen, Y., et al. (2019b). Optimization of Traced Neuron Skeleton Using Lasso-Based Model. Frontiers in Neuroanatomy, 13, 18–18. https://doi.org/10.3389/fnana.2019.00018

    Article  PubMed  PubMed Central  Google Scholar 

  • Li, S., Quan, T., Zhou, H., Huang, Q., Guan, T., Chen, Y., et al. (2020). Brain-wide shape reconstruction of a traced neuron using the convex image segmentation method. Neuroinformatics, 18(2), 199–218.

    Article  Google Scholar 

  • Lichtman, J. W., & Denk, W. (2011). The big and the small: Challenges of imaging the brain’s circuits. Science, 334(6056), 618–623. https://doi.org/10.1126/science.1209168

    Article  CAS  PubMed  Google Scholar 

  • Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2017 (pp. 2980–2988).

  • Liu, M., Luo, H., Tan, Y., Wang, X., & Chen, W. Improved V-net based image segmentation for 3D neuron reconstruction. In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 443–448).

  • Meijering, E. (2010). Neuron tracing in perspective. Cytometry. Part A, 77(7), 693–704. https://doi.org/10.1002/cyto.a.20895

    Article  Google Scholar 

  • Meijering, E. (2020). A bird’s-eye view of deep learning in bioimage analysis. Computational and Structural Biotechnology Journal, 18, 2312.

    Article  CAS  Google Scholar 

  • Milletari, F., Navab, N., & Ahmadi, S. A. (2016). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In Fourth International Conference on 3d Vision.

  • Nehme, E., Weiss, L. E., Michaeli, T., & Shechtman, Y. (2018). Deep-STORM: Super-resolution single-molecule microscopy by deep learning. Optica, 5(4), 458–464. https://doi.org/10.1364/Optica.5.000458

    Article  CAS  Google Scholar 

  • Osten, P., & Margrie, T. W. (2013). Mapping brain circuitry with a light microscope. Nature Methods, 10(6), 515–523. https://doi.org/10.1038/nmeth.2477

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Ouyang, W., Aristov, A., Lelek, M., Hao, X., & Zimmer, C. (2018). Deep learning massively accelerates super-resolution localization microscopy. Nature Biotechnology, 36(5), 460–468. https://doi.org/10.1038/nbt.4106

    Article  CAS  PubMed  Google Scholar 

  • Pan, C., Cai, R., Quacquarelli, F. P., Ghasemigharagoz, A., Lourbopoulos, A., Matryba, P., et al. (2016). Shrinkage-mediated imaging of entire organs and organisms using uDISCO. Nature Methods, 13(10), 859–867.

    Article  CAS  Google Scholar 

  • Peng, H., Long, F., & Myers, G. (2011). Automatic 3D neuron tracing using all-path pruning. Bioinformatics, 27(13), i239–i247.

    Article  CAS  Google Scholar 

  • Peng, H., Hawrylycz, M., Roskams, J., Hill, S., Spruston, N., Meijering, E., et al. (2015). BigNeuron: Large-Scale 3D Neuron Reconstruction from Optical Microscopy Images. Neuron, 87(2), 252–256. https://doi.org/10.1016/j.neuron.2015.06.036

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Quan, T., Zhou, H., Li, J., Li, S., Li, A., Li, Y., et al. (2016). NeuroGPS-Tree: Automatic reconstruction of large-scale neuronal populations with dense neurites. Nature Methods, 13(1), 51.

    Article  CAS  Google Scholar 

  • Radojević, M., & Meijering, E. (2017). Automated neuron tracing using probability hypothesis density filtering. Bioinformatics, 33(7), 1073–1080.

    PubMed  Google Scholar 

  • Salimans, T., & Kingma, D. P. (2016). Weight normalization: A simple reparameterization to accelerate training of deep neural networks. Advances in Neural Information Processing Systems, 29, 901–909.

    Google Scholar 

  • Skibbe, H., Reisert, M., Nakae, K., Watakabe, A., Hata, J., Mizukami, H., et al. (2018). Pat–probabilistic axon tracking for densely labeled neurons in large 3-d micrographs. IEEE Transactions on Medical Imaging, 38(1), 69–78.

    Article  Google Scholar 

  • Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society, 58(1), 267–288.

    Google Scholar 

  • Todorov, M. I., Paetzold, J. C., Schoppe, O., Tetteh, G., Shit, S., Efremov, V., et al. (2020). Machine learning analysis of whole mouse brain vasculature. Nature Methods, 17(4), 442–449. https://doi.org/10.1038/s41592-020-0792-1

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Türetken, E., González, G., Blum, C., & Fua, P. (2011). Automated reconstruction of dendritic and axonal trees by global optimization with geometric priors. Neuroinformatics, 9(2–3), 279–302.

    Article  Google Scholar 

  • Wang, H. D., Rivenson, Y., Jin, Y. Y., Wei, Z. S., Gao, R., Gunaydin, H., et al. (2019a). Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nature Methods, 16(1), 103–110. https://doi.org/10.1038/s41592-018-0239-0

    Article  CAS  PubMed  Google Scholar 

  • Wang, Y., Li, Q., Liu, L., Zhou, Z., Ruan, Z., Kong, L., et al. (2019b). TeraVR empowers precise reconstruction of complete 3-D neuronal morphology in the whole brain. Nature Communications, 10(1), 1–9.

    Google Scholar 

  • Wang, Y., Narayanaswamy, A., Tsai, C. L., & Roysam, B. (2011). A broadly applicable 3-D neuron tracing method based on open-curve snake. Neuroinformatics, 9(2–3), 193–217. https://doi.org/10.1007/s12021-011-9110-5

    Article  PubMed  Google Scholar 

  • Xiong, H., Zhou, Z., Zhu, M., Lv, X., Li, A., Li, S., et al. (2014). Chemical reactivation of quenched fluorescent protein molecules enables resin-embedded fluorescence microimaging. Nature Communications, 5(1), 3992. https://doi.org/10.1038/ncomms4992

    Article  CAS  PubMed  Google Scholar 

  • Yu, J., Fan, Y., Yang, J., Xu, N., Wang, Z., Wang, X., et al. (2018). Wide activation for efficient and accurate image super-resolution. arXiv preprint. http://arxiv.org/abs/1808.08718

  • Zhang, H., Fang, C., Xie, X., Yang, Y., Mei, W., Jin, D., et al. (2019). High-throughput, high-resolution deep learning microscopy based on registration-free generative adversarial network. Biomedical Optics Express, 10(3), 1044–1063. https://doi.org/10.1364/BOE.10.001044

    Article  PubMed  PubMed Central  Google Scholar 

  • Zhao, J., Chen, X., Xiong, Z., Liu, D., Zeng, J., Xie, C., et al. (2020). Neuronal Population Reconstruction From Ultra-Scale Optical Microscopy Images via Progressive Learning. IEEE Transactions on Medical Imaging, 39(12), 4034–4046.

    Article  Google Scholar 

  • Zhao, T., Xie, J., Amat, F., Clack, N., Ahammad, P., Peng, H., et al. (2011). Automated reconstruction of neuronal morphology based on local geometrical and global structural models. Neuroinformatics, 9(2–3), 247–261. https://doi.org/10.1007/s12021-011-9120-3

    Article  PubMed  PubMed Central  Google Scholar 

  • Zhou, H., Cai, R., Quan, T., Liu, S., Li, S., Huang, Q., et al. (2020). 3D high resolution generative deep-learning network for fluorescence microscopy imaging. Optics Letters, 45(7), 1695–1698.

    Article  Google Scholar 

  • Zhou, H., Li, S., Li, A., Huang, Q., Xiong, F., Li, N., et al. (2021). GTree: An open-source tool for dense reconstruction of brain-wide neuronal population. Neuroinformatics, 19(2), 305–317.

    Article  Google Scholar 

  • Zhou, Z., Kuo, H.-C., Peng, H., & Long, F. (2018). DeepNeuron: An open deep learning toolbox for neuron tracing. Brain Informatics, 5(2), 1–9.

    Article  Google Scholar 

  • Zhu, W., Huang, Y., Zeng, L., Chen, X., Liu, Y., Qian, Z., et al. (2019). AnatomyNet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy. Medical Physics, 46(2), 576–589.

    Article  Google Scholar 

Download references

Information Sharing Statement

The super-resolution neural network codes are freely available. Neural network codes and some test datasets and can be download in https://github.com/artzers/SuperSegmentation-Neuron. If one is interested in other datasets, please feel free to contact us.

Funding

This work is supported by National Natural Science Foundation of China (81327802), China Postdoctoral Science Foundation (2018M642826 and 2019T120661), Project (376390) Supported by the Scientific Research Foundation of CUIT and the Director Fund of WNLO.

Author information

Authors and Affiliations

  1. School of Computer Science, Chengdu University of Information Technology, Chengdu, Sichuan, China

    Hang Zhou

  2. Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, 430074, China

    Tingting Cao, Tian Liu, Shijie Liu, Lu Chen, Yijun Chen, Qing Huang, Shaoqun Zeng & Tingwei Quan

  3. MoE Key Laboratory for Biomedical Photonics, Collaborative Innovation Center for Biomedical Engineering, School of Engineering Sciences, Huazhong University of Science and Technology, Wuhan, Hubei, 430074, China

    Tingting Cao, Tian Liu, Shijie Liu, Lu Chen, Yijun Chen, Qing Huang, Shaoqun Zeng & Tingwei Quan

  4. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, Hubei, China

    Wei Ye

Authors
  1. Hang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tingting Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shijie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yijun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qing Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wei Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Shaoqun Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Tingwei Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZS and QT conceived the project. ZS, QT designed the model and wrote the manuscript. QT and ZH designed the algorithms. CT, LT, LS, CL, CY, HQ, YW performed image analysis.

Corresponding author

Correspondence to Tingwei Quan.

Ethics declarations

Ethics Approval

Animal experimentation: All experiments were performed in accordance with the guidelines of the Experimental Animal Ethics Committee at Huazhong University of Science and Technology.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, H., Cao, T., Liu, T. et al. Super-resolution Segmentation Network for Reconstruction of Packed Neurites. Neuroinform 20, 1155–1167 (2022). https://doi.org/10.1007/s12021-022-09594-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-022-09594-3

Keywords

Search

Navigation