Skip to main content
SpringerLink
Log in

AI-Enhanced 3D Biomedical Data Analytics for Neuronal Structure Reconstruction

  • Chapter
  • First Online:
Humanity Driven AI

Abstract

Investigation of the 3D neuron morphology plays a critical part in understanding functions and activities in brain circuits, towards discovering how the brain develops in health and how brain function can be modified to improve human well-being. One of the major procedures in rebuilding connections and structures of neural circuits is to reconstruct the neuron morphology from 3D optical microscopy images. This has traditionally been done manually and is extremely time consuming. Many research studies have thus been conducted for automatic neuronal structure reconstruction with the aim of achieving high-quality reconstruction under poor image quality. In this chapter, we present several case studies of our most recent AI method development, which consists of novel 3D deep learning models for neural structure segmentation and intelligent back-tracking approaches for tracing. These methods are generally effective for a large variety of species and provide the state-of-the-art performance in single neuron reconstruction. Such development will enable large-scale data-driven investigations in neuroscience and enhance our fundamental understanding of the human brain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adalsteinsson, D., Sethian, J.A.: A fast level set method for propagating interfaces. J. Comput. Phys. 118(2), 269–277 (1995)

    Google Scholar 

  2. Ascoli, G.A., Donohue, D.E., Halavi, M.: Neuromorpho. org: a central resource for neuronal morphologies. J. Neurosci. 27(35), 9247–9251 (2007)

    Google Scholar 

  3. Basu, S., Racoceanu, D.: Reconstructing neuronal morphology from microscopy stacks using fast marching. In: Proc. ICIP., pp. 3597–3601. IEEE (2014)

    Google Scholar 

  4. Brown, K.M., Barrionuevo, G., Canty, A.J., De Paola, V., Hirsch, J.A., Jefferis, G.S., Lu, J., Snippe, M., Sugihara, I., Ascoli, G.A.: The DIADEM data sets: representative light microscopy images of neuronal morphology to advance automation of digital reconstructions. Neuroinformatics 9(2–3), 143–157 (2011)

    Google Scholar 

  5. Budd, S., Robinson, E.C., Kainz, B.: A survey on active learning and human-in-the-loop deep learning for medical image analysis. arXiv preprint arXiv:1910.02923 (2019)

  6. Chen, H., Xiao, H., Liu, T., Peng, H.: SmartTracing: self-learning-based neuron reconstruction. Brain informatics 2(3), 135–144 (2015)

    Google Scholar 

  7. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Proc. MICCAI., pp. 424–432. Springer (2016)

    Google Scholar 

  8. Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 (2015)

  9. Feng, L., Zhao, T., Kim, J.: neutube 1.0: a new design for efficient neuron reconstruction software based on the SWC format. eNeuro 2(1) (2015)

    Google Scholar 

  10. Hampel, S., Chung, P., McKellar, C.E., Hall, D., Looger, L.L., Simpson, J.H.: Drosophila brainbow: a recombinase-based fluorescence labeling technique to subdivide neural expression patterns. Nature methods 8(3), 253–259 (2011)

    Google Scholar 

  11. Hassouna, M.S., Farag, A.A.: Multistencils fast marching methods: A highly accurate solution to the eikonal equation on cartesian domains. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1563–1574 (2007)

    Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proc. CVPR., pp. 770–778 (2016)

    Google Scholar 

  13. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

  14. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  15. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proc. CVPR., pp. 1125–1134 (2017)

    Google Scholar 

  16. Jin, D., Xu, Z., Tang, Y., Harrison, A.P., Mollura, D.J.: CT-realistic lung nodule simulation from 3D conditional generative adversarial networks for robust lung segmentation. In: Proc. MICCAI., pp. 732–740. Springer (2018)

    Google Scholar 

  17. Kandel, E.R., Schwartz, J.H., Jessell, T.M., Siegelbaum, S., Hudspeth, A.J., Mack, S.: Principles of neural science, vol. 4. McGraw-hill New York (2000)

    Google Scholar 

  18. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  19. Kong, B., Sun, S., Wang, X., Song, Q., Zhang, S.: Invasive cancer detection utilizing compressed convolutional neural network and transfer learning. In: Proc. MICCAI., pp. 156–164. Springer (2018)

    Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25, 1097–1105 (2012)

    Google Scholar 

  21. Li, R., Zeng, T., Peng, H., Ji, S.: Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction. IEEE Trans. Med. Imag. 36(7), 1533–1541 (2017)

    Google Scholar 

  22. Liu, S., Zhang, D., Liu, S., Feng, D., Peng, H., Cai, W.: Rivulet: 3D neuron morphology tracing with iterative back-tracking. Neuroinformatics 14(4), 387–401 (2016)

    Google Scholar 

  23. Liu, S., Zhang, D., Song, Y., Peng, H., Cai, W.: Triple-crossing 2.5 D convolutional neural network for detecting neuronal arbours in 3D microscopic images. In: Proc. MLMI. Workshop, pp. 185–193. Springer (2017)

    Google Scholar 

  24. Liu, S., Zhang, D., Song, Y., Peng, H., Cai, W.: Automated 3-D neuron tracing with precise branch erasing and confidence controlled back tracking. IEEE Trans. Med. Imag. 37(11), 2441–2452 (2018)

    Google Scholar 

  25. Livet, J., Weissman, T.A., Kang, H., Draft, R.W., Lu, J., Bennis, R.A., Sanes, J.R., Lichtman, J.W.: Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system. Nature 450(7166), 56–62 (2007)

    Google Scholar 

  26. Makhzani, A., Frey, B.J.: Pixelgan autoencoders. In: Proc. NeurIPS., pp. 1975–1985 (2017)

    Google Scholar 

  27. McNally, J.G., Karpova, T., Cooper, J., Conchello, J.A.: Three-dimensional imaging by deconvolution microscopy. Methods 19(3), 373–385 (1999)

    Google Scholar 

  28. Meijering, E.: Neuron tracing in perspective. Cytometry Part A 77(7), 693–704 (2010)

    Google Scholar 

  29. Milletari, F., Navab, N., Ahmadi, S.: V-Net: Fully convolutional neural networks for volumetric medical image segmentation. In: Proc. 3DV., pp. 565–571. IEEE (2016)

    Google Scholar 

  30. Ming, X., Li, A., Wu, J., Yan, C., Ding, W., Gong, H., Zeng, S., Liu, Q.: Rapid reconstruction of 3D neuronal morphology from light microscopy images with augmented rayburst sampling. PloS one 8(12), e84,557 (2013)

    Google Scholar 

  31. Mukherjee, A., Stepanyants, A.: Automated reconstruction of neural trees using front re-initialization. In: SPIE Med. Imag., vol. 8314, p. 83141I. International Society for Optics and Photonics (2012)

    Google Scholar 

  32. Mukherjee, S., Condron, B., Acton, S.T.: Tubularity flow field—a technique for automatic neuron segmentation. IEEE Trans. Image Process. 24(1), 374–389 (2014)

    Google Scholar 

  33. Nicholls, J.G., Martin, A.R., Wallace, B.G., Fuchs, P.A.: From neuron to brain, vol. 271. Sinauer Associates Sunderland, MA (2001)

    Google Scholar 

  34. Pawley, J.: Handbook of biological confocal microscopy, vol. 236. Springer Science & Business Media (2006)

    Google Scholar 

  35. Peng, H., Hawrylycz, M., Roskams, J., Hill, S., Spruston, N., Meijering, E., Ascoli, G.A.: Bigneuron: large-scale 3D neuron reconstruction from optical microscopy images. Neuron 87(2), 252–256 (2015)

    Google Scholar 

  36. Peng, H., Long, F., Myers, G.: Automatic 3D neuron tracing using all-path pruning. Bioinformatics 27(13), i239–i247 (2011)

    Google Scholar 

  37. Peng, H., Meijering, E., Ascoli, G.A.: From diadem to bigneuron (2015)

    Google Scholar 

  38. Peng, H., Zhou, Z., Meijering, E., Zhao, T., Ascoli, G.A., Hawrylycz, M.: Automatic tracing of ultra-volumes of neuronal images. Nature methods 14(4), 332–333 (2017)

    Google Scholar 

  39. Purves, D., Cabeza, R., Huettel, S.A., LaBar, K.S., Platt, M.L., Woldorff, M.G., Brannon, E.M.: Cognitive neuroscience. Sunderland: Sinauer Associates, Inc (2008)

    Google Scholar 

  40. Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Kim, L., Summers, R.M.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imag. 35(5), 1170–1181 (2015)

    Google Scholar 

  41. Santamaría-Pang, A., Hernandez-Herrera, P., Papadakis, M., Saggau, P., Kakadiaris, I.A.: Automatic morphological reconstruction of neurons from multiphoton and confocal microscopy images using 3D tubular models. Neuroinformatics 13(3), 297–320 (2015)

    Google Scholar 

  42. Senft, S.L.: A brief history of neuronal reconstruction. Neuroinformatics 9(2-3), 119–128 (2011)

    Article  Google Scholar 

  43. Sethian, J.A.: Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science, vol. 3. Cambridge university press (1999)

    Google Scholar 

  44. Sintorn, I.M., Bischof, L., Jackway, P., Haggarty, S., Buckley, M.: Gradient based intensity normalization. J. Microsc. 240(3), 249–258 (2010)

    Google Scholar 

  45. Sironi, A., Türetken, E., Lepetit, V., Fua, P.: Multiscale centerline detection. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1327–1341 (2015)

    Google Scholar 

  46. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proc. CVPR., pp. 2818–2826 (2016)

    Google Scholar 

  47. Tang, Z., Zhang, D., Liu, S., Song, Y., Peng, H., Cai, W.: Automatic 3D single neuron reconstruction with exhaustive tracing. In: Proc. ICCV. Workshop, pp. 126–133 (2017)

    Google Scholar 

  48. Tang, Z., Zhang, D., Song, Y., Wang, H., Liu, D., Zhang, C., Liu, S., Peng, H., Cai, W.: 3D conditional adversarial learning for synthesizing microscopic neuron image using skeleton-to-neuron translation. In: Proc. ISBI., pp. 1775–1779. IEEE (2020)

    Google Scholar 

  49. Tsitsiklis, J.N.: Efficient algorithms for globally optimal trajectories. IEEE Trans. Autom. Control. 40(9), 1528–1538 (1995)

    Google Scholar 

  50. Van Uitert, R., Bitter, I.: Subvoxel precise skeletons of volumetric data based on fast marching methods. Medical physics 34(2), 627–638 (2007)

    Google Scholar 

  51. Wallén, P., Carlsson, K., Liljeborg, A., Grillner, S.: Three-dimensional reconstruction of neurons in the lamprey spinal cord in whole-mount, using a confocal laser scanning microscope. J. Neurosci. Methods 24(2), 91–100 (1988)

    Google Scholar 

  52. Wang, C.W., Lee, Y.C., Pradana, H., Zhou, Z., Peng, H.: Ensemble neuron tracer for 3D neuron reconstruction. Neuroinformatics 15(2), 185–198 (2017)

    Google Scholar 

  53. Wang, H., Song, Y., Zhang, C., Yu, J., Liu, S., Peng, H., Cai, W.: Single neuron segmentation using graph-based global reasoning with auxiliary skeleton loss from 3D optical microscope images. ISBI (2021)

    Google Scholar 

  54. Wang, H., Zhang, D., Song, Y., Liu, S., Gao, R., Peng, H., Cai, W.: Memory and time efficient 3D neuron morphology tracing in large-scale images. In: Proc. DICTA., pp. 1–8. IEEE (2018)

    Google Scholar 

  55. Wang, H., Zhang, D., Song, Y., Liu, S., Huang, H., Chen, M., Peng, H., Cai, W.: Multiscale kernels for enhanced U-shaped network to improve 3D neuron tracing. In: Proc. CVPR. Workshop, pp. 1105–1113 (2019)

    Google Scholar 

  56. Wang, H., Zhang, D., Song, Y., Liu, S., Wang, Y., Feng, D., Peng, H., Cai, W.: Segmenting neuronal structure in 3D optical microscope images via knowledge distillation with teacher-student network. In: Proc. ISBI., pp. 228–231. IEEE (2019)

    Google Scholar 

  57. Wang, Y., Narayanaswamy, A., Tsai, C.L., Roysam, B.: A broadly applicable 3-D neuron tracing method based on open-curve snake. Neuroinformatics 9(2-3), 193–217 (2011)

    Google Scholar 

  58. Xiao, H., Peng, H.: App2: automatic tracing of 3D neuron morphology based on hierarchical pruning of a gray-weighted image distance-tree. Bioinformatics 29(11), 1448–1454 (2013)

    Google Scholar 

  59. Yang, J., Gonzalez-Bellido, P.T., Peng, H.: A distance-field based automatic neuron tracing method. BMC bioinformatics 14(1), 1–11 (2013)

    Google Scholar 

  60. Yang, J., Liu, S., Grbic, S., Setio, A.A.A., Xu, Z., Gibson, E., Chabin, G., Georgescu, B., Laine, A.F., Comaniciu, D.: Class-aware adversarial lung nodule synthesis in CT images. In: Proc. ISBI., pp. 1348–1352. IEEE (2019)

    Google Scholar 

  61. Zhang, D., Liu, S., Liu, S., Feng, D., Peng, H., Cai, W.: Reconstruction of 3D neuron morphology using rivulet back-tracking. In: Proc. ISBI., pp. 598–601. IEEE (2016)

    Google Scholar 

  62. Zhang, D., Liu, S., Song, Y., Feng, D., Peng, H., Cai, W.: Automated 3D soma segmentation with morphological surface evolution for neuron reconstruction. Neuroinformatics 16(2), 153–166 (2018)

    Google Scholar 

  63. Zhou, Z., Liu, X., Long, B., Peng, H.: TReMAP: automatic 3D neuron reconstruction based on tracing, reverse mapping and assembling of 2D projections. Neuroinformatics 14(1), 41–50 (2016)

    Google Scholar 

  64. Zhou, Z., Sorensen, S.A., Peng, H.: Neuron crawler: An automatic tracing algorithm for very large neuron images. In: Proc. ISBI., pp. 870–874. IEEE (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University of Sydney, Sydney, NSW, Australia

    Heng Wang, Zihao Tang, Chaoyi Zhang, Jianhui Yu, Dongnan Liu & Weidong Cai

  2. School of Computer Science and Engineering, University of New South Wales, Sydney, NSW, Australia

    Yang Song

  3. Monash e-Research Centre, Monash University, Melbourne, VIC, Australia

    Donghao Zhang

  4. Digital Services, Digital Technology & Innovation, Paige AI, New York, NY, USA

    Siqi Liu

Authors
  1. Heng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zihao Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chaoyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianhui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dongnan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Donghao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Siqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Weidong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weidong Cai .

Editor information

Editors and Affiliations

  1. University of Technology Sydney, Sydney, NSW, Australia

    Fang Chen

  2. University of Technology Sydney, Sydney, NSW, Australia

    Jianlong Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wang, H. et al. (2022). AI-Enhanced 3D Biomedical Data Analytics for Neuronal Structure Reconstruction. In: Chen, F., Zhou, J. (eds) Humanity Driven AI. Springer, Cham. https://doi.org/10.1007/978-3-030-72188-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-72188-6_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72187-9

  • Online ISBN: 978-3-030-72188-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation