The DIADEM Metric: Comparing Multiple Reconstructions of the Same Neuron

Abstract

Digital reconstructions of neuronal morphology are used to study neuron function, development, and responses to various conditions. Although many measures exist to analyze differences between neurons, none is particularly suitable to compare the same arborizing structure over time (morphological change) or reconstructed by different people and/or software (morphological error). The metric introduced for the DIADEM (DIgital reconstruction of Axonal and DEndritic Morphology) Challenge quantifies the similarity between two reconstructions of the same neuron by matching the locations of bifurcations and terminations as well as their topology between the two reconstructed arbors. The DIADEM metric was specifically designed to capture the most critical aspects in automating neuronal reconstructions, and can function in feedback loops during algorithm development. During the Challenge, the metric scored the automated reconstructions of best-performing algorithms against manually traced gold standards over a representative data set collection. The metric was compared with direct quality assessments by neuronal reconstruction experts and with clocked human tracing time saved by automation. The results indicate that relevant morphological features were properly quantified in spite of subjectivity in the underlying image data and varying research goals. The DIADEM metric is freely released open source (http://diademchallenge.org) as a flexible instrument to measure morphological error or change in high-throughput reconstruction projects.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Notes

  1. 1.

    http://www.neuronland.org/NLMorphologyConverter/MorphologyFormats/SWC/Spec.html

References

  1. Ascoli, G. A. (2002). Neuroanatomical algorithms for dendritic modelling. Network, 13, 247–260.

    PubMed  Article  Google Scholar 

  2. Ascoli, G. A., Alonso-Nanclares, L., Anderson, S. A., Barrionuevo, G., Benavides-Piccione, R., Burkhalter, A., et al. (2008). Petilla terminology: nomenclature of features of GABAergic interneurons of the cerebral cortex. Nature Reviews Neuroscience, 9, 557–568.

    PubMed  Article  CAS  Google Scholar 

  3. Baloyannis, S. J. (2009). Dendritic pathology in Alzheimer’s disease. Journal of the Neurological Sciences, 283, 153–157.

    PubMed  Article  CAS  Google Scholar 

  4. Binzegger, T., Douglas, R. J., & Martin, K. A. (2004). A quantitative map of the circuit of cat primary visual cortex. Journal of Neuroscience, 24, 8441–8453.

    PubMed  Article  CAS  Google Scholar 

  5. Brown, K. M., Donohue, D. E., D’Alessandro, G., & Ascoli, G. A. (2005). A cross-platform freeware tool for digital reconstruction of neuronal arborizations from image stacks. Neuroinformatics, 3, 343–359.

    PubMed  Article  Google Scholar 

  6. Brown, K. M., Gillette, T. A., & Ascoli, G. A. (2008). Quantifying neuronal size: summing up trees and splitting the branch difference. Seminars in Cell & Developmental Biology, 19, 485–493.

    Article  Google Scholar 

  7. Brown, K. M., Barrionuevo, G., Canty, A. J., De Paola, V., Hirsch, J. A., Jefferis, G. S. X. E., et al. (2011) The DIADEM data sets: representative light microscopy images of neuronal morphology to advance automation of digital reconstructions. Neuroinformatics, doi:10.1007/s12021-010-9095-5.

  8. Bülow, T., Lorenz, C., Wiemker, R., & Honko, J. (2006). Point based methods for automatic bronchial tree matching and labeling. Proceedings of the SPIE, 7, 225–234.

    Google Scholar 

  9. Canty, A. J. & De Paola, V. (2011) Axonal reconstructions going live. Neuroinformatics, doi:10.1007/s12021-011-9112-3.

  10. Capowski, J. J. (1983). An automated neuron reconstruction system. Journal of Neuroscience Methods, 8, 353–364.

    PubMed  Article  CAS  Google Scholar 

  11. Cardona, A., Saalfeld, S., Arganda, I., Pereanu, W., Schindelin, J., & Hartenstein, V. (2010). Identifying neuronal lineages of Drosophila by sequence analysis of axon tracts. Journal of Neuroscience, 30, 7538–7553.

    PubMed  Article  CAS  Google Scholar 

  12. Charnoz, A., Agnus, V., Malandain, G., Soler, L., & Tajine, M. (2005). Tree matching applied to vascular system. In L. Brun & M. Vento (Eds.), Graph-based representations in pattern recognition (pp. 183–192). Berlin: Springer.

    Google Scholar 

  13. Chklovskii, D. B., Vitaladevuni, S., & Scheffer, L. K. (2010). Semi-automated reconstruction of neural circuits using electron microscopy. Current Opinion in Neurobiology, 20, 667–675.

    PubMed  Article  CAS  Google Scholar 

  14. Cline, H. (2001). Dendritic arbor development and synaptogenesis. Current Opinion in Neurobiology, 11, 118–126.

    PubMed  Article  CAS  Google Scholar 

  15. Cuntz, H., Forstner, F., Borst, A., & Häusser, M. (2011). The TREES toolbox – probing the basis of axonal and dendritic branching. Neuroinformatics, in press.

  16. Drechsler, K., Laura, C. O., Chen, Y., & Erdt, M. (2010). Semi-automatic anatomical tree matching for landmark-based elastic registration of liver volumes. Journal of Healthcare Engineering, 1, 101–124.

    Article  Google Scholar 

  17. Gillette, T. A., & Grefenstette, J. J. (2009). On comparing neuronal morphologies with the constrained tree-edit-distance. Neuroinformatics, 7, 191–194.

    PubMed  Article  Google Scholar 

  18. Glaser, E. M., & Van der Loos, H. (1965). A semi-automatic computer microscope for the analysis of neuronal morphology. IEEE Transactions on Biomedical Engineering, 12, 22–40.

    PubMed  Article  CAS  Google Scholar 

  19. Glaser, J. R., & Glaser, E. M. (1990). Neuron imaging with Neurolucida—a PC-based system for image combining microscopy. Computerized Medical Imaging and Graphics, 14, 307–317.

    PubMed  Article  CAS  Google Scholar 

  20. Goldberg, J., Hamzei-Sichani, F., MacLean, J., Tamas, G., Urban, R., & Yuste, R. (2006). From dendrites to networks: optically probing the living brain slice and using principal component analysis to characterize neuronal morphology. In L. Zaborszky, F. G. Wouterlood, & J. L. Lanciego (Eds.), Neuroanatomical tract-tracing 3: Molecules, neurons, and systems (pp. 452–476). US: Springer.

    Google Scholar 

  21. Hao, H., & Shreiber, D. I. (2007). Axon kinematics change during growth and development. Journal of Biomechanical Engineering, 129, 511–522.

    PubMed  Article  Google Scholar 

  22. Haug, H. (1987). Brain sizes, surfaces, and neuronal sizes of the cortex cerebri: a stereological investigation of man and his variability and a comparison with some mammals (primates, whales, marsupials, insectivores, and one elephant). American Journal of Anatomy, 180, 126–142.

    PubMed  Article  CAS  Google Scholar 

  23. Heumann, H., & Wittum, G. (2009). The tree-edit-distance, a measure for quantifying neuronal morphology. Neuroinformatics, 7, 179–190.

    PubMed  Article  Google Scholar 

  24. Jaeger, D. (2001) Accurate reconstruction of neuronal morphology. In E. de Schutter (ed.), Computational neuroscience: Realistic modeling for experimentalists. CRC Press, pp. 159–178.

  25. Kaspirzhny, A. V., Gogan, P., Horcholle-Bossavit, G., & Tyc-Dumont, S. (2002). Neuronal morphology data bases: morphological noise and assesment of data quality. Network, 13, 357–380.

    PubMed  Article  Google Scholar 

  26. Kasthuri, N., & Lichtman, J. W. (2010). Neurocartography. Neuropsychopharmacology, 35, 342–343.

    PubMed  Article  Google Scholar 

  27. Koene, R. A., Tijms, B., van Hees, P., Postma, F., de Ridder, A., Ramakers, G. J., et al. (2009). NETMORPH: a framework for the stochastic generation of large scale neuronal networks with realistic neuron morphologies. Neuroinformatics, 7, 195–210.

    PubMed  Article  Google Scholar 

  28. Krichmar, J. L., Nasuto, S. J., Scorcioni, R., Washington, S. D., & Ascoli, G. A. (2002). Effects of dendritic morphology on CA3 pyramidal cell electrophysiology: a simulation study. Brain Research, 941, 11–28.

    PubMed  Article  CAS  Google Scholar 

  29. Li, Y., Brewer, D., Burke, R. E., & Ascoli, G. A. (2005). Developmental changes in spinal motoneuron dendrites in neonatal mice. Journal of Comparative Neurology, 483, 304–317.

    PubMed  Article  Google Scholar 

  30. Lin, B., & Masland, R. H. (2005). Synaptic contacts between an identified type of ON cone bipolar cell and ganglion cells in the mouse retina. The European Journal of Neuroscience, 21, 1257–1270.

    PubMed  Article  Google Scholar 

  31. Losavio, B. E., Liang, Y., Santamaría-Pang, A., Kakadiaris, I. A., Colbert, C. M., & Saggau, P. (2008). Live neuron morphology automatically reconstructed from multiphoton and confocal imaging data. Journal of Neurophysiology, 100, 2422–2429.

    PubMed  Article  Google Scholar 

  32. Lu, J., Tapia, J. C., White, O. L., & Lichtman, J. W. (2009). The interscutularis muscle connectome. PLoS Biology, 7, e1000032.

    Google Scholar 

  33. Luisi, J., Narayanaswamy, A., Galbreath, Z., & Roysam, B. (2011). The FARSIGHT Trace Editor: An Open Source Tool for 3-D Inspection and Efficient Pattern Analysis Aided Editing of Automated Neuronal Reconstructions. Neuroinformatics, doi:10.1007/s12021-011-9115-0.

  34. Mainen, Z., & Sejnowski, T. (1996). Influence of dendritic structure on firing pattern in model neocortical neurons. Nature, 382, 363–366.

    PubMed  Article  CAS  Google Scholar 

  35. Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004). Interneurons of the neocortical inhibitory system. Nature Reviews Neuroscience, 5, 793–807.

    PubMed  Article  CAS  Google Scholar 

  36. Marks, W. B., & Burke, R. E. (2007). Simulation of motoneuron morphology in three dimensions. I. Building individual dendritic trees. The Journal of Comparative Neurology, 503, 685–700.

    PubMed  Article  Google Scholar 

  37. Metzen, J. H., Kröger, T., Schenk, A., Zidowitz, S., Peitgen, H., & Jiang, X. (2009). Matching of anatomical tree structures for registration of medical images. Image and Vision Computing, 27, 923–933.

    Article  Google Scholar 

  38. Meyer-Luehmann, M., Spires-Jones, T. L., Prada, C., Garcia-Alloza, M., de Calignon, A., Rozkalne, A., et al. (2008). Rapid appearance and local toxicity of amyloid-beta plaques in a mouse model of Alzheimer’s disease. Nature, 451, 720–724.

    PubMed  Article  CAS  Google Scholar 

  39. Mize, R. R. (1984). Computer applications in cell and neurobiology: a review. International Review of Cytology, 90, 83–124.

    PubMed  Article  CAS  Google Scholar 

  40. Overdijk, J., Uylings, H. B. M., Kuypers, K., & Kamstra, A. W. (1978). An economical semi-automatic system for measuring cellular tree structures in three dimensions, with special emphasis on Golgi-impregnated neurons. Journal of Microscopy, 114, 271–284.

    PubMed  CAS  Google Scholar 

  41. Peng, H., Ruan, Z., Long, F., Simpson, J. H., & Myers, E. W. (2010). V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nature Biotechnology, 28, 348–353.

    PubMed  Article  CAS  Google Scholar 

  42. Peng, H., Ruan, Z., Atasoy, D., & Sternson, S. (2010). Automatic reconstruction of 3D neuron structures using a graph-augmented deformable model. Bioinformatics, 26, i38–i46.

    PubMed  Article  CAS  Google Scholar 

  43. Peng, H., Long, F., Zhao, T., & Myers, E. (2011). Proof-editing is the bottleneck of 3D neuron reconstruction: the problem and solutions. Neuroinformatics, doi:10.1007/s12021-010-9090-x.

  44. Rodriguez, A., Ehlenberger, D. B., Hof, P. R., & Wearne, S. L. (2009). Three-dimensional neuron tracing by voxel scooping. Journal of Neuroscience Methods, 184, 169–175.

    PubMed  Article  Google Scholar 

  45. Schaap, M., Metz, C. T., van Walsum, T., van Der Giessen, A. G., Weustink, A. C., Mollet, N. R., et al. (2009). Standardized evaluation methodology and reference database for evaluating coronary artery centerline extraction algorithms. Medical Image Analysis, 13, 701–714.

    PubMed  Article  Google Scholar 

  46. Schaefer, A. T., Larkum, M. E., Sakmann, B., & Roth, A. (2003). Coincidence detection in pyramidal neurons is tuned by their dendritic branching pattern. Journal of Neurophysiology, 89, 3143–3154.

    PubMed  Article  Google Scholar 

  47. Scorcioni, R., Lazarewicz, M. T., & Ascoli, G. A. (2004). Quantitative morphometry of hippocampal pyramidal cells: differences between anatomical classes and reconstructing laboratories. The Journal of Comparative Neurology, 473, 177–93.

    PubMed  Article  Google Scholar 

  48. Senft, S. L. (2011). A brief history of neuronal reconstruction. Neuroinformatics , doi:10.1007/s12021-011-9107-0.

  49. Stepanyants, A., & Chklovskii, D. (2005). Neurogeometry and potential synaptic connectivity. Trends in Neuroscience, 28, 387–394.

    Article  CAS  Google Scholar 

  50. Stepanyants, A., Tamás, G., & Chklovskii, D. B. (2004). Class-specific features of neuronal wiring. Neuron, 43, 251–259.

    PubMed  Article  CAS  Google Scholar 

  51. Sugihara, I., Wu, H., & Shinoda, Y. (1996). Morphology of axon collaterals of single climbing fibers in the deep cerebellar nuclei of the rat. Neuroscience Letters, 217, 33–36.

    PubMed  Article  CAS  Google Scholar 

  52. Tschirren, J., McLennan, G., Palágyi, K., Hoffman, E. A., & Sonka, M. (2005). Matching and anatomical labeling of human airway tree. IEEE Transactions on Medical Imaging, 24, 1540–1547.

    PubMed  Article  Google Scholar 

  53. Tyrrell, J. A., di Tomaso, E., Fuja, D., Tong, R., Kozak, K., Jain, R. K., et al. (2007). Robust 3-D modeling of vasculature imagery using superellipsoids. IEEE Transactions on Medical Imaging, 26, 223–237.

    PubMed  Article  Google Scholar 

  54. Van Ooyen, A., Duijnhouwer, J., Remme, M., & van Pelt, J. (2002). The effect of dendritic topology on firing patterns in model neurons. Network: Computation in Neural Systems, 13, 311–325.

    Article  Google Scholar 

  55. Van Pelt, J., Uylings, H. B. M., Verwer, R. W. H., Pentney, R. J., & Woldenberg, M. J. (1992). Tree asymmetry—a sensitive and practical measure for binary topological trees. Bulletin of Mathematical Biology, 54(5), 759–784.

    PubMed  Article  Google Scholar 

  56. van Praag, H., Kempermann, G., & Gage, F. H. (2000). Neural consequences of environmental enrichment. Nature Reviews Neuroscience, 1, 191–198.

    PubMed  Article  Google Scholar 

  57. Vetter, P., Roth, A., & Häusser, M. (2001). Propagation of action potentials in dendrites depends on dendritic morphology. Journal of Neurophysiology, 85, 926–937.

    PubMed  CAS  Google Scholar 

  58. Wearne, S. L., Rodriguez, A., Ehlenberger, D. B., Rocher, A. B., Henderson, S. C., & Hof, P. R. (2005). New techniques for imaging, digitization and analysis of three-dimensional neural morphology on multiple scales. Neuroscience, 136, 661–680.

    PubMed  Article  CAS  Google Scholar 

  59. Wong, R. O., & Ghosh, A. (2002). Activity-dependent regulation of dendritic growth and patterning. Nature Reviews Neuroscience, 3, 803–812.

    PubMed  Article  CAS  Google Scholar 

  60. Zhang, K. (1996). A constrained edit distance between unordered labeled trees. Algorithmica, 15, 205–222.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We are grateful to Dr. Karel Svoboda for early discussions on the development of the DIADEM metric. This work was supported in part by HHMI and NIH grant R01NS39600.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Giorgio A. Ascoli.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gillette, T.A., Brown, K.M. & Ascoli, G.A. The DIADEM Metric: Comparing Multiple Reconstructions of the Same Neuron. Neuroinform 9, 233 (2011). https://doi.org/10.1007/s12021-011-9117-y

Download citation

Keywords

  • Algorithm
  • Automation
  • Axon
  • Computational neuroanatomy
  • Dendrite
  • Digital tracing
  • Morphology
  • Optical imaging