The development of multi-photon imaging technique has greatly facilitated in vivo time-lapse imaging and enables comparison of the fine morphological structures of individual neurons over time. Despite the fact that 4D data acquisition has become easier and can be applied to a variety of brain tissues, both in vivo and in tissue slices, the analysis of these 4D data remains extremely laborious and painstaking. Manual analysis greatly limits the pace of research and introduces errors. Recent work suggests that an automated dynamic analysis tool can be successfully applied to time-lapse images of cultured neurons,Footnote 1 which are flat and relatively simple. So far no such automated analysis program has yet been reported for in vivo 4D image analysis, which poses several technical challenges, including alignment of complex 3 dimensional structures across time points, and identification of persistent and dynamic structures.

The goals of the Diadem Challenge were to generate algorithms for automated reconstruction of light microscope images of 3D neural structures. Quantitative characterization of neuronal structure will be extremely valuable for the identification of cell types, to establish and interpret neuronal connectivity maps, for comparative analysis across individuals, brain regions or experimental conditions, and to identify plasticity events and mechanisms that regulate plasticity, for instance by analysis of the structure of neurons from animals of different genotypes or with different experience or training. These goals are based on the premise that neuronal morphology is stereotypic, so that data from single reconstructions are meaningful within a dataset.

Participants of the Diadem Challenge explored different model-based algorithms based on either local voxel information or global objective function of the image to automatically reconstruct the neuronal structure.Footnote 2 The ultimate goal is to achieve completely automated reconstruction with high precision and efficiency. Progress demonstrated by participants of the Diadem Challenge indicates that the development of algorithms for automated reconstruction of neuronal structures is very promising and will be able to significantly facilitate supervised reconstruction of neurons in complex data sets. This progress is very exciting and will have a huge impact on current research in basic neuroscience and as it pertains to human disease.

One important advance from this point will be to automate analysis of time-lapse imaging data. Applications of automated analysis of time-lapse imaging data include analysis of the development of neuronal structure and acquisition of cell type specific morphological features, analysis of mechanisms of structural plasticity, and events underlying degenerative and regenerative events.

Classical Work Comparing Morphologies of Populations of Neurons

Classical experiments have demonstrated that neuronal structure changes in response to sensory experience. For instance, using both electrophysiological and anatomical methods, Hubel, Weisel, Levay and colleagues showed ocular dominance column plasticity in non-human primates (Fig. 1) and other mammals with front-facing eyes.Footnote 3 Using the three-eyed frog experimental system,Footnote 4 Reh and Constantine-Paton showed that blockade of action potential activity with tetrodotoxin (TTX) desegregated ocular dominance columns,Footnote 5 , Footnote 6 a result that was subsequently reproduced in mammals.6 These results indicated that action potential activity in sensory inputs was an important element in organizing central projections. Reh and Constantine-Paton made another important observation: they reported that TTX treatment increased the elaboration of retinal axon arbors.5 Although the mechanisms underlying this widely observed effect of TTX treatment are still unclear, this was the first demonstration that neuronal activity affected the growth and structure of individual neurons. Subsequently Antonini and Stryker conducted heroic experiments (Fig. 1) which demonstrated that the morphology of individual geniculocortical axons changes over periods of days in response to decreased visual experience.Footnote 7 Specifically, by comparing populations of neurons from animals treated with monocular deprivation, they found that geniculocortical axons carrying information in the open eye pathway elaborated more complex axon arbors than axons in the deprived-eye pathway.Footnote 8 These experiments were important because they demonstrated that sensory input activity governed the elaboration of neuronal axons and that the gross re-organization of ocular dominance columns in monocularly-deprived animals seen using radioactive tracers reported a population-level change in neuronal structure, rather than, for instance, a change in the distribution of axons within layer 4 of visual cortex. It is important to point out that these conclusions were generated by comparing populations of neurons from animals at different stages and treated with different visual stimulation or deprivation paradigms, so that specific information about cellular mechanisms governing elaboration or regression of axon arbor development could not be determined.

Fig. 1
figure 1

a Ocular dominance plasticity in non-human primates following monocular deprivation, according to autoradiographic visualization of radiolabel injected into one eye. Adapted from Hubel, Levay and Weisell, 1977. b Examples of geniculocortical axon arbor structures from cats with monocular deprivation. Axons from the closed eye and open eye pathways are shown. Adapted from Antonini and Stryker, 1993

Full size image

Time-Lapse in vivo Imaging: the Value of 4 Dimensional Data Sets

Many studies have documented the invasion and development of axon arbors by comparing samples across different developmental timepoints.Footnote 9 In parallel, other studies demonstrated an increase followed by a gradual decrease in synapse density.Footnote 10 Together these studies suggested a model in which axon arbors go through a period of exuberant elaboration and excess synaptogenesis followed by an elimination phase, in which both synapses and axon branches were pruned. As described by Hua and Smith,Footnote 11 this classical model of sequential axon arbor elaboration and pruning is not borne out by time-lapse in vivo imaging of developing retinotectal axons in Xenopus frog tadpoles and Zebrafish.Footnote 12 , Footnote 13 Rather, branch addition and synaptogenesis are concurrent with branch retraction and synapse elimination for both axons and dendrites, as suggested by light microscopy time-lapse dataFootnote 14 and demonstrated more conclusively by combining in vivo time-lapse imaging with subsequent serial section electron microscopy.Footnote 15 Importantly, the final structure of the axon is indistinguishable (Fig. 2), and these fundamental differences in the cellular mechanisms, and therefore the molecular/genetic/signaling events underlying arbor development, would only be recognized by time-lapse in vivo imaging data. One repercussion of the concurrent model is that the protracted period of synaptic and branch dynamics may provide greater opportunities for treatment interventions in the event of trauma to the nervous system. This serves as but one example of the essential need for in vivo time-lapse imaging data for accurate identification of mechanisms governing brain development, circuit plasticity and neurological diseases.

Fig. 2
figure 2

Diagram of the differences between the sequential and concurrent models of neuronal arbor development. In the ‘sequential’ model, axons have a period of exuberant process outgrowth and synaptogenesis, followed by a separate period of branch pruning and synapse elimination. In the ‘concurrent’ model, branch addition and synaptogenesis are contemporaneous with branch retraction and synapse elimination. Modified from Hua and Smith, 2004

Full size image

Analysis of 4 Dimensional Data Sets

The development of multi-photon imaging techniques has greatly facilitated in vivo time-lapse imaging, which enables comparison of the fine morphological structures of individual neurons over time. Despite the fact that 4D data acquisition has become easier and can be applied to a variety of brain tissues, both in vivo and in tissue slices, the analysis of these 4D data remains extremely laborious and painstaking. Although there have been many successful studies in this area, which have provided intriguing and valuable information for understanding mechanisms of neuronal structural plasticity, the analysis is mostly done by manually scoring the features of neuronal structures, such as branch tips/spines/boutons in either 3D reconstructions or 2D maximum intensity projection images, and comparing these features across reconstructions from separate time points. This manual analysis greatly limits the pace of research and introduces errors. In addition, different scoring criteria used by different labs or different individuals within labs likely contribute to discrepancies in the experimental results, which brings additional difficulties in the comparison of results from different labs.Footnote 16

The magnitude of structural changes in time-lapse imaging data can determine the choice of analysis and type of data presentation. The first time-lapse in vivo imaging experiments of retinotectal axon arbor development collected images at daily intervals.13 Because the changes in the arbor structure were often large (Fig. 3), data were analyzed by quantifying cumulative increases in features of neuronal structure, such as total branch length or branch tip numbers, based on 2 dimensional drawings of reconstructed neurons.13 Studies in which images were collected at more frequent intervals allowed identification of specific dynamic events, including branch addition, extension, shortening or retraction or branch stabilization within axonal or dendritic arbors either from manual drawings of neurons,Footnote 17 or maximal image projections and manual analysis of spine dynamics or using custom macros, for instance in MatLab.Footnote 18 Although these studies do reveal structural dynamics associated with a variety of normal and pathological conditions, analysis of 2 dimensional data with manual scoring introduces a variety of errors.

Fig. 3
figure 3

Examples of large-scale rearrangements of retinotectal axon arbor structure observed by in vivo time-lapse images collected at daily intervals. From O’Rourke and Fraser, 1990

Full size image

Subsequently, reconstructions were done to maintain the 3 dimensional information content, so more accurate identification of neuronal branches and spines could be made and more accurate measurements of changes in dendritic, axonal or spine structures could be accomplished using macros written in NIH Image, Image JFootnote 19 or other custom analysis software.Footnote 20 Quantitative analysis of structural dynamics includes changes in dynamic behaviors of individual branches, branch lifetimes, and rates of branch additions or retractions.Footnote 21 Further automated quantitative analysis of changes in length of dynamic structures required the experimenter to manually assign an identification number manually to each branch tip or spine and that the same id number be maintained for the branch for all time points.Footnote 22 Currently, quantitative analysis of time-lapse imaging data is done predominantly by labor-intensive comparisons of sequential pairs of 3 dimensional reconstructions of structures from individual time-points until reconstructions through the entire time-lapse sequence have been compared. Each branch within the axonal or dendritic arbor is assigned an identifier, each maintained branch is then identified in the subsequent reconstruction and length changes in each maintained branch are determined by comparing measurements in the 3-dimensional datasets. In addition, branches which either disappear or are added during the imaging interval are noted, with the assignment of new identifiers, as necessary (Fig. 4). Quantitative analysis of time-lapse data requires particular care be taken in the initial 3D reconstructions of the neuronal structures, so the errors in the identification of branch dynamic events do not arise from misidentification of branches throughout the imaging sequence. In our experience, computer-assisted manual reconstruction of the entire dendritic or axonal arbor for a single time-point of one neuron using commercially available software takes from ~1–4 h depending on the complexity of the neuronal morphology. As mentioned above, alignment of reconstructions from multiple time-points for dynamic analysis and quantitative analysis requires that branchpoints, branch tips/spines/boutons have unique identifiers that are propagated through the time-series. Assignment and propagation of the identifiers through two aligned reconstructions can take an additional 1–5 h, again depending on the complexity and plasticity of the structures. In addition to increasing the pace of quantitative analysis of 4 D data sets, broadly applicable and broadly available automated alignment and automated quantitative analysis methods would significantly improve the reproducibility of the analysis.

Fig. 4
figure 4

Analysis and presentation of data from time-lapse images of dendritic arbor plasticity. a Dendritic arbor plasticity observed by in vivo time-lapse images collected at 2 h intervals over 6 h. This imaging protocol allows every branch in the arbor to be identified and their dynamic behaviors to be presented in a chronogram (right). Adapted from Wu and Cline, 1998. b Rearrangements of dendritic arbor structures observed with images collected at 2-hour intervals over 6 h permit a variety of representations of branch dynamics. Here, branches were color coded according to their dynamics: the grey arbor is stable throughout the imaging period, red branches were present at the first image and disappeared by the last image, green branches were transient, and yellow branches were added during the imaging period and maintained to the final image. Adapted from Haas et al., 2006. c Branch dynamics can be quantified as branch additions, extensions, retractions, eliminations, and presented as the relative proportion of branches with different behaviors under separate experimental conditions

Full size image

Some recent progress in automated analysis of time-lapse image data has been made for comparison of neuronal structures, based on algorithms to identify differences in sequential 2 dimensional images with greater than 85% accuracy.1 Several major technical obstacles remain for analysis of time-lapse 3 dimensional image data sets. One is alignment of the complex neuronal arbors across different time points. It is particularly difficult to align images of complex structures over longer imaging intervals where structural changes between subsequent 3D images may be large (as in Fig. 3). A second technical issue is the identification of persistent and new structures (branch tips, boutons, spines) in sequential images. Despite the fact that comparison of 3 dimensional data sets remains a significant challenge, recent progress in a number of labs suggests that semi-automated analysis of time-lapse images of neuronal structures is a tractable problem within the near future. Such automation will have a significant impact on the ability to assess developmental, plasticity-induced, regressive or therapeutic changes in nervous system structure and will follow smoothly from the advances seen as a result of the Diadem Challenge.