NeuroPath2Path: Classification and elastic morphing between neuronal arbors using path-wise similarity

Abstract

Neuron shape and connectivity affect function. Modern imaging methods have proven successful at extracting morphological information. One potential path to achieve analysis of this morphology is through graph theory. Encoding by graphs enables the use of high throughput informatic methods to extract and infer brain function. However, the application of graph-theoretic methods to neuronal morphology comes with certain challenges in term of complex subgraph matching and the difficulty in computing intermediate shapes in between two imaged temporal samples. Here we report a novel, efficacious graph-theoretic method that rises to the challenges. The morphology of a neuron, which consists of its overall size, global shape, local branch patterns, and cell-specific biophysical properties, can vary significantly with the cell’s identity, location, as well as developmental and physiological state. Various algorithms have been developed to customize shape based statistical and graph related features for quantitative analysis of neuromorphology, followed by the classification of neuron cell types using the features. Unlike the classical feature extraction based methods from imaged or 3D reconstructed neurons, we propose a model based on the rooted path decomposition from the soma to the dendrites of a neuron and extract morphological features from each constituent path. We hypothesize that measuring the distance between two neurons can be realized by minimizing the cost of continuously morphing the set of all rooted paths of one neuron to another. To validate this claim, we first establish the correspondence of paths between two neurons using a modified Munkres algorithm. Next, an elastic deformation framework that employs the square root velocity function is established to perform the continuous morphing, which, as an added benefit, provides an effective visualization tool. We experimentally show the efficacy of NeuroPath2Path, NeuroP2P, over the state of the art.

Appendix: : NeuroP2P in detail

Feature extraction on a path

We extract a set of discriminating features from each path f_i ∈Γ of H, which are bifurcation angle (b_i), concurrence (C_i), hierarchy (ξ_i), divergence (λ_i), segment length (β_i), tortuosity (κ_i), and partition asymmetry (α_i). Therefore, ${\Theta }_{i} = [b_{i}, C_{i}, \xi _{i}, \lambda _{i}, {\upbeta }_{i}, \kappa _{i}, \alpha _{i}]\in \mathbb {R}^{\phi \times 7}$. Each feature encodes a specific structural property of a neuronal arbor, as described in the next section. A schematic of different features along with the systematic quantification is shown in Fig.4.

Description of features

We extract a set of discriminating features on each path f_i ∈Γ of H, which are bifurcation angle (b_i), concurrence (C_i), hierarchy (ξ_i), divergence (λ_i), segment length (β_i), tortuosity (κ_i), and partition asymmetry (α_i).

• Bifurcation angle is a key morphometric that dictates the span and the spatial volume of an arbor. It is hypothesized that the span of an arbor at each level of bifurcation depends on the bifurcation of its previous level (López-Cruz et al. 2011; Batabyal et al. 2018b; Bielza et al. 2014), suggesting the influence of Bayesian philosophy. This organizational principle is utilized in several stochastic generative models (López-Cruz et al. 2011) for the synthesis of specific neuron cell types. The sequence of bifurcation angles at bifurcation vertices located on a path of a neuron captures local geometry. For example, a sequence of non-increasing bifurcation angles from the root to the dendritic terminal of a path indicates the pyramidal shape geometry of the neuron. For a location with multifurcation, we use the maximum of the bifurcation angles computed using pairwise branches originated from that location towards the dendritic terminals.

• Concurrence, hierarchy and divergence encode the effect of phenomenological factors, which are exploration (ex. Purkinje fanning out rostrocaudally) and competition (ex. retinal ganglion cells), that contribute in the growth of a neuron. The definition of concurrence and hierarchy are already given in Section 3. The divergence of a location on a path, f_i is proportional to the repulsive force that the location experiences from its neighborhood path segments. Let $C_{f_{i}}$ be the sequence of concurrence values of the path f_i ∈Γ when one visits the locations from the root to the dendritic terminal. As an open curve, each path can be parameterized with the parameter t ∈ [0, 1]. $C_{f_{i}}(t_{s}) = k;~t_{s}\in [0,1]$ indicates that k(≤|Γ|) paths share the location t_s on f_i. The divergence λ of a location f_i(t_s) is defined as λ(f_i(t_s)) =$1_{\{f_{j}~|~|f_{j}(t)-f_{i}(t_{s})|\le \epsilon , f_{j}\neq f_{i} , f_{j}\nsucc f_{i} \}}$. Here, 1 is the indicator function computing the number of such f_js which follow the conditions |f_j(t) − f_i(t_s)|≤ δ, f_j≠f_i and $f_{j}\nsucc f_{i}$. The first condition implies that a location of f_j has to be in the 𝜖 neighborhood of f_i. $f_{j}\nsucc f_{i}$ indicates that the location of bifurcation at which f_j deviates from f_i does not appear after f_i(t_s) on the path f_i.

• Tortuosity and partition asymmetry are two important anatomical features of a neuron. Tortuosity refers to the amount of ‘zig-zag’ or bending of a path. Let us take a segment on a path f_i as f_i([t₁, t₂]); 0 ≤ t₁ < t₂ ≤ 1. Let there be m − 1 locations in [t₁, t₂]. The tortuosity of the segment is defined as $\kappa = \frac {{\sum }^{m}_{j=1}||f_{i}(t_{j+1}) - f_{i}(t_{j})||_{2}}{||f_{i}(t_{2}) - f_{i}(t_{1})||_{2}}$ with t_m+ 1 = t₂. Partition asymmetry accounts for how the size of a neuron tree varies within the neuron. We use a variant of caulescence, proposed in Brown et al. (2008), as a measure of tree asymmetry. Caulescence at a bifurcation location is evaluated by way of $\alpha = \frac {|l-r|}{l+r}$, where l is the size of the left tree and r of the right tree of the bifurcation vertex. We define the size of a tree by the number of paths or equivalently the number of dendritic terminals. Note that the quantity (l + r) + 1 is the concurrence value of the bifurcation vertex.

Path alignment and path distance measure, μ

Given an unequal number of samples in a pair of paths, finding the appropriate distance between two paths or open curves is challenging. Due to the resampling bias imposed by a given tracer, in general, a path contains erroneous sampled locations which could alter the path statistics. For example, adding an extra leaf vertex changes the concurrence values of all the locations on a path. Unlike conventional approaches that used different resampling procedures, such as mid-point based resampling, RANSAC sampling, and spectral sampling, we use the help of the branch order as mentioned in Section 3 for suboptimal alignment.

Consider two neurons, G₁ and G₂, with the corresponding path models given as H₁ and H₂, respectively. Let f and g be the two paths that are arbitrarily selected from Γ₁ and Γ₂, respectively. Without loss of generality, let us assume that f and g contain ${\phi _{1}^{b}}$ and ${\phi _{2}^{b}}$, the number of locations from which the current paths bifurcate. In the case, where ${\phi _{1}^{b}} < {\phi _{2}^{b}}$, we append $({\phi _{2}^{b}}-{\phi _{1}^{b}})$ zeros at the end (standard branch order) or at the front (reverse branch order) of a feature vector on f.

Experimental evidence (Bielza et al. 2014) suggests that the importance of a bifurcation location on a path decays as one travels the path from the soma to the dendritic terminal. We utilize this relative importance by way of hierarchy values of the bifurcation locations on a path. Let the sequential order of hierarchy values from the root to the terminal on f be $\xi _{f} = [\xi _{1}, \xi _{2},...,\xi _{{\phi ^{b}_{1}}}]$. Using ξ_f, the k^th importance weight is given by $w_{k} = \frac {1}{\xi _{k}+\epsilon }/{\sum }_{j=1}^{{\phi ^{b}_{1}}}\frac {1}{\xi _{j}+\epsilon }$. 𝜖 is introduced to avoid the indeterminate case. According to the hierarchy, it is obvious that $\xi _{1} < \xi _{2} < ... < \xi _{{\phi ^{b}_{1}}}$. Thus, $w_{1} > w_{2}>...>w_{{\phi ^{b}_{1}}}$. Let us consider a feature υ ∈{b, C, λ, κ,β, α}. The values of the feature on the paths, f and g, are defined by

$$ \begin{array}{@{}rcl@{}} \upsilon^{f} &=& [{\upsilon_{1}^{f}}, {\upsilon_{2}^{f}},...,\upsilon_{{\phi_{1}^{b}}}^{f}, \underbrace{0,.., 0}_{{\phi_{2}^{b}}-{\phi_{1}^{b}}}]\\ \upsilon^{g} &=& [{\upsilon_{1}^{g}}, {\upsilon_{2}^{g}},...,\upsilon_{{\phi_{2}^{b}}}^{g}] \end{array} $$

(3)

The distance between υ^f and υ^g, weighted by the importance factor, is given by

$$ \begin{array}{@{}rcl@{}} d(\upsilon^{fg}) = \sqrt{\frac{1}{{\phi^{b}_{2}}}{\sum}_{k=1}^{{\phi^{b}_{2}}}w_{k}({\upsilon^{f}_{k}}-{\upsilon^{g}_{k}})^{2}} \end{array} $$

(4)

This distance is computed for each υ ∈{b, C, λ,β, κ, α}. The overall distance between the paths f and g can be expressed as a weighted average of individual distances.

$$ \begin{array}{@{}rcl@{}} \mu^{fg} &=& \delta_{1}d(b^{fg})+\delta_{2}d(C^{fg})+\delta_{3}d(\lambda^{fg})\\ && + \delta_{4}d(\kappa^{fg}) + \delta_{5}d({\upbeta}^{fg})+\delta_{6}d(\alpha^{fg}). \end{array} $$

(5)

For simplicity, we take $\delta _{i}=\frac {1}{6}\forall i$ and consider the final distance as the intrinsic distance between the neurons. For classification, we determine δ through optimization using maximizing − interclass − minimizing − intraclass distance strategy (See algorithm 1 and Section “Weight determination”). We term δ as the relative importance of features.

Path assignment and self-similarity

Let the number of paths in H₁ be |Γ₁| = n₁. Similarly, for H₂, this value is |Γ₂| = n₂. Without loss of generality, let us assume n₁ ≤ n₂. Using Eq. 5, the cost matrix of paths between G₁ and G₂ becomes $\mathcal {D}$ ($\mathcal {D}_{ij} = \mu ^{ij},~i\in {\Gamma }_{1},j\in {\Gamma }_{2}$). By applying an analogy for the path assignment as a job assignment problem with n₁ workers and n₂ jobs, we adopt the Munkres algorithm to find the optimal assignment of jobs to the workers from $\mathcal {D}$. In most cases, including inter- and intra-cellular neurons, the job assignment problem is an unbalanced n₁ < n₂. We append (n₂ − n₁) zero rows to $\mathcal {D}$ to serve as dummy workers. ElasticPath2Path (Batabyal and Acton 2018a) employed this technique and resulted in an output of n₁ optimally matched paths between G₁ and G₂. However, this is essentially subgraph matching, which may lead to misclassification while dealing with two structurally similar, but different, cell types. For example, hippocampal CA3 pyramidal and cerebellar Purkinje cells have similar dendritic branch patterns, but significantly different number of paths. To resolve this problem we devise an algorithm 2, by applying Munkres algorithm repeatedly to obtain a full-tree matching. To meet such criterion, the algorithm gives n₂ pair of paths. Let the pair be $(\gamma _{11},\gamma _{21}),...,(\gamma _{1n_{2}},\gamma _{2n_{2}})$, where γ_1i ∈Γ₁ and γ_2j ∈Γ₂. Recall that n₁ < n₂, which implies that some of the γ_1i are repeated while forming the pair. Finally, the distance between G₁ an G₂ is given by

$$ \begin{array}{@{}rcl@{}} \chi_{G_{1}G_{2}} = \sum\limits_{k=1}^{n_{2}}\mu^{\gamma_{1k}\gamma_{2k}} \end{array} $$

(6)

Let $\lfloor \frac {n2}{n1}\rfloor = T$. Then, this procedure to find the correspondence is termed as T −regular matching, which in turn can be thought of T nearly self-similar structures akin to a fractal system. The detailed algorithm is provided in .

There are four modules that are sequentially executed in the algorithm. The first module mathematically deciphers the relatively self-similar anatomy of a larger neuron compared to a smaller one, yielding the number of copies of the smaller one needed to stitch together to approximately obtain the larger one. The routine runs for $\lfloor \frac {n_{2}}{n_{1}}\rfloor$ times, which indicates that each path in neuron 1 (containing n₁ paths), is matched with $\lfloor \frac {n_{2}}{n_{1}}\rfloor$ paths of neuron 2 (containing n₂ paths). Here n₂ > n₁.

The second module runs for the remaining unpaired paths of neuron 2. The assigned correspondence is added to the list of paired paths from the first module. However, not all the pairs are anatomically consistent. This is dictated by an internal constraint of Munkres algorithm, in which the assignment is carried out without replacement. In the Munkres algorithm, if one ‘worker’(a path from neuron 1) is assigned a ‘job’ (a path from neuron 2), then the ’job’ is not available for further assignment. Therefore, if the distance between two paths is significantly large, it demands further inspection whether the pair of paths is morphologically different to each other or the algorithmic constraint induces the large distance value. This motivates us to introduce the third module.

In the third module, we inspect the pair of paths having distances more than a threshold. The threshold is selected based on the skewness, median and standard deviation of the distance values. As mentioned earlier, in order to find the distance of a feature on two paths (4), we append zeros to the path having relatively fewer number of locations than the other. The choice of traversal order dictates to which side the zeros are appended. Notice that more zeros lead to higher distance value between paths, and this happens only when there is significant mismatch in the highest level of hierarchy. This fact can be interpreted from the morphological viewpoint. A path with a large number of bifurcation locations (so, large maximum hierarchy value), called a central path of a neuron, exploits the environment of the neuron extensively when compared to path with fewer number of bifurcations. Unless otherwise required, a path with large hierarchy values should not be compared with a path with much smaller maximum hierarchy value. The highest level of hierarchy values of two paths are given by h₁ and h₂ with h₁ < h₂. We set a criteria that if $|h1-h2| > \frac {max[h1,h2]}{2}$, we do not consider the distance between the pair, and opt for the best match in terms of minimum distance for each path of the pair separately. This is outlined in the reassignment module. The reassigned pairs are added to the list of paired paths serving as the list of correspondence.

Path morphing

Once the correspondence of paths between neurons is established, it is imperative to know the structural similarity between the paths - whether a pair of paths are structurally similar to each other, or the pair is structurally incoherent but the algorithm outputs such a pair due to its internal constraints. This is achieved in two ways: with a visual representation by morphing the paths of one neuron to that of the other using an elastic framework, and by extracting path statistics.

A rooted path of a neuron can be considered as an open curve as shown in Fig. 19 (Batabyal and Acton 2018a; Srivastava et al. 2011). Each location on the path can be considered as a function of a parameter, t ∈ [0, 1]. The square root velocity function (SRVF) that is applied on a location f(t) is defined as $q(t) = \frac {\dot {f}(t)}{\sqrt {||\dot {f}(t)||}}$. For a pair of paths i and j, we obtain q_i and q_j, which assists in retrieving the intermediate deformations as linear combinations of q_i and q_j given by $q_{ij}^{n} = q_{i}(1-n) + nq_{j};~n\in [0,1]$. n denotes the intermediate algorithmic time steps. Although the deformations are exhibited using the 3D coordinates of the locations of a path, the deformations can also be computed in the feature domain. An example of the continuous morphing process between two pyramidal neurons from the secondary visual cortex of the mouse is shown in Fig. 6. The 15 paths of the former neuron merge with 11 paths of the latter upon termination of the morphing process. This implies that more than one path of the first neuron have the same final destination path of the second neuron. It is noted that our algorithm does not consider the costs that are incurred by the merging or splitting of paths during progression. The assessment of such costs requires biophysical measurements of neurons, such as metabolic cost of merging or splitting of branches. Therefore, the cost between paths in Eq. 5 is proportional to the cost of structural disparity instead of biophysical costs.

The prime question is: why do we need to inspect intermediate deformations? Statistical assessment of anat-omical similarities between paths is sufficient to validate the correspondence that is obtained from the Munk-res algorithm. However, to make the correspondence necessary, the intermediate deformations should comply with key cell-type characteristics (Srivastava et al. 2011). So we use the SRVF framework to show the deformations so that any noticeable incoherence can be attributed to the feature selection, distance measurement, or both algorithms even though we might obtain improved classification accuracy in the end.

Weight determination

Let the combined distance vector containing the individual feature distances be D^fg = [d(b^fg) d(C^fg) d(λ^fg) d(κ^fg) d(β^fg) d(α^fg)]^T. The corresponding unknown weight vector is δ = [δ₁,...δ₆]. While comparing two neurons of sizes N and M with N <= M, the distance computation after applying the Munkres algorithm repeatedly will produce M pairs of paths, indicating M such D^fgs. The desired characteristic of each component of δ is positivity. In addition, we enforce $\sum \delta _{i}~=~1$, implying a probability estimate. δ_i thus indicates the relative importance of the feature υ_i.

We adopt the constrained maximizing-betweenclass-minimizing-withinclass distances strategy to find our desired δ. Mathematically,

$$ \begin{array}{@{}rcl@{}} \delta^{opt} = -\underset{\delta}{\arg\min}\frac{1}{2}\delta^{T}\left( \underset{i<j}{\sum\limits^{S}_{i,j=1}}\sum\limits_{k=1}^{N_{i}}\sum\limits_{l=1}^{N_{j}}\sum\limits_{z=1}^{M_{kl}}D^{z}(D^{z})^{T}\right)\delta \end{array} $$

$$ \begin{array}{@{}rcl@{}} && + \delta^{T}\left( \sum\limits_{i=1}^{S}\tau_{i}\underset{k\neq l}{\sum\limits^{N_{i}}_{k,l=1}}\sum\limits_{z}^{M_{kl}}D^{z}(D^{z})^{T}\right)\delta \\ && - \omega_{1}log\delta + \omega_{2}\left( \sum\limits_{i=1}^{6}\delta_{i} - 1\right) \end{array} $$

(7)

The first term in the above equation encompasses all the distances between neurons from pairwise classes. The second term encodes the intraclass distances, implying the distances between neurons for each class. The third term enforces positivity of each weight δ_i. This is a logarithmic barrier penalty term that restricts the evolution of δ at intermediate iterations to the region where $\delta > \bar {0}$. The last term accounts for the probabilistic interpretation of δ. S is the number of classes.

Equation 7 is solved by using gradient descent. The equation and its derivative can be simply written as,

$$ \begin{array}{@{}rcl@{}} L(\delta) &=& -\frac{\delta^{T}{\Pi}\delta}{2} + \delta^{T}\sum\limits_{i=1}^{S}\frac{\tau_{i}{\Pi}_{i}}{2}\delta-\omega_{1}log\delta+\omega_{2}\big(\delta\boldsymbol{1}-1\big)\\ \frac{dL}{d\delta} &=& -{\Pi}\delta +\sum\limits_{i=1}^{S}\tau_{i}{\Pi}_{i}\delta-\frac{\omega_{1}}{\delta}+\omega_{2} \end{array} $$

(8)

We use this derivative term in the following algorithm 1 to obtain optimal δ.

Distance between neurons

The algorithm to find distance between a pair of neurons consists of four stages - finding self-similarity (routine-1), remaining path assignment (routine-2), finding pairs with hierarchy mismatch (routine-3) and reassignment of the defective pairs (routine-4).