Automated Reconstruction of Dendritic and Axonal Trees by Global Optimization with Geometric Priors

Abstract

We present a novel probabilistic approach to fully automated delineation of tree structures in noisy 2D images and 3D image stacks. Unlike earlier methods that rely mostly on local evidence, ours builds a set of candidate trees over many different subsets of points likely to belong to the optimal tree and then chooses the best one according to a global objective function that combines image evidence with geometric priors. Since the best tree does not necessarily span all the points, the algorithm is able to eliminate false detections while retaining the correct tree topology. Manually annotated brightfield micrographs, retinal scans and the DIADEM challenge datasets are used to evaluate the performance of our method. We used the DIADEM metric to quantitatively evaluate the topological accuracy of the reconstructions and showed that the use of the geometric regularization yields a substantial improvement.

Appendix: Pseudo-code for the ACO-RTS Algorithm

In this appendix, we describe in more details our ACO-RTS algorithm and supply pseudo-code for it.

Notations and Overview

ACO-RTS is designed to minimize the primary cost function of Eq. 25 and uses the auxiliary cost function of Eq. 26 during tree construction. To this end, it maintains three sets of trees, namely the best-so-far solutions T _bs , the restart-best solutions T _rb , and the iteration-best solutions T _ib . Each one of these sets contains a group of trees with cardinalities {1, ..., k _limit}, where k _limit is the maximum cardinality of the trees to be constructed. Its value is updated at each iteration of the algorithm so as to focus the search on the regions of most likely cardinalities and to discard very high ones.

Given a directed graph G = (V,E), let H be the set of all directed paths (e _us ,e _st ) containing exactly two consecutive edges. The pheromone model ${\cal P}$, first introduced in section “Optimizing the Image-Based Term”, is constructed by assigning a pheromone value τ _ust to each one of these edge pairs and τ _rv to each edge emanating from the root vertex x _r . More formally,

$$ \begin{array}{rll} \label{eq:PheromoneModel} \mathcal{P} &=& \{ \tau_{ust} \in [\tau_{\min}, \; \tau_{\max}] \mid e_{st} \in E, e_{us} \in E \}\\ &\cup& \{ \tau_{rv} \in [\tau_{\min}, \; \tau_{\max}] \mid e_{rv} \in E_{r} \}, \end{array} $$

(27)

where τ _min  ∈ ℝ and τ _max  ∈ ℝ are lower and upper limits for the pheromone values and E _r is the set of edges emanating from x _r . In this work, τ _min and τ _max are set to 0.01 and 0.99, respectively. In effect, this model amounts to introducing a likelihood distribution over edge pairs, which improves convergence by resolving conflicts at crossovers.

Figure 15 shows a simplified flowchart of the algorithm. At each iteration, n _a directed trees are probabilistically constructed by combining the pheromone information and the auxiliary cost of Eq. 25. The tree cardinality is bounded by the variable k _limit, whose value is dynamically updated at run-time. Tree construction always starts at the root vertex x _r , with edges being probabilistically selected and iteratively added to the tree. Edge pairs whose contribution to the auxiliary cost is small are given higher priority and care is taken not to violate the branching factor limit.

Fig. 15

ACO-RTS algorithm flowchart

We rely on dynamic programming (Blum 2007) to find the best l-cardinality tree for all l from 1 to k _limit in each one of the n _a trees that have been built. For each cardinality, we select from the resulting n _a trees the one that minimizes the auxiliary objective function and store it in T _ib . Each l-cardinality tree in T _rb or T _bs is then updated by the l-cardinality tree in T _ib , if the latter has a lower auxiliary cost. The best cardinality k ^* for the next iteration is then taken to be the one that minimizes the primary objective function of Eq. 25 among the trees in T _bs . The cardinality limit k _limit is taken to be the average of k ^* and |V| − 1, the maximum possible cardinality of a tree. The pheromone values are updated by using the k ^*-cardinality trees in T _ib , T _rb and T _bs so that values corresponding to edge pairs belonging to these k ^*-cardinality trees are increased while others are decreased. Finally, when the algorithm has run for a preset amount of time, it returns the k ^*-cardinality tree in T _bs .

Pseudo Code

Figure 16 depicts the pseudo-code that implements the approach outlined above. It includes the following functions:

• ConstructDirectedTree(${\cal P}, k_{\rm limit}, x_r$): This function constructs a k_limit-cardinality tree T = (V _T ,E _T ) with vertices $V_T \subseteq V$, edges $E_T \subseteq E$, and pairs of consecutive edges $H_T \subseteq H$. The construction starts from the given root vertex x _r , that is, initially T = ({x _r }, ∅ ). At each step, a tree is stochastically selected from a neighborhood of the current solution. This neighborhood is generated by adding an edge and its target vertex to the tree under construction such that the tree property is maintained (i.e., no cycles are formed) and no bifurcation with a branching factor greater than a predefined threshold b _th is introduced. The set of candidate edges is generated as follows.

Fig. 16

ACO-RTS: pseudo-code for ant colony optimization for the reconstruction of tree-like structures

As a convention, let the source vertex of an edge e _st  ∈ E be denoted by x _s and the target vertex by x _t . Let also $N \subseteq V$ be the set of vertices such that N ∩ V _T  = ∅ and for each x _t  ∈ N, there exists at least one edge e _st  ∈ E with x _s  ∈ V _T . Moreover, let $E^t \subseteq E$ denote the set of edges that can connect the vertex x _t to T such that the following conditions hold:

• 1. For each $e_{st} \in E^t$, $deg^+(x_s) < b_{th}$, where $deg^+(x_s)$ denotes the out-degree of vertex x _s in tree T.

• 2. There does not exist an edge e _sl  ∈ E _T such that x _s and x _l are co-located vertices as defined in section “Sampling the Image”.

Then, the set of candidate edges is taken to be

$$ \label{eq:CandidateEdges} \mathcal{C} = \{ \underset{e_{st} \in E^t}{\operatorname{argmin}} \{w(e_{st})\} \mid x_t \in N\}, $$

(28)

where w(e _st ) is the effective weight of an edge e _st , which can be expressed in terms of the cost terms of Eq. 26.

$$ w(e_{st}) = \begin{cases} d_{st} + b_{st} & \text{ , if } x_{s} = x_{r} \;, \\ d_{st} + a_{ust} & \text{ , if } x_{s} \neq x_{r}, e_{us} \in E_T \; . \end{cases} $$

(29)

At each construction step, with a constant probability q ∈ [0,1], the algorithm chooses an edge $e_{st} \in \mathcal{C}$ that deterministically minimizes τ _ust / w(e _st ), where $\tau_{ust} \in {\cal P}$ denotes the pheromone value assigned to the edge pair e _st and e _us  ∈ E _T . Otherwise, with probability 1 − q, $e_{st} \in \mathcal{C}$ is chosen probabilistically in proportion to τ _ust / w(e _st ).

The resulting trees may contain incorrect connections at crossovers. We observed that, while growing the tree from the root vertex, one of the crossing branches usually grows more rapidly than the other and dominates the crossover region by taking the ownership of the continuation edges. This is mainly caused by contrast differences in the original data and local classifier errors, which lead to high-cost edges not being selected to bridge gaps along the branches.

To correct such errors, we introduce an additional neighborhood structure, which we name crossover neighborhood. Two trees are said to be crossover-neighbors if one can be obtained from the other by simply changing possession of continuation edges at two co-located crossover vertices. Figure 17 gives an example of such a neighborhood for a crossover with three continuation edges. After each construction step, we search over this neighborhood of the constructed tree and minimize the primary cost function. Note that, this minimization reduces to the minimization of the sum of the pairwise edge weights in this case, since all the trees in the neighborhood have the same set of edges, and hence the unary edge costs are common.

• DynamicTree(T,k_limit): This runs a dynamic programming algorithm (Blum 2007), which we extended to handle directed graphs. For each l = 1,...,k_limit, it returns the best l-cardinality tree rooted at x _r . The quality of a tree is determined by the auxiliary cost function.

• Update(T _a , T _b , k_limit): This function updates the set of trees T _a by the set T _b for each cardinality. That is, for l = 1,..., k_limit, if F′ _g (T _a [l]) > F′ _g (T _b [l]), then T _a [l] = T _b [l], where T _a [l] and T _b [l] denote the l-cardinality trees in T _a and T _b , respectively.

• PheromoneUpdate(cf, bs_update, ${\cal P}$, $T_{ib}[k^*]$, $T_{rb}[k^*]$, $T_{bs}[k^*]$): The pheromone values are updated using the three solutions $T_{ib}[k^*]$, $T_{rb}[k^*]$ and $T_{bs}[k^*]$ that have the same k^* cardinality. Their influence on the update depends on the state of convergence of the algorithm. This procedure is the same as the one described in Blum and Blesa (2005) for the k-cardinality tree problem, except that the pheromone values are assigned to edge pairs instead of individual edges. We refer the interested reader to our earlier publication for further details.

• ComputeConvergenceFactor(${\cal P}, T_{rb}[k^*], k^*$): In this procedure, the convergence factor of the algorithm is computed based on the convergence of the restart-best tree with cardinality k^*. We take it to be

  $$ \label{eq:ConvergenceFactor} cf = \frac{ {\displaystyle \sum\limits_{h \in {\cal P}_{T_{rb}[k^*]}}}{\tau_h} }{ k^* . \tau_{\max} } \; , $$

  (30)

  where ${\cal P}_{T_{rb}[k^*]}$ denotes the set of pheromone values of the edges and edge pairs included in $T_{rb}[k^*]$ and τ _max is the upper limit for the pheromone values. Note that, the convergence factor can only assume values in the interval [0 1].

Fig. 17

Crossover neighborhood for a junction with two branches and three continuation edges. The green circles are vertices. The blue dotted lines represent one of the crossing branches and the red solid lines represent the other one