Fast Cross Correlation for Limited Angle Tomographic Data

Abstract

The cross-correlation is a fundamental operation in signal processing, as it is a measure of similarity and a tool to find translations between signals. Its implementation in Fourier space is used for large datasets, as it is faster than the one in real space, however, it does not consider any special properties which signals may have, as is the case of Limited Angle Tomography. The Fourier space of limited angle tomograms, which are reconstructed from a reduced number of projections, has a large number of empty values. As a consequence, most operations needed to calculate the cross-correlation are executed on empty data. To address this issue, we propose the projected Cross Correlation (pCC) method, which calculates the cross-correlation between a reference and a limited angle tomogram more efficiently. To reduce the number of operations, pCC follows a project, cross-correlate, reconstruct process, instead of the typical reconstruct, cross-correlate process. Both methods are equivalent, but the proposed one has lower computational complexity and provides significant speedup for larger tomograms, as we confirm with our experiments. Additionally, we propose the usage of a $l_1$ penalty on the cross-correlation to improve its sensitivity and its robustness to noise. Our experimental results show that the improvements are achieved with no significant additional computational cost.

1 Introduction

Cross-correlation is a fundamental and widely used operation in signal processing. It can be used as a measure of similarity between signals [14], and for estimation of the translation between them. Due to computational efficiency it is calculated in Fourier space, and, as a general method, it does not take into consideration any special property the signals may have. This is the case, for example, of limited angle tomographic data, where the Fourier space is sparse.

Limited angle tomography is encountered when the data acquisition is restricted to a reduced number of views. For example, in cryo-electron tomography (cryoET) for structural biology [6] the samples are very sensitive to radiation damage. This limits the acquisition to only 41 projections with very low SNR, filling, in some cases, 20% of the Fourier space or less. This field is the inspiration for the present work, as a typical workflow in cryoET includes the registration of $1 000$ to $50 000$ tomograms to one or several references by cross-correlation. This alignment process takes multiple days on modern hardware.

In this paper we propose a new method to calculate the cross-correlation between a reference and a set of projections, which reduces the number of operations by using the sparsity of the Fourier space of limited angle tomograms (Fig. 1). The method, called projected Cross Correlation (pCC), has lower computational complexity and provides significant speedup for larger tomograms. Furthermore, we use the $l_1$ penalty function to the cross-correlation to improve its robustness to noise. The lower complexity of the proposed method and its improved robustness makes it potentially useful for cryoET data processing.

Fig. 1.

Scheme of the computation of 3D cross correlation between a set of 2D projections and a 3D reference. (a) The typical computation follows a reconstruct then cross-correlate process. (b) The proposed projected Cross Correlation (pCC) follows a project, cross-correlate and reconstruct process

1.1 Related Work

The main proposals to speed up the computation of the cross-correlation are the precalculation of look-up tables [9] and split the calculations into blocks [11]. These approaches work for any kind of signals and do not take into account any type of property those signals may have.

Similar to our approach, the Projection-based Volume Alignment (PBVA) [15] method uses the project, cross-correlate approach, but it estimates the peak of cross-correlation instead of calculating it. PBVA has similar computational complexity to pCC, and it can be seen as a special case of it: it approximates the peak value instead of reconstructing the cross-correlation. It is a faster procedure, but it is more sensitive to noise and it does not find the translation between signals.

2 Theoretical Context

The Fourier central slice theorem is the mathematical foundation for transmission tomography. It defines a relationship between a $N^{m}$-dimensional signal and a $N^{m-1}$-dimensional projection of it.

2.1 Fourier Central Slice Theorem

Let s and S be a signal and its Fourier transform, and $\mathbf {F}$, the discrete Fourier transform matrix, such that $S = \mathbf {F}s$ and $s = \mathbf {F}^{-1}S$. Then, given the matrix $\mathbf {E}_{\theta _i}$, that extracts a slice crossing the center of coordinates in the angle $\theta _i$, the projection $p_{\theta _i}$ is defined as [7]:

$$\begin{aligned} p_{\theta _i} = \mathbf {F}^{-1}\mathbf {E}_{\theta _i}\mathbf {F}s. \end{aligned}$$

(1)

Equation (1) can be interpreted as: The projection $p_{\theta _i}$ of s in the direction $\theta _i$ is the inverse Fourier transform of the slice through S in the corresponding direction

To recover s we have to write $\mathbf {E}_{\theta _i}$ as the product of the rotation $\mathbf {R}_{\theta _i}$ and the binary mask $\mathbf {M}_{\theta _i}$. Then, by using the element-wise multiplication operator $\odot$, Eq. (1) takes the following form:

$$\begin{aligned} p_{\theta _i} = \mathbf {F}^{-1}\mathbf {R}_{\theta _i}\mathbf {M}_{\theta _i} \odot (S). \end{aligned}$$

(2)

We must note that the rotation matrix $\mathbf {R}_{\theta _i}$ is invertible while the mask matrix $\mathbf {M}_{\theta _i}$ is not. Then, by using $P_{\theta _i} = \mathbf {F}p_{\theta _i}$ we have:

$$\begin{aligned} \mathbf {R}_{\theta _i}^{-1}P_{\theta _i} = \mathbf {M}_{\theta _i} \odot ( S ). \end{aligned}$$

(3)

This last equation defines an insertion-like operation, as the projection $P_{\theta _i}$ fills the slice of S defined by the matrix $\mathbf {M}_{\theta _i}$. To recover S we need multiple projections in different orientations such that $\mathbf {M}_{\theta _i} \ne 0$ can be inverted [3].

2.2 Analytic Tomogram Reconstruction

Let $\tilde{S}$ be the reconstruction of S from a given set of projections $p_{\theta _i}$, and let $\mathbf {W}_\varTheta$ be an unknown weighting matrix. Then, $\tilde{S}$ can be calculated using the following equation:

$$\begin{aligned} \tilde{S} = \mathbf {W}_\varTheta \odot \left( \sum _i \mathbf {R}_{\theta _i}^{-1}P_{\theta _i} \right) \end{aligned}$$

(4)

Using Eq. (3) and by defining $\mathbf {M}_\varTheta = \sum _i \mathbf {M}_{\theta _i}$, we can rewrite the last equation as $\tilde{S} = \mathbf {W}_\varTheta \odot \left( \mathbf {M}_\varTheta \odot S \right)$. Finally, to obtain the proper reconstruction $\tilde{S} = S$, the matrix $\mathbf {W}_\varTheta$ must be:

$$\begin{aligned} \mathbf {W}_\varTheta = \mathbf {M}^{-1}_\varTheta . \end{aligned}$$

(5)

If the Fourier space of S is fully and uniformly sampled, matrix $\mathbf {W}_\varTheta$ takes the form $\mathbf {W}_\varTheta = 1/|w|$. This form is called radial filter and it is used in the Weighted Back Projection reconstruction algorithm [12]. If the Fourier space is fully but non-uniformly sampled, we can use Eq. (5) to find $\mathbf {W}_\varTheta$ [13].

If the Fourier space is not fully sampled some values of $\mathbf {M}_\varTheta$ will be 0, making the reconstruction an ill-posed inverse problem, as $\mathbf {M}_\varTheta$ is non-invertible matrix. This is the case of limited angled tomography, where we have few projections of the signal we want to reconstruct and, therefore, a sparse Fourier transform of the reconstructed tomogram (Fig. 2). The approaches to address this issue use a priori information of the signal, and solve the ill-posed system by imposing a regularization factor [1, 4, 5, 8, 10]

Fig. 2.

(a), (b) Shepp–Logan phantom and its power spectrum. (b), (c) limited angle tomogram from 25 projections (from $-60^{\circ }$ to $60^{\circ }$, $5^{\circ }$ step), and its power spectrum.

In our case we do not need the reconstruction of the limited angle tomogram to calculate its cross-correlation against a reference: the proposed projected Cross Correlation method calculates it directly. Nevertheless, a reconstruction algorithm is used in the final stage of the pCC and it can use a priori information to improve its results. If two signals are similar, the cross-correlation ressembles a delta dirac function, which is sparse in real space. In the following section we describe the pCC method and how to use the $l_1$ regularization to add the real space sparsity constrain to the cross-correlation.

3 Fast Correlation for Limited Angle Tomography

In this section we find a relationship between the cross-correlation of a reference signal s and a set of projections $p_{\theta _i}$. We start by stating the definition of cross-correlation $\varphi$: let $\overline{\mathbf {F}s}$ be the complex conjugate of $\mathbf {F}s$, and given two signals s and g, then:

$$\begin{aligned} \varphi _{sg} = \mathbf {F}^{-1}\left( \mathbf {F}s \odot \overline{\mathbf {F}g} \right) . \end{aligned}$$

(6)

The relationship we describe in this section can be applied, in a similar way, to the normalized cross-correlation (NCC), zero-normalized cross-correlation (ZNCC) and the phase correlation. But, for the sake of simplicity, and without losing generality, we use this Eq. (6) to represent all the family of cross-correlation functions.

3.1 Cross Correlation for Limited Angle Tomograms

Given s, a signal used as reference, and $\tilde{p}$, a signal reconstructed from the set of projections $p_{\theta _i}$, we use Eq. (6) to calculate the cross-correlation between them:

$$\begin{aligned} \varphi _{\tilde{p}s} = \mathbf {F}^{-1}\left( \mathbf {F}\tilde{p} \odot \overline{\mathbf {F}s} \right) . \end{aligned}$$

(7)

The projection $\varphi _{\tilde{p}s}^{(\theta _i)}$ is found using Eq. (2):

$$\begin{aligned} \varphi _{\tilde{p}s}^{(\theta _i)} = \mathbf {F}^{-1}\mathbf {R}_{\theta _i}\mathbf {M}_{\theta _i} \odot \left( \mathbf {F}\tilde{p} \odot \overline{\mathbf {F}s} \right) . \end{aligned}$$

(8)

As the binary mask $M_{\theta _i}$ is idempotent, we expand the last equation into:

$$\begin{aligned} \varphi _{\tilde{p}s}^{(\theta _i)} = \mathbf {F}^{-1}\mathbf {R}_{\theta _i}\left( \mathbf {M}_{\theta _i} \odot \mathbf {M}_{\theta _i}\right) \odot \left( \mathbf {F}\tilde{p} \odot \overline{\mathbf {F}s} \right) , \end{aligned}$$

(9)

and by rearranging the element-wise multiplications and by using Eq. (3):

$$\begin{aligned} \varphi _{\tilde{p}s}^{(\theta _i)} = \mathbf {F}^{-1}\mathbf {R}_{\theta _i} ( \underbrace{(\mathbf {M}_{\theta _i}\odot \mathbf {F}\tilde{p})}_{\mathbf {R}_{\theta _i}^{-1}\mathbf {F}p_{\theta _i}} \odot \underbrace{(\mathbf {M}_{\theta _i}\odot \overline{\mathbf {F}s})}_{\mathbf {R}_{\theta _i}^{-1}\overline{\mathbf {F}s_{\theta _i}}} ). \end{aligned}$$

(10)

Finally, we factorize $\mathbf {R}^{-1}_{\theta _i}$:

$$\begin{aligned} \varphi _{\tilde{p}s}^{(\theta _i)} = \mathbf {F}^{-1}\mathbf {R}_{\theta _i} ( \mathbf {R}_{\theta _i}^{-1}(\mathbf {F}p_{\theta _i} \odot \overline{\mathbf {F}s_{\theta _i}}) ), \end{aligned}$$

(11)

and use the definition of cross-correlation to obtain:

$$\begin{aligned} \varphi _{\tilde{p}s}^{(\theta _i)} = \underbrace{ \mathbf {F}^{-1} \left( \mathbf {F}p_{\theta _i} \odot \overline{\mathbf {F}s_{\theta _i}} \right) }_{\varphi _{p_{\theta _i}s_{\theta _i}}}. \end{aligned}$$

(12)

This results in the intuitive property: A projection of the cross-correlation of two signals is the cross-correlation of the projection of those signals in the same slice. Furthermore, we can calculate the cross-correlation of two signals from their projections by reconstructing the cross-correlation of those projections:

$$\begin{aligned} \mathbf {F}\varphi _{\tilde{p}s} = \mathbf {W}_\varTheta \odot \left( \sum _i \mathbf {R}_{\theta _i}^{-1}\left( P_{\theta _i} \odot \overline{\mathbf {E}_{\theta _i}S} \right) \right) . \end{aligned}$$

(13)

This equation is the core of the projected Cross Correlation (pCC) method, which calculates the cross-correlation between a reference and a set of projections (Fig. 1) and it is described in Algorithm 1. We must note here that we reconstruct a cross-correlation function instead of a signal, allowing us to impose constrains to the reconstruction process, like sparsity in real space.

Computational Complexity. Given s, a signal with $N^m$ elements, and a set of k projections $p_{\theta _i}$, with $N^{m-1}$ elements each, the computational cost of calculating the cross-correlation using Eq. (6) is:

$$\begin{aligned} N_{op}(\varphi _{\tilde{p}s}) = \underbrace{O(kN^m)}_{reconstruction} + \underbrace{3O(N^m log N)}_{FFTs} + \underbrace{O(N^m)}_{\odot }. \end{aligned}$$

(14)

In the other hand, the computational cost of calculating the cross-correlation using Eq. (13) is:

$$\begin{aligned} N_{op}(pCC(p_{\theta _i},s)) = \underbrace{3O(kN^{m-1} log N)}_{FFTs} + \underbrace{O(kN^{m-1})}_{\odot } + \underbrace{O(kN^m)}_{reconstruction}. \end{aligned}$$

(15)

If $k<N$, then $N_{op}(\varphi _{\tilde{p}s}) < N_{op}(pCC(p_{\theta _i},s))$: for limited angle tomography the calculation of pCC is faster than the calculation of $\varphi _{\tilde{p}s}$.

3.2 Fast Implementations

The computational performance of the implementation of Algorithm 1 is dominated by the reconstruction algorithm used. We can improve this performance by using an approximation of Eq. (13) or by reducing the size of the reconstruction. The first approach is called Projection-Based Volume Alignment (PBVA) and it will be used as a reference algorithm. We call the second one pCC on a Region Of Interest (pCC_ROI), and we add the sparsity constrain to the pCC_ROI to obtain the Regularizated pCC on a ROI (pCC_ROI_Reg). These three algorithm will be explained next.

Projection-Based Volume Alignment (PBVA). The cross-correlation is used to assess the similarity between two signals by finding its maximum value. We can reduce the number of operations if we focus on finding the peak of cross-correlation instead of calculating the whole cross-correlation function. If we explore Eq. (13), and by noting that $0 \le \mathbf {W}_\varTheta \le 1$:

$$\begin{aligned} \mathbf {F}\varphi _{\tilde{p}s} = \mathbf {W}_\varTheta \odot \left( \sum _i\mathbf {R}_{\theta _i}^{-1}(P_{\theta _i}\odot \overline{\mathbf {E}_{\theta _i}S})\right) \le \sum _i\mathbf {R}_{\theta _i}^{-1}(P_{\theta _i}\odot \overline{\mathbf {E}_{\theta _i}S}). \end{aligned}$$

(16)

This equation defines an upper bound for the cross-correlation, and it is used in the PBVA method [15] as a fast estimator of $max(\varphi _{\tilde{p}s})$:

$$\begin{aligned} \tilde{m}_\varphi = \max (\varphi _{\tilde{p}s}) \le PVBA(\tilde{p},s) = \sum _i max\left( \varphi _{p_{\theta _i}s_{\theta _i}}\right) . \end{aligned}$$

(17)

Calculation of $PVBA(\tilde{p},s)$ is fast, as it does not need the final reconstruction, but it is not as robust as the cross-correlation, and it cannot calculate the translation between the signals. PBVA is described in Algorithm 2.

3.3 Localized Reconstruction

The cross-correlation is used to find a spatio-temporal translation between two signals, which is done by finding $t_\varphi$, the position of the peak of cross-correlation. Typically, the maximum translation is constrained to a known area. Let $M_{L}$ be a matrix with L non-zero elements that defines the area of search for the peak of cross-correlation. We can find $t_\varphi$ by solving:

$$\begin{aligned} t_\varphi = \mathop {\mathrm {arg max}}\limits \left\{ M_{L} \odot ( \varphi _{sg} ) \right\} . \end{aligned}$$

(18)

From this result, we can devise an algorithm that only reconstructs the cross-correlation in the area defined by $M_L$. The number of operations and memory requirements are reduced, as we only need to find $m_\varphi = \max (\varphi _{\tilde{p} s})$ and $t_\varphi$. To achieve this, we propose pCC_ROI (Algorithm 3), which reconstructs the cross-correlation on a Region of Interest (ROI) and returns $m_\varphi$ and $t_\varphi$.

Each element of the cross-correlation can be calculated independently, making this algorithm suitable for implementations on GPUs. The computational complexity of the pCC_ROI is:

$$\begin{aligned} N_{op}(pCC\_ROI(p_{\theta _i},s)) = \underbrace{3O(kN^{m-1} log N)}_{FFTs} + \underbrace{O(kN^{m-1})}_{\odot } + \underbrace{O(kL)}_{reconstruction}. \end{aligned}$$

(19)

3.4 Localized Reconstruction with Regularization

As we mentioned before, the cross-correlation between similar signals ressembles a delta dirac function, which has the property of being sparse in real space. We use said property to improve the cross-correlation peak and to make it more robust to noise. The sparsity condition is imposed in the following optimization problem: Let $\varphi _{\varTheta }$ be a collection of projections $\varphi ^{(\theta _i)}$, then the estimated $\tilde{\varphi }$ is obtained by: $\mathop {\mathrm {arg min}}\limits _{\tilde{\varphi }} || Proj(\tilde{\varphi },\varTheta ) - \varphi _{\varTheta } ||_2^2 + \lambda || \tilde{\varphi } ||_1.$ Solving this requires the reconstruction of $\tilde{\varphi }$ multiple times. To save computation time we work with the reconstructed cross-correlation $\varphi$, and impose the sparse condition to it. The new optimization problem to solve is:

$$\begin{aligned} \mathop {\mathrm {arg min}}\limits _{\tilde{\varphi }} || \tilde{\varphi } - \varphi ||_2^2 + \lambda || \tilde{\varphi } ||_1, \end{aligned}$$

(20)

and we use the Alternating Direction Method of Multipliers (ADMM) algorithm to solve it [2]. Let $\mathcal {S}_{\lambda }()$ be the shrinkage function, $\lambda$ be the regularization parameter, and $\rho$ be the fidelity coefficient; then the formulas for the j-th ADMM iteration are:

$$\begin{aligned} x^{j+1} = \frac{\varphi + \rho (z^j-u^j)}{1+\rho }, \end{aligned}$$

(21)

$$\begin{aligned} z^{j+1} = \mathcal {S}_{\lambda }( x^{j+1} + u^j ) = \left\{ \begin{array}{ll} (x^{j+1} + u^j) - \lambda &{}\,, (x^{j+1} + u^j) > \lambda \\ 0 &{}\,, |x^{j+1} + u^j| \le \lambda \\ (x^{j+1} + u^j) + \lambda &{}\, , (x^{j+1} + u^j) < \lambda , \end{array} \right. \end{aligned}$$

(22)

$$\begin{aligned} u^{j+1} = u^ j + x^{j+1} - z^{j+1}. \end{aligned}$$

(23)

Finally, we devise the Regularizated pCC_ROI algorithm (Algorithm 4).

As in the previous algorithm, the reconstruction and also the ADMM iteration can be implemented independently for each value of the cross-correlation, making it also suitable for implementations on GPUs. The computational complexity is also similar to the previous one, with the only addition of the J iterations of the ADMM algorithm:

$$ N_{op}(pCC\_ROI\_reg(p_{\theta _i},s)) = N_{op}(pCC\_ROI(p_{\theta _i},s)) + \underbrace{O(JL)}_{ADMM} $$

4 Experimental Results

We performed three computational experiments to test the performance of the proposed algorithms in terms of correctness, robustness against noise, and execution time. We used the Shepp–Logan phantom of various sizes as the test image, and we projected it using three schemes that are similar to the ones used in cryoET: 41 projections, from $-60^{\circ }$ to $60^{\circ }$ with $3^{\circ }$ step; 25 projections, from $-60^{\circ }$ to $60^{\circ }$ with $5^{\circ }$ step; and 17 projections, from $-64^{\circ }$ to $64^{\circ }$ with $8^{\circ }$ step.

The algorithms were implemented in Matlab and in C++, in the form of MEX files, and executed on an Intel Core i7-8550U CPU (8 cores, 1.8 GHz), with Linux as operative system and 16 GB of RAM memory.

4.1 Cross Correlation and Noise

For this experiment we used a $256 \times 256$ pixels Shepp–Logan phantom and we generate two sets of projections from it. The first one adds a shift to the reference image (Fig. 3(A)), and the second one adds an extra rotation (Fig. 3(B)). Both sets were projected using the scheme with $5^{\circ }$ step ($K=25$), and, we added gaussian noise to both sets (Fig. 3(C) and 3(D)). Equation 6 was used to calculate the cross-correlation between the reference and the transformed images, and between the reference and the reconstructions. Algorithms 1, 3, and 4 were used to calculate the cross-correlation between the reference and the projections sets. We do not include PBVA in this test as it does not calculate the cross-correlation.

The results show that the cross-correlations calculated using the pCC and the pCC_ROI algorithms are similar to the ones obtained using Eq. 6 for all the projections sets. For the aligned ones, the pCC_ROI_REG algorithm gives a well defined peak. When the projections are not aligned, pCC_ROI_REG scales the intensity of the cross-correlation only. This is an expected behavior, as the cross-correlation of different signals is no longer sparse. Nevertheless, the maximum value of cross-correlation of the not aligned data is always lower than the aligned one, as we will show in the next test.

Fig. 3.

On the left: setup of the experiment, showing the projections sets used (A,B,C and D). On the right: reconstructed images, and the cross-correlation of the reference against the transformed images (CC (GT)), against the reconstructions (CC), and between the reference and the projections (using Algorithms 1, 3, and 4).

4.2 Alignment

In this experiment we find the rotation angle between the reference and a set of projections [6]. The reference is rotated multiple times and cross-correlated with the projections, we pick the angle that gives the highest peak of cross-correlation. For the projected Cross Correlation-based algorithms the rotation is embedded into the Extraction part of the algorithms. We also add two levels of noise, and set the ADMM parameters for the pCC_ROI_REG algorithm to $\rho =0.02$, $\lambda =0.05$, and $J=20$. And additional test with $\lambda =0.1$ was performed.

The result (Fig. 4) shows that most algorithms find the angles correctly, as PBVA fails to do it when the noise level is high. The cross-correlation values found using CC, pCC and pCC_ROI, are almost identical, showing that the algorithms are equivalent. PBVA behaves as an upper bound with no sharp peaks where it reaches its maximum value. In contrast, the regularization used in pCC_ROI_Reg makes sharper peaks of cross-correlation, but this depends on the level of noise and the value of $\lambda$.

4.3 Running Time

To measure the running time, we tested different sizes for the reference, from $100\times 100$ pixels to $800\times 800$ pixels, with increments of 100 pixels. The ROI for the reconstructions were circles with radius of 10, and 20 pixels.

Fig. 4.

Angular search: cross-orrelation values for different angles between a reference and a set of rojections. (a), (b), and (c) uses a 41 projections scheme, while (d), (e) and (f), uses 17 projections. The SNR, in dB, of the projections are $-35.39, -54.15, -60.15, -29.80, -50.75$ and $-56.36$, for (a) to (f) respectively.

The results shows a clear quadratic behavior for the traditional cross correlation calculation (Eq. 6), and also the pCC, while the other algorithms behave in an almost linear manner (Fig. 5). We can also note that there is almost no difference in running time between pCC_ROI and pCC_ROI_Reg. This is because the ADMM iterations are fast and simple, making the use of the regularized version of the cross-correlation negligibly fast. Finally, we can see a difference between PBVA and pCC_ROI. This difference is due to the reconstruction step, and it was expected from the computational complexity analysis of the algorithms.

Fig. 5.

Comparison of the running time of alignment of projections against a template. Eq. (6) and the full pCC behave quadratically while the other algorithms, linearly.

5 Conclusions

In the present document we propose a method that exploits the sparsity inherent to the limited angle tomography to calculate the cross-correlation with lower computational complexity. For images, the initial quadratic complexity is reduced to almost a linear one; and for volumes, the cubic complexity could be reduced to a quadratic one. This is achieved by using a localized reconstruction algorithm to reduce the number of operations. Additionally, we added a regularization constrain to the calculation of the cross-correlation at almost no computational cost, which enhances its robustness to noise.

The proposed algorithms could be used to speed up computationally demanding tasks that involves the usage of cross correlation. This tasks can be image or volume alignment, classification or segmentation, and it can be used in areas like cryo-electron tomography (cryoET) or computed tomography (CT).