Mapping molecular statistics with balanced super-resolution optical fluctuation imaging (bSOFI)

Abstract

Background

Super-resolution optical fluctuation imaging (SOFI) achieves 3D super-resolution by computing temporal cumulants or spatio-temporal cross-cumulants of stochastically blinking fluorophores. In contrast to localization microscopy, SOFI is compatible with weakly emitting fluorophores and a wide range of blinking conditions. The main drawback of SOFI is the nonlinear response to brightness and blinking heterogeneities in the sample, which limits the use of higher cumulant orders for improving the resolution.

Balanced super-resolution optical fluctuation imaging (bSOFI) analyses several cumulant orders for extracting molecular parameter maps, such as molecular state lifetimes, concentration and brightness distributions of fluorophores within biological samples. Moreover, the estimated blinking statistics are used to balance the image contrast, i.e. linearize the brightness and blinking response and to obtain a resolution improving linearly with the cumulant order.

Results

Using a widefield total-internal-reflection (TIR) fluorescence microscope, we acquired image sequences of fluorescently labelled microtubules in fixed HeLa cells. We demonstrate an up to five-fold resolution improvement as compared to the diffraction-limited image, despite low single-frame signal-to-noise ratios. Due to the TIR illumination, the intensity profile in the sample decreases exponentially along the optical axis, which is reported by the estimated spatial distributions of the molecular brightness as well as the blinking on-ratio. Therefore, TIR-bSOFI also encodes depth information through these parameter maps.

Conclusions

bSOFI is an extended version of SOFI that cancels the nonlinear response to brightness and blinking heterogeneities. The obtained balanced image contrast significantly enhances the visual perception of super-resolution based on higher-order cumulants and thereby facilitates the access to higher resolutions. Furthermore, bSOFI provides microenvironment-related molecular parameter maps and paves the way for functional super-resolution microscopy based on stochastic switching.

Background

The spatial resolution in classical optical microscopes is limited by diffraction to about half the wavelength of light. During the last two decades, several super-resolution concepts have been developed. Based on the on-off-switching of fluorescent probes, these concepts overcome the diffraction limit by up to an order of magnitude (Huang et al. 2009). A straightforward method consists of digitally post-processing an image sequence of stochastically blinking emitters acquired with a standard wide-field fluorescence microscope. Densely packed single fluorophores can be distinguished in time by using high-precision localization algorithms, used for instance in photo-activation localization microscopy (PALM) (Betzig et al. 2006; Hess et al. 2006) and stochastic optical reconstruction microscopy (STORM) (Heilemann et al. 2008; Rust et al. 2006), or by analysing the statistics of the temporal fluctuations as exploited in super-resolution optical fluctuation imaging (SOFI) (Dertinger et al. 2009; Dertinger et al. 2010). SOFI is based on a pixel-wise auto- or cross-cumulant analysis, which yields a resolution enhancement growing with the cumulant order in all three dimensions (Dertinger et al. 2009). Uncorrelated noise, stationary background, as well as out-of-focus light are greatly reduced by the cumulants analysis. While PALM and STORM are commonly based on a frame-by-frame analysis of images of individual fluorophores, SOFI processes the entire image sequence at once and therefore presents significant advantages in terms of the number of required photons per fluorophore and image, as well as in acquisition time (Geissbuehler et al. 2011). Localization microscopy requires a meta-stable dark state for imaging individual fluorophores (van de Linde et al. 2010). In contrast, SOFI relies solely on stochastic, reversible and temporally resolvable fluorescence fluctuations almost regardless of the on-off duty cycle (Geissbuehler et al. 2011). The main drawback of SOFI is the amplification of heterogeneities in molecular brightness and blinking statistics which limits the use of higher-order cumulants and therefore resolution. In this article, we revisited the original SOFI concept and propose a reformulation called balanced super-resolution optical fluctuation imaging (bSOFI), which in addition to improving structural details opens a new functional dimension to stochastic switching-based super-resolution imaging. bSOFI allows the extraction of the super-resolved spatial distribution of molecular statistics, such as the on-time ratio, the brightness and the concentration of fluorophores by combining several cumulant orders. Moreover, this information can be used to balance the image contrast in order to compensate for the nonlinear brightness and blinking response of conventional SOFI images. Consequently, bSOFI enables higher-order cumulants to be used and thereby achieves higher resolutions.

Methods

Theory and algorithm

SOFI is based on the computation of temporal cumulants or spatio-temporal cross-cumulants. Cumulants are a statistical measure related to moments. Because cumulants are additive, the cumulant of a sum of independently fluctuating fluorophores corresponds to the sum of the cumulant of each individual fluorophore. This leads to a point-spread function raised to the power of the cumulant order n and therefore a resolution improvement of $\sqrt{n}$, respectively almost n with subsequent Fourier filtering (Dertinger et al. 2010). So far, SOFI has been used exclusively to improve structural details in an image (Dertinger et al. 2009; Dertinger et al. 2010). Information about the on-time ratio, the molecular brightness and the concentration has to our knowledge never been exploited before and therefore represents a new potential for super-resolved imaging.

In the most general sense, the cumulant of order n of a pixel set $\mathbb{P}=\{{\overrightarrow{r}}_1,{\overrightarrow{r}}_2,\dots ,{\overrightarrow{r}}_n\}$ with time lags $\overrightarrow{\tau }=\{{\tau }_1,{\tau }_2,\dots ,{\tau }_n\}$ can be calculated as (Leonov and Shiryaev 1959)

$$\begin{array}{l}{\kappa }_n\left(\overrightarrow{r}=\frac{1}{n}\sum _{i=1}^n{\overrightarrow{r}}_i;\overrightarrow{\tau }\right)=\sum _P{(-1)}^{\left|P\right|-1}(\left|P\right|-1)\\ \prod _{p\in P}{〈\prod _{i\in p}I({\overrightarrow{r}}_i,t-{\tau }_i)〉}_t,\end{array}$$

(1)

where stands for averaging over the time t. P runs over all partitions of a set $\mathbb{S}=\{1,2,\dots ,n\}$, which means all possible divisions of $\mathbb{S}$ into non-overlapping and non-empty subsets or parts that cover all elements of $\mathbb{S}$. |P| denotes the number of parts of partition P and p enumerates these parts. $I({\overrightarrow{r}}_i,t)$ is the intensity distribution measured over time on a detector pixel ${\overrightarrow{r}}_i$. We used the cross-cumulant approach without repetitions to increase the pixel grid density and eliminate any bias arising from noise contributions in auto-cumulants (Geissbuehler et al. 2011). A 4x4 neighborhood around every pixel was considered to compute all possible n-pixel combinations excluding pixel repetitions. For computational reasons, we kept only a single combination featuring the shortest sum of distances with respect to the corresponding output pixel $\overrightarrow{r}=\frac{1}{n}\sum _{i=1}^n{\overrightarrow{r}}_i$. For even better signal-to-noise ratios, it would be beneficial to average over multiple combinations per output pixel. The heterogeneity in output pixel weighting arising from the different pixel combinations has been accounted for by the distance factor as described in (Dertinger et al. 2010).

Considering a sample composed of M independently fluctuating fluorophores and assuming a simple two-state blinking model (with characteristic lifetimes τ _on,τ _off) with slowly varying molecular parameters compared to the size of the point-spread function (PSF), the cumulant of order n without time-lags can be interpreted as

$$\begin{array}{l}{\kappa }_n\left(\overrightarrow{r}\right)\propto \sum _{k=1}^M{\epsilon }_k^n{U}^n(\overrightarrow{r}-{\overrightarrow{r}}_k)f_n\left({\rho }_{\mathrm{on},k}\right)\\ \approx {\epsilon }^n\left(\overrightarrow{r}\right)f_n({\rho }_{\mathrm{on}};\overrightarrow{r})\sum _{k=1}^M{U}^n(\overrightarrow{r}-{\overrightarrow{r}}_k)\end{array}$$

(2)

with $\epsilon \left(\overrightarrow{r}\right)$ the spatial distribution of the molecular brightness and ${\rho }_{\mathrm{on}}\left(\overrightarrow{r}\right)=\frac{{\tau }_{\mathrm{on}}\left(\overrightarrow{r}\right)}{{\tau }_{\mathrm{on}}\left(\overrightarrow{r}\right)+{\tau }_{\text{off}}\left(\overrightarrow{r}\right)}$ the on-time ratio. $U\left(\overrightarrow{r}\right)$ is the system’s PSF and $f_n({\rho }_{\mathrm{on}};\overrightarrow{r})$ is the n-th order cumulant of a Bernoulli distribution with probability ρ _on:

$$\begin{array}{l}{f}_1({\rho }_{\mathrm{on}};\overrightarrow{r})={\rho }_{\mathrm{on}}\\ f_2({\rho }_{\mathrm{on}};\overrightarrow{r})={\rho }_{\mathrm{on}}(1-{\rho }_{\mathrm{on}})\\ ⋮\\ f_n({\rho }_{\mathrm{on}};\overrightarrow{r})={\rho }_{\mathrm{on}}(1-{\rho }_{\mathrm{on}})\frac{\partial f_{n-1}}{\partial {\rho }_{\mathrm{on}}}\end{array}$$

(3)

Assuming a uniform spatial distribution of molecules inside a detection volume V centered at $\overrightarrow{r}$, we may further approximate

$$\sum _{k=1}^M{U}^n(\overrightarrow{r}-{\overrightarrow{r}}_k)\approx {\mathcal{ℰ}}_V\left\{U^n\left(\overrightarrow{x}\right)\right\}N\left(\overrightarrow{r}\right),$$

(4)

where ${\mathcal{ℰ}}_V\left\{U^n\left(\overrightarrow{x}\right)\right\}=1/V\underset{V}{\int }{U}^n\left(\overrightarrow{x}\right)d\overrightarrow{x}$ is the expectation value of $U^n\left(\overrightarrow{x}\right)$ or the n-th moment of $U\left(\overrightarrow{x}\right)$ (see (Kask et al. 1997) for some examples) and $N\left(\overrightarrow{r}\right)$ denotes the number of molecules within the detection volume V . Finally, we can write

$${\kappa }_n\left(\overrightarrow{r}\right)\approx {\mathcal{ℰ}}_V\left\{U^n\left(\overrightarrow{x}\right)\right\}N\left(\overrightarrow{r}\right){\epsilon }^n\left(\overrightarrow{r}\right)f_n({\rho }_{\mathrm{on}};\overrightarrow{r}).$$

(5)

Based on at least three different cumulant orders and approximation (5), it is possible to extract the molecular parameter maps $\epsilon \left(\overrightarrow{r}\right)$, $N\left(\overrightarrow{r}\right)$ and ${\rho }_{\mathrm{on}}\left(\overrightarrow{r}\right)$ by solving an equation system, or by using a fitting procedure. For example, the cumulant orders two to four can be used to build the ratios

$$\begin{array}{ll}{K}_1\left(\overrightarrow{r}\right)& =\frac{{\mathcal{ℰ}}_V\left\{U^2\left(\overrightarrow{x}\right)\right\}{\kappa }_3}{{\mathcal{ℰ}}_V\left\{U^3\left(\overrightarrow{x}\right)\right\}{\kappa }_2}\left(\overrightarrow{r}\right)\\ =\epsilon \left(\overrightarrow{r}\right)\left(1-2{\rho }_{\mathrm{on}}\left(\overrightarrow{r}\right)\right)\\ K_2\left(\overrightarrow{r}\right)& =\frac{{\mathcal{ℰ}}_V\left\{U^2\left(\overrightarrow{x}\right)\right\}{\kappa }_4}{{\mathcal{ℰ}}_V\left\{U^4\left(\overrightarrow{x}\right)\right\}{\kappa }_2}\left(\overrightarrow{r}\right)\\ ={\epsilon }^2\left(\overrightarrow{r}\right)\left(1-6{\rho }_{\mathrm{on}}\left(\overrightarrow{r}\right)+6{\rho }_{\mathrm{on}}^2\left(\overrightarrow{r}\right)\right)\end{array}$$

(6)

and to solve for the molecular brightness

$$\epsilon \left(\overrightarrow{r}\right)=\sqrt{3K_1^2\left(\overrightarrow{r}\right)-2K_2\left(\overrightarrow{r}\right)},$$

(7)

the on-time ratio

$${\rho }_{\text{on}}\left(\overrightarrow{r}\right)=\frac{1}{2}-\frac{K_1\left(\overrightarrow{r}\right)}{2\epsilon \left(\overrightarrow{r}\right)},$$

(8)

and the molecular density

$$N\left(\overrightarrow{r}\right)=\frac{{\kappa }_2\left(\overrightarrow{r}\right)}{{\epsilon }^2\left(\overrightarrow{r}\right){\rho }_{\text{on}}\left(\overrightarrow{r}\right)\left(1-{\rho }_{\text{on}}\left(\overrightarrow{r}\right)\right)}.$$

(9)

The spatial resolution of the estimation is limited by the lowest order cumulant, i.e. the second order in this case. However, the presented solution is not unique. Basically any three distinct cumulant orders could have provided a solution. Furthermore, the method is not limited to a two-state system; it can be extended to more states as long as the differences in fluorescence emission are detectable. Additional details on the analytical developments as well as a theoretical investigation of the estimation accuracy of the different parameters under different conditions are given in the Additional file 1.

In order to correct for the amplified brightness and blinking heterogeneities without compromising the resolution, the cumulants have to be deconvolved first. For this purpose, we used a Lucy-Richardson algorithm (Lucy 1974; Richardson 1972) implemented in MATLAB (deconvlucy, The MathWorks, Inc.), which is an iterative deconvolution without regularization that computes the most likely object representation given an image with a known PSF and assuming Poisson distributed noise. Apart from the estimate of the cumulant PSF and the specification of a maximum of 100 iterations, all arguments have been left at their default values. Assuming a perfect deconvolution without regularization, the result could be interpreted as

$${\breve{\kappa }}_n\left(\overrightarrow{r}\right)\approx {\epsilon }^n\left(\overrightarrow{r}\right)f_n({\rho }_{\mathrm{on}};\overrightarrow{r})\sum _{k=1}^M\delta (\overrightarrow{r}-{\overrightarrow{r}}_k).$$

(10)

Taking then the n-th root linearizes the brightness response without cancelling the resolution improvement of the cumulant. To reduce the amplified noise and masking residual deconvolution artefacts, small values (typically 1-5% of the maximum value) are truncated and the image is reconvolved with $U\left(n\overrightarrow{r}\right)$. This leads to a final resolution improvement of almost n-fold compared to the diffraction-limited image, which is physically reasonable since the frequency support of the cumulant-equivalent optical transfer function (OTF) is n-times the support of the system’s OTF (cf. (Dertinger et al. 2010)). In contrast to Fourier reweighting (Dertinger et al. 2010), which is equivalent to a Wiener filter deconvolution and reconvolution with $U\left(n\overrightarrow{r}\right)$, we explicitly split these two steps and use an improved but computationally more expensive deconvolution algorithm that is appropriate for the subsequent linearization.

Since the cumulants are proportional to $f_n({\rho }_{\mathrm{on}};\overrightarrow{r})$, which contains n roots for ${\rho }_{\mathrm{on}}\in \left[0,1\right]$, there might still be hidden image features in these brightness-linearized cumulants (result after deconvolution, n-th root and reconvolution with $U\left(n\overrightarrow{r}\right)$). However, using the on-ratio map ${\rho }_{\mathrm{on}}\left(\overrightarrow{r}\right)$, it is straightforward to identify the structural gaps around the roots of f _n and fill them in with the brightness-linearized cumulant of order n-1. To this end, the locations where f _n approaches zero are translated into a weighting mask with smoothed edges around these locations. The thresholds have been defined by computing the crossing points of ${\left|f_n\right|}^{1/n}$ and ${\left|f_{n-1}\right|}^{1/(n-1)}$. This mask is then applied on the n-th order brightness-linearized cumulant and its negation is applied on order n-1 (see Additional file 1 for further details). The result is a balanced cumulant image. It should be noted that the cancellation of $f_n\left(\overrightarrow{r}\right)$ by division is possible but not recommended, because it amplifies noisy structures in the vicinity of the roots of f _n . The combination of multiple cumulant orders in a balanced cumulant image results in a better overall image quality. However, it is also possible that the on-ratio varies only slightly throughout the field of view, such that a combination of multiple cumulant orders is not necessary. Figure 1 illustrates the different steps of the algorithm based on a simulation of randomly blinking fluorophores, arranged in a grid of increasing density from right to left, increasing brightness from left to right and increasing on-time ratio from top to bottom.

Figure 1

bSOFI algorithm. Flowchart to illustrate the different steps of the bSOFI algorithm. (a) Raw data. (b) Cross-cumulant computation up to order n according to equation (1) without time lags. (c) Cumulant ratios K ₁ and K ₂ according to equation (6). (d) Deconvolved cumulant of order n. (e) Solution for the spatial distribution of the molecular brightness ε, on-time ratio ρ _onand density N using equations (7-9). (f) Balanced cumulant of order n, obtained by computing the n-th root of the deconvolved cumulant, reconvolving with $U\left(n\overrightarrow{r}\right)$ and filling in the structural gaps around the roots of f _n with a lower-order cumulant. (g) Color-coded molecular parameter maps overlaid with a balanced cumulant as a transparency mask.

Experiments

In order to verify the concept experimentally, we used a custom-designed objective-type total internal reflection (TIR) fluorescence microscope with a high-NA oil-immersion objective (Olympus, APON 60XOTIRFM, NA 1.49, used at 100x magnification), blue (490nm, 8mW, epi-illumination) and red (632nm, 30mW, TIR illumination) laser excitation and an EMCCD detector (Andor Luca S). We imaged fixed HeLa cells with Alexa647-labelled alpha tubulin and used a chemical buffer containing cysteamine and an oxygen-scavenging system (Heilemann et al. 2008) (see Additional file 1 for further details) to generate reversible blinking and an increased bleaching stability. The blue laser excitation was used to accelerate the reactivation of dark fluorophores and to reduce the acquisition time. For data processing, 5000 images acquired at 69 frames per second (fps) were divided into 10 subsequences significantly shorter than the bleaching lifetime to avoid correlated dynamics among the fluorophores (Dertinger et al. 2010). The final bSOFI images are obtained by averaging over the processed subsequences.

Results and discussion

Figure 2 compares the performance of bSOFI with conventional SOFI and widefield fluorescence microscopy. The peak signal-to-noise ratio (pSNR; noise measured on the background) in a single frame was 20-23dB for the brightest molecules, which is rather low for performing localization microscopy, but more than sufficient for SOFI (Geissbuehler et al. 2011). Additionally, we observed significant read-out noise at this acquisition speed (fixed-pattern noise in the average image, Figure 2a,i), which was effectively removed in the cross-cumulants analysis (Figure 2b-e and j,k). The estimated molecular on-time ratio (c,k), brightness (d) and density (e) are shown overlaid with the 5th order balanced cumulant as a transparency mask. Due to the overemphasis of slight heterogeneities in molecular brightness and blinking, the dynamic range of the conventional 5th order SOFI image (b and j), where values above 1% of the maximum are truncated, is too high to be represented meaningfully. Figures 2f-h are the projected profiles of the widefield, SOFI, Fourier reweighted SOFI (Dertinger et al. 2010) and bSOFI images along the cuts 1-1’, 2-2’ and 3-3’, respectively. The second profile, with two microtubule structures separated by 80nm, illustrates a situation close to the Rayleigh criteria in the bSOFI case. This is consistent with the measured full width at half maximum (FWHM) of 78nm (Figure 2h). Although the Fourier reweighted SOFI features a FWHM of 75nm (Figure 2h), it does not resolve the two microtubules at 2-2’. Due to the nonlinear brightness response only a single one is visible (Figure 2g). When considering the effective width of the microtubule of 22nm as well as a 15nm linker length, this translates into a bSOFI-equivalent PSF with 64nm FWHM, respectively a 4.6-fold resolution improvement compared to widefield microscopy. The resolution improvement of the conventional 5th order SOFI image is close to the expected factor $\sqrt{5}$. With the red TIR illumination, the excitation intensity decreases exponentially along the optical axis. Assuming a homogeneous illumination in the x-y plane, both the molecular brightness and the on-time ratio can be interpreted as a depth encoding because they are related to the illumination intensity (van de Linde et al. 2011). Obviously, a depth encoding based on molecular parameters can only be used as a qualitative impression of depth information rather than real 3D imaging, because it does not resolve additional structures in depth. Moreover, when looking at smaller scales (Figure 2k), the depth impression of color-coded molecular parameters gets less evident, which can be explained by the influence of local differences in the chemical microenvironment or by the stochastic nature of individual fluorophores.

Figure 2

bSOFI demonstration. Experimental demonstration of bSOFI on fixed HeLa cells with Alexa647-labelled microtubules. (a) Summed TIRF microscopy image [Widefield]. (b) Conventional 5th order cross-cumulant SOFI [SOFI5]. (c-e) Color-coded molecular on-time ratio, brightness and density overlaid with the 5th order balanced cumulant [BC5]. (f-h) Profiles along the cuts 1-1’, 2-2’ and 3-3’. In yellow we added the corresponding profiles when Fourier reweighting (cf. (Dertinger et al. 2010)) with a damping factor of 5% is applied on the 5th order cross-cumulant SOFI image. (i-k) Magnified views from the white insets highlighted in (a-c). Scale bars:2μm(a-e); 500nm (i-k).

Although the used Lucy-Richardson deconvolution performed well on our measurements, it is not specifically adapted for cumulant images, because it assumes a Poisson-distributed noise model and an underlying signal that is strictly positive. For the n-th order cumulant, the signal on a single image can vary between positive and negative values according to the underlying on-ratios. Furthermore, initially Poisson-distributed noise leads to a modified noise distribution in the cumulant image. In our experiments, the local on-ratio variations were small, which proves to be unproblematic for a deconvolution with a positivity constraint, when the negative and the positive parts are considered separately. However, a deconvolution algorithm specifically adapted for cumulant images using an appropriate noise model may improve the results of balanced cumulants in the future.

If the cumulants are computed for different sets of time lags and the acquisition rate oversamples the blinking rate, it is also possible to extract absolute estimates on the characteristic lifetimes of the different states. The temporal extent of the curve obtained by computing the second-order cross-cumulant as a function of time lag τ(corresponding to a centred second-order cross-correlation) before it approaches zero yields an estimate on the blinking period, provided that the timeframe of the measurement includes many blinking periods. In our case however, with only 10 to 20 blinking periods within a measurement window of 500 frames (@69fps) and a low on-ratio, the temporal extent of the correlation curve rather corresponds to the characteristic on-time. Figure 3a,b show the resulting on- and off-time maps overlaid with a 5th order balanced cumulant as a transparency mask. The reported on-times correspond to the time position where the curve decreased to e^-1 of the value at zero time lag. The off-time map is obtained by calculating ${\tau }_{\text{off}}={\tau }_{\mathrm{on}}\left({\rho }_{\mathrm{on}}^{-1}-1\right)$. The off-time map hardly shows a dependency on the illumination intensity, which is in line with the deep penetration into the sample of the blue activation light. In the present case, the lifetime of the off-state is influenced mainly by the chemical composition of the microenvironment surrounding the probe (van de Linde et al. 2011).

Figure 3

On- and off-times. Spatial distribution of the estimated on- (a) and off-times (b) overlaid with a 5th order balanced cumulant as a transparency mask. The images correspond to an average over 10 subsequences of 500 frames each. (c) Second-order cross-cumulant with different time lags, averaged over the x-y-plane and 10 subsequences of 500 frames each. An exponential fit to the measured curve is shown in black. Scale bars: 2μm.

For estimating the average on-time, we computed the second-order cross-cumulant as a function of time lag and averaged it over the x-y-plane and 10 subsequences of 500 frames (Figure 3c). The fitted exponential curve has a characteristic time constant of $\overline{{\tau }_{\text{on}}}=31\text{ms}$.

Conclusions

bSOFI is an extended version of SOFI and shares its advantages of simplicity, speed, rejection of noise and background, and compatibility with various blinking conditions. Since the bSOFI-PSF shrinks in all three dimensions with increasing cumulant orders, bSOFI can easily be extended to the axial dimension by acquiring multiple depth planes and performing the analysis in three dimensions. In contrast to SOFI, the bSOFI response to brightness and blinking heterogeneities in the sample is nearly linear, which allows higher resolutions to be obtained by computing higher cumulant orders. The additional information on the spatial distribution of molecular statistics may be used to monitor static differences and/or dynamic changes of the probe-surrounding microenvironments within cells and thus may enable functional super-resolution imaging with minimum equipment requirements.