Abstract
Multi-dimensional electron microscopy has recently gained considerable interest thanks to the advent of microscopes with unprecedented analytical and in situ capabilities. These information-rich imaging modes, though, are often subject to long acquisition times and large data generation. In this paper, we explore novel acquisition strategies and reconstruction algorithms to retrieve reliable reconstructions from datasets that are limited in terms of both per image and tilt series angular sampling. We show that inpainting techniques are capable of restoring scanning transmission electron microscopy images in which a very restricted number of pixels are scanned, while compressed sensing tomographic reconstruction is capable of minimising artefacts due to angular subsampling. An example of robust reconstruction from data constituting a dose reduction of 10× is presented, using an organic/inorganic core-shell nanowire as a test sample. The combination of these novel acquisition schemes and image recovery strategies provides new avenues to reduced-dose and high-speed imaging.
Similar content being viewed by others
Background
Electron tomography (ET) is an invaluable technique for the three-dimensional (3D) characterisation of nanomaterials, frequently playing a unique role in elucidating nanostructure-property relationships [1]. ET entails acquiring a set of transmission electron microscopy (TEM) images at different tilt angles and subsequently employing a reconstruction algorithm to retrieve the 3D object [2]. Typically, bright field TEM is used in the life sciences, while high-angle annular dark-field scanning TEM (HAADF-STEM) is preferred in materials science because of its relative insensitivity to diffraction effects and atomic-number (Z) contrast [3]. ET based on these imaging modes is nowadays often performed in an automated or semi-automated manner and is routinely employed for the study of morphology and distribution of features in 3D (e.g. [4-7]).
Recently, ‘multi-dimensional electron microscopy’ (often ET plus a further dimension) has gained significant interest, with the desire to follow structural changes in space and time and to generate chemically sensitive volume reconstructions that can be analysed quantitatively [8]. More specifically, with the availability of high-brightness electron sources, monochromators, X-ray detectors with improved efficiency and energy loss spectrometers with high speed and dynamic range, ground-breaking new insights are within reach.
Nonetheless, 3D imaging coupled with dynamic and/or spectroscopic dimensions faces challenges with regard to achieving sufficient signal-to-noise ratio (SNR) at an acceptable acquisition time and electron dose. To date, the reconstruction fidelity has generally been optimised by acquiring many projections over as wide a tilt range as possible. Conventional reconstruction algorithms, such as weighted backprojection (WBP) or the simultaneous iterative reconstruction technique (SIRT), yield reasonable tomograms when the dataset is acquired with a high SNR and a fine angular sampling increment [1,2 and references therein]. Unfortunately, such standard acquisition parameters cannot easily be extended to multi-dimensional experiments. STEM-based spectroscopies such as energy dispersive X-ray spectroscopy (EDX) or electron energy loss spectroscopy (EELS) generally require long per-pixel acquisition times and, due to the additional spectral dimension, generate large amounts of data. Extension of these methods to 3D spectroscopic imaging using TEM or STEM requires the sample to be beam-resistant (e.g. [9]) and/or a sacrifice of the image resolution by reducing the size of the spectrum-images. Using state-of-the art TEMs, voxel spectroscopy by simultaneous EELS and EDX tomography was demonstrated by Haberfehlner et al. in [10], after generating 2,920 spectral volumes individually. Likewise, in situ dynamic tomography requires high-speed tilt series acquisition, ideally of multiple tilt series, in order to capture 3D snapshots. In Kwon and Zewail’s work, time resolved 4D ET was performed on a multi-walled carbon nanotube, with the acquisition of 4,000 spatiotemporal projections, followed by alignment, reconstruction and rendering of each 3D temporal volume separately [11].
There is a strong need to develop both acquisition and processing tools that are specifically adapted to low-dose and high-speed ET and for STEM-based techniques in particular. Different approaches can be explored for this purpose, with various consequences on the quality of the signal and hence the reliability of the reconstruction. These approaches can be classified as tilt-downsampling (T-DS) and image-downsampling (I-DS) techniques.
T-DS techniques
T-DS techniques rely on either reducing the tilt range or increasing the tilt increment. In general, datasets with limited tilt range tend to result in reconstructions with artefacts that are harder to correct than those related to large tilt increments. Indeed, a limited tilt range results in a large ‘missing wedge’ of information in the Fourier space of the object. Consequent artefacts, such as blurring and elongation of features in the missing wedge direction, deteriorate the quality of the reconstruction and make quantification difficult (e.g. [12]). So far, therefore, tilt series acquisition times have generally been reduced by acquiring fewer images over the widest tilt range available and employing advanced reconstruction algorithms such as discrete algebraic reconstruction technique (DART) or compressed sensing ET (CS-ET). The common feature of these techniques is the incorporation of prior knowledge in the reconstruction process. DART has been introduced to ET by Batenburg et al. [8] for the reconstruction of objects with a known number of grey levels [13]. Recently, a variant of DART was employed for the 3D atomic reconstruction of a crystalline nanoparticle, using only two HAADF-STEM projections and strong assumptions about its crystallographic structure [14]. CS-ET is a more general technique that takes advantage of any prior knowledge about the sparsity of the object in a chosen transform domain and seeks a sparse solution that is consistent with the acquired data [15]. It has been successfully applied to heavily undersampled datasets, using different sparsifying domains depending on the nature of the signal [16-18].
I-DS techniques
I-DS techniques include (1) reduction of the frame size, (2) reduction of the per-pixel dwell time, (3) reduction of the beam current density and (4) recording only a subset of pixels of the full frame (e.g. a random selection of pixels). Reducing the frame size (option 1), by increasing the pixel size to reduce the total number of pixels per frame, is the most straightforward approach but leads to an irretrievable loss of resolution. Options (2) and (3), recently explored in [19] for imaging beam-sensitive materials, are becoming increasingly popular in the area of high resolution STEM (HR-STEM) in which the acquisition of a series of low-dose images is performed, followed by registration and averaging [20,21].
Option (4) is commonly explored for example in magnetic resonance imaging (MRI), where various patterns are tested and implemented, with the aim to accelerate dynamic examinations [22]. Similar manipulation of scanning patterns has attracted the interest of the microscopy community, with the difference that MRI (sub-)sampling is in the Fourier domain, while the microscopy counterpart takes place in the image domain. Notably, Lissajous scan patterns were recently applied to atomic force microscopy for time-lapsed imaging and were shown to yield reduced distortions compared to fast raster-based scanning [23]. Random subsampling has also been explored for high-speed electron microscopy data collection, mainly for dose rate considerations. Anderson et al. simulated such an experiment in the scanning electron microscope (SEM) by selecting a random subset of pixel locations and recovering the full frame image by interpolation, using image smoothness as a prior knowledge [24]. This approach was also simulated on HR-STEM images of grain boundaries and lower magnification STEM images of extremely beam-sensitive materials, with the scanning of only 5% of the total number of pixels in the original image and recovery achieved by Bayesian dictionary learning technique [25].
Although automated low-dose ET acquisition packages have been developed and algorithms that tackle datasets with large tilt increments have been successfully implemented, I-DS techniques based on recording a subset of pixels have not yet, to our knowledge, been tested on ET datasets.
In this paper, we explore a combination of T-DS and I-DS strategies in order to lower the electron dose and acquisition time. I-DS restoration is achieved by total variation (TV) inpainting, while the tomographic reconstructions are performed by CS-ET. The approach is tested on an organic/inorganic core-shell CoPc/ZnO nanowire (NW), where ET is employed to examine the integrity of the core and the porous structure of the shell.
Methods
HAADF-STEM tomography of a CoPc/ZnO NW
CoPc/ZnO NWs are 1D organic/inorganic core-shell materials formed by an inner organic single-crystal NW of cobalt phthalocyanine (CoPc) conformally covered with a zinc oxide (ZnO) layer. The synthesis procedure entails a two-step vacuum methodology. First, single-crystal CoPc NWs are grown by physical vapour deposition, using substrates decorated with metal nanoparticles that act as nucleation centres. Then, a ZnO shell is grown at room temperature using plasma-enhanced chemical vapour deposition in a remote configuration. The properties of CoPc/ZnO NWs depend strongly on the integrity of the organic core after the plasma deposition of the shell [26,27]. This property, along with the detailed morphology of the ZnO shell along the full NW, was investigated here by HAADF-STEM tomography. For this purpose, the sample was dispersed on a commercial TEM grid possessing a 5 nm-thick carbon film on a thicker 30 nm holey carbon film (Pacific Grid Tech, San Francisco, CA, USA). Using a probe-corrected TITAN microscope (FEI, Columbus, OH, USA) operating at 300 kV, a NW oriented parallel to the rotation axis was selected, and a HAADF-STEM tilt series was acquired from −70° to +68° with 2° increment. The frame size was 1024 × 1024, with a pixel size of 1.54 nm and a frame time of 15 s. The tilt series was exported to Inspect3D (FEI, Hillsboro, OR, USA) for alignment by cross-correlation. This full dataset was then used as a reference and subsequently downsampled for I-DS and T-DS simulations. In addition to the CS-ET reconstructions, a reference reconstruction was also generated using the full dataset and a conventional SIRT algorithm, implemented in Inspect3D.
Simulation of I-DS and T-DS set-ups
To simulate randomised subsampling regimes across the full 1024×1024 frame size, patterns of ones and zeros, following Bernoulli distribution, were generated in Matlab (MathWorks, Natick, MA, USA). The patterns were then used as masks over the fully sampled projections. Experimentally, this set-up simulates a reduction in dose (and acquisition time) per image, without altering the SNR of the sampled pixels. Additionally, the angular increment can be increased in order to further reduce the total dose for the tilt series. This effectively results in fewer pixels scanned in the dataset, but each could achieve a higher SNR than in a conventional low-dose tomography experiment where dose fractionation approach is commonly used (i.e. all pixels are sampled and a maximum permissible number of projections are acquired but with a lower dose and SNR per pixel).
Restoration of the I-DS projections
In this work, in-house code written in Matlab is used to perform TV inpainting, following the approach described in [28]. In order to interpolate the missing pixels in the projection images, we compute a solution of the optimisation problem:
Here, f denotes the sampled pixels of the projection image, S is the subsampling operation that selects only those pixels of a full sized image û that are known and TV(û) is the discrete isotropic TV of û (see [29] for explicit notation). Thus, the optimisation problem (1) is an interpolation method that aims at finding an image û for which the given pixels of the projection image are unaffected, while missing pixels are filled such that the TV of the whole image û is minimal. In order to solve (1) computationally, we reformulate it to:
Note that other regularisation functions (such as the squared two-norm of the gradient of the image, the one-norm of a wavelet transformation of the image or the total generalised variation of the image (cf. [30]) for instance) are possible and yield different interpolation results.
CS-ET reconstruction of the T-DS datasets
The CS-ET algorithm was also implemented in Matlab. For a comprehensive description, the reader is referred to [15]. In summary, the projections are first Fourier transformed to obtain radial samples of the object in the Fourier domain. An initial reconstruction is then obtained from the radial Fourier data using the non-uniform fast Fourier transform (NUFFT) developed by Fessler and Sutton [31]. In conjunction with the NUFFT, the conjugate gradient descent algorithm of Lustig et al. [32] is then used to solve the optimization problem defined by:
where \( \widehat{x} \) is the reconstruction of the true signal x, Φ is the undersampled Fourier transform operator, b is the Fourier transform of the acquired tilt series and Ψ is the chosen sparsifying transform. Similar to the TV term in Equation 1 and the ℓ 1-norm term in Equation 2, the ℓ 1-norm term in Equation 3 promotes sparsity in the chosen transform domain and is defined as the sum of the absolute values. λ is a regularisation parameter that determines the relative importance of sparsity in the reconstruction. Essentially, the algorithm promotes sparsity in the Ψ basis, subject to consistency with the measured data. For the CoPC/ZnO NW, sparsity was promoted in both the image and gradient domains. In this case, Equation 3 can be reformulated as:
where similarly λ I and λ TV are regularisation parameters that reflect the degree of sparsity imposed in the image and gradient domains, respectively. These λ I and λ TV values are chosen by visual assessment of reconstruction quality on selected slices, aiming for the optimal trade-off between minimization of reconstruction artefacts and loss of genuine signal. In the CS-ET implementation used here, TV \( \left(\widehat{x}\right) \) is calculated as the isotropic TV, although other variants are applicable.
Results
Reconstruction of the fully sampled dataset
Figure 1a shows the zero tilt HAADF-STEM projection of the NW from the tilt series, where Z-contrast gives insight into the different components present (see the full tilt series in Additional file 1). A voxel projection visualisation of a conventional SIRT reconstruction shows the porous nanocolumnar structure of the ZnO shell (Figure 1b). As can be appreciated in the cross-sectional view through the tomogram in Figure 1c, the columns do not lie perpendicular to the organic core of the NW but at an inclination of ca. 30°. This is due to the relative orientation of the organic NW and the direction of the incoming plasma species during the synthesis process. The organic core was found to have a rectangular cross-section (Figure 1d), with no evident damage throughout the length of the NW (Figure 1c).
HAADF-STEM tomography of a CoPc/ZnO nanowire. (a) HAADF-STEM image of the CoPc/ZnO NW taken at 0° tilt angle. (b) Voxel projection visualisation of the SIRT reconstruction, showing the integrity of the organic core (*) and the nanocolumnar structure of the ZnO shell (**). (c) xy cross-sectional slice, showing a 30° inclination of the branches in the shell. (d) xz cross-sectional slice, showing the rectangular shape of the core and the porous structure of the shell.
The movie in Additional file 2 shows the CS-ET reconstruction of the fully sampled dataset, used as ground truth in the rest of the paper. Note that a better signal-to-background ratio is achieved with CS-ET than with SIRT (Figure 1c,d), although qualitatively both of these reconstructions permit similar levels of structural details to be discerned. As shown in e.g. [15,16], however, CS-ET can substantially outperform SIRT when the tilt increment is large and when seeking to analyse the reconstruction quantitatively.
TV inpainting of I-DS datasets
Figure 2b,e,h shows the 0° tilt projections obtained with only 60%, 30% and 10% of pixels randomly sampled, as compared to the fully sampled image (Figure 2a). Figure 2c,f,i shows the restored images after TV inpainting.
Downsampling by simulated random scanning of the full frame image (a), using a Bernoulli pattern. (b,e,h) show images where only 60%, 30% and 10% of the pixels are sampled from the image in (a), respectively. (c,f,i) show the corresponding TV inpainting results and root mean square errors. Image grey levels have been normalised to the maximum of the fully sampled image in order to ease the comparison. (d,g,j) are line profiles through the yellow line in the inset in (d), comparing the reference, subsampled and TV-inpainted images. Scale bar: 200 nm.
Good recovery is achieved for the 60% and 30% I-DS simulations, with a root mean square error (RMSE) of 1.09% and 2.82%, respectively. The quality of the inpainting can also be appreciated qualitatively in the insets in Figure 2c,f, and in the line profiles in Figure 2d,g, where a good match between the ground-truth image (full black line) and the recovered image (dashed red line) is observed.
Details of the nanoporous structure of the shell are not as well recovered with 10% I-DS (inset in Figure 2i, RMSE = 5.63%). The overall shape of the core-shell NW is, however, still successfully retrieved, as seen in the line profile (Figure 2j).
Combination of I-DS and T-DS
For simulating reduced-dose ET based on both I-DS and T-DS, tilt series are generated by changing the Bernoulli subsampling pattern for each tilt increment in order to reduce coherent artefacts in the reconstruction (see Additional files 3, 4 and 5). I-DS projections are restored individually (see Additional files 6, 7 and 8), and CS-ET with sparsity in the image and TV domains is employed to reconstruct the NW under T-DS settings.
Figure 3 shows a selected xz orthoslice obtained with various degrees of I-DS (rows) and T-DS (columns). The fidelity of the reconstructions is affected differently by the two downsampling schemes. In all T-DS scenarios, CS-ET suppresses well-known ‘streaking’ artefacts that would occur in conventional reconstructions from few projections. For a given I-DS setting, details of the outer branching in the shell are consistently retrieved, while small pores become more difficult to discern as the T-DS increases (see for example the insets in Figure 3a,d and the corresponding line profiles in Figure 4). This is consistent, for example, with the difference in quantified resolution achieved using a fully sampled tilt series with 2° tilt increment and one with 8° tilt increment (6.31 and 11.7 nm, respectively) or between 30% I-DS tilt series with 4° and 6° tilt increment (8.21 and 8.8 nm, respectively). The resolution was estimated by Fourier shell correlation, wherein two volumes are reconstructed, each based on half of the available tilt series. We note though that Fourier shell correlation of reconstructions with large tilt increments can lead to errors in the resolution estimation and must be carefully interpreted and complemented with qualitative evaluation; for example, the reconstruction with 100% I-DS and 8° tilt increment is visually of higher fidelity than the reconstruction with 10% I-DS and 2° tilt increment (based on Figures 3 and 4), even though the estimated resolution is worse.
CS-ET reconstructions from a combination of I-DS (rows) and T-DS (columns) settings. (a-d) correspond to 100% I-DS setting, (e-h) to 60%, (i-l) to 30% and (m-p) to 10%. Each row shows reconstructions obtained with 2°, 4°, 6° and 8° T-DS settings. I-DS tilt series were restored by TV inpainting prior to CS-ET reconstruction. Red insets in (d) and (f) show the reconstructed pores obtained with a relative dose of 0.25 and 0.3, respectively, compared to the fully sampled reconstruction (yellow inset in (a)). Green insets in (k) and (m) both show the reconstructed pores obtained with a factor of 0.1 dose reduction but using different I-DS and T-DS settings. Scale bar: 200 nm.
In general the level of T-DS permissible is highly dependent on the complexity of the particular object under study, the prior knowledge incorporated in the CS-ET optimization and the level of detail desired in the ET reconstruction.
I-DS can also have a noticeable impact on the reconstruction, potentially even more than T-DS. For this data set, 60% and 30% I-DS lead to similar results. The small pores are recovered in both settings, but with a better visibility in the case of 60% I-DS, which correlates with the lower RMSE of the TV-inpainted tilt series (see Figure 2c,f). This impact of I-DS is also reflected by the higher resolution achieved with 60% I-DS and 4° increment (7.69 nm) compared to 30% with 4° increment (8.21 nm). The 10% I-DS reconstructions retrieve the overall shape of the NW but fail to recover details of both the outer branching of the shell and the pores, even with 2° tilt increment (green inset in Figure 3m). This is because the TV-inpainted projections (Figure 2i) do not contain the information about such fine structures. Nonetheless, 10% I-DS might be sufficient for ET of nanoparticles with basic morphology and may be necessary for 3D dynamic studies.
It is worth noting that various I-DS and T-DS settings with similar dose lead to reconstructions with different resolution (Table 1). For example, 60% I-DS with 4° increment yields a markedly better reconstruction than 100% I-DS with 8° increment (red inset in Figure 3d,f and corresponding line profiles in Figure 4), even though the relative dose is similar (0.3 and 0.25, respectively). Likewise, both 30% I-DS with 6° increment and 10% with 2° increment correspond to a relative dose of 0.1, but the first setting leads to higher fidelity reconstruction than the latter (see green insets in Figure 3k,m and corresponding line profiles in Figure 4). A CS-ET reconstruction of the entire NW using the 30% I-DS and 6° increment setting is displayed in Figure 5 and is comparable to the fully sampled reconstruction in Figure 1 - recovering the salient details of the nanoporous structure of the shell and the integrity of the core (see cross-sections through the NW in Figure 5c,d,e and the movie in Additional file 9).
CS-ET reconstruction of the entire NW from a dataset with 30% I-DS and 6° T-DS, corresponding to a dose reduction of 10×, compared to Figure 1. (a) voxel projection visualisation of the CS-ET reconstruction. (b) xy cross-sectional slice. (c, d, e) xz cross-sectional slices, at y positions indicated in (b).
From these results, it seems crucial, for a successful dose reduction in ET experiments, to select I-DS and T-DS settings prudently, taking into account the structural complexity of the object to be reconstructed.
Discussion
In this paper, TV inpainting and CS-ET were combined to simulate low-dose ET experiments with random STEM pixel subsampling and reduced numbers of tilt series projections. It was found that a dose reduction of 10× still enabled high fidelity reconstructions to be made. Although these proof-of-principle simulations have been performed on conventional HAADF-STEM projections, an important impact on multi-dimensional microscopy is expected: EDX and EELS maps are often acquired with a short dwell time, leading to low SNR signals and difficulties in applying spectral analysis techniques. Both I-DS and T-DS approaches may provide the signal level required for reliable processing of the sampled pixels. In the case of in situ tomography, I-DS and T-DS strategies may enable the study of changes at finer time scales. There is also the potential, with both the TV inpainting and CS-ET reconstruction, to incorporate prior knowledge (viz. sparsity) over time to further enhance the reconstructions and/or enable even greater subsampling.
The proposed approach may be further optimised by exploring other tilting strategies such as the Saxton scheme [33] or equally sloped tomography [34] or other tilting regimes specifically adapted for particular scanning trajectories or sample geometry. Moreover, the TV inpainting and CS-ET algorithms are closely related, making it possible to incorporate the inpainting approach into the CS-ET reconstruction. In general, performing the CS-ET optimization in 3D, with prior knowledge about the full object rather than individual 2D slices, should yield higher fidelity ET reconstructions (see [10] for an illustrative demonstration).
It must be emphasised that the degree of downsampling is strongly dependent on the level of detail in the object to be reconstructed: While a dose reduction of 10× was demonstrated on the CoPC/ZnO NW, the dose can probably be reduced even further for constant-density nanoparticles with a regular shape but may need to be increased for identifying, for example, the distribution of small particles on a complex support. As seen in Figure 3 the quality of the reconstruction can depend heavily on the quality of the restored I-DS tilt series. Therefore, it is crucial to optimise downsampling by exploring various scan patterns, interpolation techniques and tilting regimes that are best tailored to the specimen under consideration and practical implementation.
Scanning trajectories can be scripted (for example, using the Gatan Digiscan system) and should be designed in such a way that mechanical and electronic instabilities are minimised.
Finally, in addition to enabling multi-dimensional ET of a wide range of nanomaterials in the physical sciences, I-DS and T-DS may be of significant interest to the life science community, where samples are often extremely dose-sensitive.
Conclusions
This paper has explored novel acquisition and recovery strategies for ET experiments with reduced dose and acquisition time. The combination of TV inpainting and CS-ET reconstruction was found to yield reliable reconstructions from significantly undersampled datasets. The next steps are to address the practical implementation of the proposed strategies and to further develop the recovery procedures.
Abbreviations
- ET:
-
electron tomography
- 3D:
-
three-dimensional
- TEM:
-
transmission electron microscopy
- HAADF-STEM:
-
high-angle annular dark-field scanning TEM
- Z:
-
atomic-number
- SNR:
-
signal-to-noise ratio
- WBP:
-
weighted backprojection
- SIRT:
-
simultaneous iterative reconstruction technique
- EDX:
-
energy dispersive X-ray spectroscopy
- EELS:
-
electron energy loss spectroscopy
- T-DS:
-
tilt-downsampling
- I-DS:
-
image-downsampling
- DART:
-
discrete algebraic reconstruction technique
- CS-ET:
-
compressed sensing ET
- HR-STEM:
-
high resolution STEM
- MRI:
-
magnetic resonance imaging
- SEM:
-
scanning electron microscope
- TV:
-
total variation
- NW:
-
nanowire
- CoPc:
-
cobalt phthalocyanine
- ZnO:
-
zinc oxide
- NUFFT:
-
non-uniform fast Fourier transform
- RMSE:
-
root mean square error
References
Midgley, PA, Ward, EPW, Hungria, AB, Thomas, JM: Nanotomography in the chemical, biological and materials sciences. Chem. Soc. Rev. 36, 1477–1494 (2007)
Frank, F: Electron tomography: methods for three-dimensional visualization of structures in the cell. Springer, New York (2006)
Midgley, PA, Weyland, M: 3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography. Ultramicroscopy 96, 413–431 (2003)
Lucic, V, Forster, F, Baumeister, W: Structural studies by electron tomography: from cells to molecules. Annu. Rev. Biochem. 74, 833–865 (2005)
Medalia, O, Weber, I, Frangakis, AS, Nicastro, D, Gerisch, G, Baumeister, W: Macromolecular architecture in eukaryotic cells visualized by cryoelectron tomography. Science 298, 1209–1213 (2002)
Midgley, PA: Dunin-Borkowski: Electron tomography and holography in materials science. Nat. Mater. 8, 271–280 (2009)
Perassi, EM, Hernandez-Garrido, JC, Moreno, S, Encina, ER, Coronado, EA, Midgley, PA: 2104 10(6), 2097 (2010)
Midgley, PA, Thomas, JM: Multi-dimensional electron microscopy. Angew. Chem. Int. Ed. 53, 8614–8617 (2014)
Lepinay, K, Lorut, F, Pantel, R, Epicier, T: Chemical 3D tomography of 28 nm high K metal gate transistor: STEM XEDS experimental method and results. Micron 47, 43–49 (2013)
Haberfehlner, G, Orthacker, A, Albu, M, Li, J, Kothleitner, G: Nanoscale voxel spectroscopy by simultaneous EELS and EDS tomography. Nanoscale 6, 14563–14569 (2014)
Kwon, O-H, Zewail, AH: 4D electron tomography. Science 328, 1668–1673 (2010)
Kawase, N, Kato, M, Nishioka, H, Jinnai, H: Transmission electron microtomography without the “missing wedge” for quantitative structural analysis. Ultramicroscopy 107, 8–15 (2007)
Batenburg, KJ, Bals, S, Sijbers, J, Kubel, C, Midgley, PA, Hernandez, JC, Kaiser, U, Encina, ER, Coronado, EA, Van Tendeloo, G: 3D imaging of nanomaterials by discrete tomography. Ultramicroscopy 109, 730–740 (2009)
Van Aert, S, Batenburg, KJ, Rossell, MD, Erni, R, Van Tendeloo, G: Three-dimensional atomic imaging of crystalline nanoparticles. Nature 470, 374–377 (2011)
Leary, R, Saghi, Z, Midgley, PA, Holland, DJ: Compressed sensing electron tomography. Ultramicroscopy 131, 70–91 (2013)
Saghi, Z, Holland, DJ, Leary, R, Falqui, A, Bertoni, G, Sederman, AJ, Gladden, LF, Midgley, PA: 3D morphology of iron oxide nanoparticles with reactive concave surfaces - a compressed sensing-electron tomography (CS-ET) approach. Nano Lett. 11(11), 4666–4673 (2011)
Nicoletti, O, De la Peña, F, Leary, R, Holland, D, Ducati, C, Midgley, P: Three-dimensional imaging of localized surface plasmon resonances of metal nanoparticles. Nature 502, 80–84 (2013)
Goris, B, Bals, S, Van den Broek, W, Carbo-Argibay, E, Gomez-Grana, S, Liz-Marzan, LM, Van Tendeloo, G: Atomic-scale determination of surface facets in gold nanorods. Nat. Methods. 11, 930–935 (2012)
Buban, JP, Ramasse, Q, Gipson, B, Browning, ND, Stahlberg, H: High-resolution low-dose scanning transmission electron microscopy. J. Electron. Microsc. 59(2), 103–112 (2010)
Berkels, B, Binev, P, Blom, AD, Dahmen, W, Sharpley, RC, Vogt, T: Optimized imaging using non-rigid registration. Ultramicroscopy 138, 46–56 (2014)
Yankovich, AB, Berkels, B, Dahmen, W, Binev, P, Sanchez, SI, Bradley, SA, Li, A, ISzlufarska, I, Voyles, PM: Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts. Nat. Commun. 5, 4155 (2014)
Lustig, M, Pauly, J: SPIRiT: iterative self-consistent parallel imaging reconstruction from arbitrary k-space sampling. Magn. Reson. Med. 64(2), 457–471 (2010)
Tuma, T, Lygeros, J, Kartik, V, Sebastian, A, Pantazi, A: High-speed multiresolution scanning probe microscopy based on Lissajous scan trajectories. Nanotechnology 23, 185501 (2012)
Anderson, HS, Ilic-Helms, J, Rohrer, B, Wheeler, J, Larson, K: Sparse imaging for fast electron microscopy. Proc. of SPIE-IS&T Electronic Imaging 8657, 86570C-1-12 (2013)
Stevens, A, Yang, H, Carin, L, Arslan, I, Browning, ND: The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images. Microscopy 0, 1–13 (2013)
Macias-Montero, M, Filippin, AN, Saghi, Z, Aparicio, FJ, Barranco, A, Espinos, JP, Frutos, F, Gonzalez-Elipe, AR, Borras, A: Vertically aligned hybrid core/shell semiconductor nanowires for photonics applications. Adv. Funct. Mater. 23, 5981–5989 (2013)
Alcaire, M, Sanchez-Valencia, JR, Aparicio, FJ, Saghi, Z, Gonzalez-Gonzalez, JC, Barranco, A, Oulad-Zian, Y, Gonzalez-Elipe, AR, Midgley, JP, Groening, P, Borras, A: Soft plasma processing of organic nanowires: a route for the fabrication of 1D organic heterostructures and the template synthesis of inorganic 1D nanostructures. Nanoscale 3, 4554–4559 (2011)
Zhang, X, Burger, M, Osher, S: A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011)
Benning, M, Gladden, L, Holland, D, Schönlieb, CB, Valkonen, T: Phase reconstruction from velocity-encoded MRI measurements – a survey of sparsity-promoting variational approaches. J. Magn. Reson. 238, 26–43 (2014)
Bredies, K, Kunisch, K, Pock, T: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)
Fessler, JA, Sutton, BP: Non uniform fast Fourier transforms using min-max interpolation. IEEE Trans. Sig. Proc. 51, 560–574 (2003)
Lustig, M, Donoho, D, Pauly, JM: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007)
Saxton, WO, Baumeister, W: Three-dimensional reconstruction of imperfect two-dimensional crystals. Ultramicroscopy 13(1–2), 57–70 (1984)
Miao, J, Forster, F, Levi, O: Equally sloped tomography with oversampling reconstruction. Phys. Rev. B 72, 052103 (2005)
Acknowledgements
The research leading to these results has received funding from the European Union Seventh Framework Programme under Grant Agreement 312483 - ESTEEM2 (Integrated Infrastructure Initiative-I3), as well as from the European Research Council under the European Union’s Seventh Framework Programme (FP/2007-2013)/ERC grant agreement 291522 - 3DIMAGE. We thank JJ Fernandez for his assistance in the resolution estimation by FSC. RL acknowledges the Junior Research Fellowship from Clare College. AB and MM-M acknowledge project MAT2013-42900-P from the Spanish Ministry of Economy and Competitiveness.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
ZS designed and carried out the experiment, participated in the processing of the data and drafted the manuscript. PAM participated in the design of the experiment. MM-M and AB synthesised the nanowires. MB provided the TV inpainting code. RL helped with the CS reconstructions and interpretation of the results. All authors read and approved the final manuscript.
Additional files
Additional file 1:
Fully sampled tilt series, with 100% I-DS and 2° increment. Video of the fully sampled tilt series, corresponding to 100% I-DS (all pixels are scanned) and 2° tilt increment.
Additional file 2:
3D visualisation of the CS-ET reconstruction of the NW, using 100% I-DS and 2° tilt increment. Voxel projection visualisation and slices through the CS-ET reconstruction of the NW, obtained from the fully sampled tilt series (100% I-DS with 2° tilt increment).
Additional file 3:
Tilt series with 60% I-DS and 2° increment. Video of an undersampled tilt series, corresponding to 60% I-DS and 2° tilt increment.
Additional file 4:
Tilt series with 30% I-DS and 2° increment. Video of an undersampled tilt series, corresponding to 30% I-DS and 2° tilt increment.
Additional file 5:
Tilt series with 10% I-DS and 2° increment. Video of an undersampled tilt series, corresponding to 10% I-DS and 2° tilt increment.
Additional file 6:
TV inpainting of the tilt series with 60% I-DS and 2° increment. Video of the 60% I-DS tilt series, restored by TV inpainting. Tilt increment was 2°.
Additional file 7:
TV inpainting of the tilt series with 30% I-DS and 2° increment. Video of the 30% I-DS tilt series, restored by TV inpainting. Tilt increment was 2°.
Additional file 8:
TV inpainting of the tilt series with 10% I-DS and 2° increment. Video of the 10% I-DS tilt series, restored by TV inpainting. Tilt increment was 2°.
Additional file 9:
3D visualisation of the CS-ET reconstruction of the NW, using 30% I-DS and 6° tilt increment. Voxel projection visualisation and slices through the CS-ET reconstruction of the NW, obtained from an undersampled tilt series with 30% I-DS and 6° tilt increment.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Saghi, Z., Benning, M., Leary, R. et al. Reduced-dose and high-speed acquisition strategies for multi-dimensional electron microscopy. Adv Struct Chem Imag 1, 7 (2015). https://doi.org/10.1186/s40679-015-0007-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40679-015-0007-5