Background

Detailed understanding of the evolution of interfaces is of both scientific and practical interest in many disciplines. Scientists wish to identify unifying principles of pattern formation in processes such as solidification and aggregation [1]. Engineers are interested in controlling the evolution of interfaces and their morphologies. For example, morphological instabilities, particularly dendrite formation, can cause catastrophic failure in rechargeable batteries or lower the quality of electroplated coatings, yet they may be useful in forming porous deposits such as porous electrodes and catalytic surfaces; real-time observations provide essential information on the process by which they form [26]. Nanocrystal growth is another example where morphology control enables tuning of mechanical, electrical, and optical properties [710]. Liquid cell electron microscopy [11, 12] can address such problems. It allows us to image, in real time and with nanoscale resolution, processes such as the evolution of the growth front during electrochemical deposition and etching [1319], nanocrystal formation and dissolution [9, 2023], and bubble nucleation, growth, and migration [11, 12, 1416, 20, 21, 24] to provide insights into the mechanisms controlling morphology and mass transport.

Making the best use of the information obtained during liquid cell experiments requires quantitative measurements from each frame in a video sequence as a phenomenon of interest takes place. In particular, tracking the position of interfaces between different materials is a common first step. For example, Woehl et al. [22] studied the nucleation and growth of silver nanocrystals utilizing image processing to obtain particle count and radius as functions of time and beam conditions, and Ievlev et al. [23] used interface detection algorithms to investigate platinum nanocrystal growth properties via local growth velocity. We have therefore developed a general use interface-tracking algorithm based on image analysis needs over several different liquid cell experiments. Here we demonstrate its use, extract several quantitative measures of dynamical processes, and outline the algorithm details for easy adaptation of our open source software [25] to other experimental systems. Our workflow is illustrated with experimental data from three liquid cell experiments: copper electrodeposition [18, 19], electron beam-induced formation of gold nanoprisms [9, 21], and bubble nucleation and growth [26]. For copper electrodeposition from an acidified copper solution [18, 19], we extract the growth front location as a function of time. The data are then used to calculate the projected area and mass of the deposit, the root mean square (RMS) roughness of the interface to identify the growth habit, the local normal velocity distribution (akin to current density) to provide the relationship between geometry and point-wise growth rate, and the wavelengths associated with interface morphology, as their control is important for structured surface manufacturing. For growth and dissolution of single-crystal Au nanoparticles mediated by the electron beam [9, 20, 21], the algorithm adjusts for particle translation and extracts the particle orientation to properly determine growth kinetics. Finally, for nanobubble nucleation, growth, and migration [24, 26], we extract bubble radius and position as functions of time. The automation of the process allows for analysis of datasets with large numbers of frames.

Image processing

Several processes were combined to enable unsupervised, non-parametric image processing, and we outline the individual steps below. By non-parametric, we mean that frame-specific parameters are automatically calculated without user input. The data used for illustration were obtained as described in the experimental methods section. The open source code has been well-documented to make it accessible to new Mathematica users. A video walkthrough of the code is available in the supplement (Additional file 1: Video S1) and on YouTube [27].

Raw data pre-preparation

To perform automated analysis of image sequences, the raw data must be in a consistent, easy to modify form. We assume that the raw data are in a video format that is readable by Mathematica since, by default, this is often the case. Non-video image sequences can be handled with simple modifications to the code. When needed, Virtualdub [28] (Windows Only) and ImageJ [29] are useful open source software packages for basic video editing and format conversion. Once the raw data are in an acceptable video file format, we select a reference frame and manually define the rotation, if any, the crop indices (crop area), and the background for subtraction. It is necessary to crop out any time stamp and experimental data overlays prior to the automated processes described below, as they will interfere with image thresholding. We illustrate pre-preparation of the raw data using a movie recorded during electrodeposition of copper [19, 30], Fig. 1a. In data such as this, which includes an electrode edge, rotation is particularly useful to ensure that the edge has the same orientation, i.e., horizontal, in all the images to simplify post processing. Figure 1a is a raw image with overlaid crop indices (red disks) and rotation angle. Figure 1b is the resulting, rotated, cropped image with the time stamp removed. From here on, we shall refer to images of the form in Fig. 1b as the sub-images.

Fig. 1
figure 1

Bright-field image of copper electrodeposition in the nanoaquarium operating in a three-terminal configuration with Pt electrodes controlled by a Gamry potentiostat. Aqueous solutions of 0.1 M CuSO4 + 0.18 M H2SO4 were prepared in doubly deionized water and were not deaerated before use. The interface morphology evolution was imaged during galvanostatic deposition with a total current of 400 nA. The transmitted beam current of 0.1 nA was spread over ~2-μm diameter resulting in a dose rate of 7.5 × 106 (Gy/s) or 2 (e2-s). The liquid thickness at the edge of the silicon nitride window is fixed to 200 nm by lithography. a Example raw data of Cu electrodeposition with overlaid crop indices (red disks) and the desired angle of rotation. b The resulting, cropped, rotated image. We refer to b as the sub-image. The lateral field of view is approximately 1850 nm in a and 1000 nm in b

Image preparation—noise filtering and background subtraction

To minimize damage caused by the electron beam, it is important to operate at the minimum dose rate required to form a usable image [31, 32]. This results in images with a relatively low signal-to-noise-ratio (SNR). Since the Rose criterion requires the SNR to be about 5 for an image to be detectable by the human eye [33, 34], it is useful to take precautions to reduce the apparent noise in each frame of the video sequence while preserving edges. We achieve this reduction through background subtraction and the application of a total variation filter (TV filter). We illustrate this technique using experimental data from beam-induced gold nanocrystal formation [9, 21].

De-noising is necessary when high accuracy object extraction is desired. Since we are interested in tracking interfaces, which are seen as edges or intensity discontinuities in images, we choose an image restoration algorithm that maintains the fidelity of the edges. The TV filter, pioneered by Rudin et al. [35, 36], poses noise removal as a constrained optimization problem that assumes the noise to be Gaussian and additive (superimposed on top of real object image, or “truth image”). Figure 2a, b, and c shows, respectively, a nanocrystal sub-image, a filtered image, and the difference between the two. Notice that to the eye, the filtered version looks almost unaltered, which is desirable. The smoothing provided by this filter mitigates undesired artifacts during subsequent processing steps.

Fig. 2
figure 2

Beam-induced Au crystal growth in the nanoaquarium with transmitted beam current of 7.8 nA spread over ~2 μm diameter, resulting in a dose rate of 5.9 × 108 (Gy/s) or 160 (e2-s). Stock solutions of chloroauric acid (Sigma-Aldrich) were made by dissolving the salt in deionized (DI) water at concentration of 20 mM. The liquid thickness within the imaged region is 50–100 nm. The left column shows de-noising: a a nanocrystal sub-image, b the same image after TV filter application, and c the difference between images a and b. The second column shows background subtraction: d the image used as the background model, e the frame of interest prior to background subtraction, and f the foreground image. The right column shows edge detection: g a thresholded foreground image (after TV filter), h the extracted edges from the foreground image in g, and i the extracted edges without image preparation (undesired)

Background subtraction is useful for increasing contrast of objects in scenes imaged in a fixed laboratory frame. Several algorithms have been developed to estimate the “background model” using temporal information [37]. Carefully designed experiments to capture reference images can provide a frame or an average of a few frames for the background model. In the case of the nanocrystal of Fig. 2, for the background model, we select a frame before the nucleation of the particle. Figure 2d, e, and f (middle column) shows, respectively, the background model, the frame of interest, and the foreground image. The image difference is obtained using pixel-by-pixel subtraction of the image intensities,

$$I_{\text{F}} \left( {x,y,t} \right) = I\left( {x,y,t} \right) - I_{0} \left( {x,y} \right),$$
(1)

where \(I\left( {x,y,t} \right)\) is the intensity of pixel \(\left( {x,y} \right)\) at time t, and \(I_{0}\) refers to the background model. The foreground image is then rescaled to have intensity levels between 0 and 1 as shown in Fig. 2f, where 0 represents black in a digital image.

Thresholding and edge detection

Once noise is filtered and contrast enhanced, edge detection can be performed. Edge detection works best on a binary image. Hence, thresholding is first performed. The standard Otsu’s method [38] is utilized to convert the grayscale image into one with only two values (additional gray levels can be included when needed). This process once again casts the image analysis as an optimization problem, where now we assume that pixels can be classified into two classes, the foreground and background, with maximized separability. That is, the pixels are assumed to have a bi-modal histogram and the algorithm tests all possible threshold values, where the chosen threshold value is the one where the sum of the variances of the two classes is minimal. Figure 2g shows a binarized foreground image that was TV filtered prior to thresholding.

Once the binary image is generated, one can find the edges of the object. The edges are a direct measure of the interface as projected from the image plane. A Canny style edge detector [39] is extremely robust for images of the type depicted in Fig. 2g. Edges are defined as local maxima in the first-order directional Gaussian gradient of the intensity function. Since the image has already been denoised and binarized, the edges are extracted with great fidelity as shown in Fig. 2h. Had the image not been pre-processed with the TV filter and background subtraction, many false positives would have been identified by the edge detection (Fig. 2i).

Translation correction and object centroids

Often, image sequences contain drift and/or migration of the imaged object. When examining the growth of the object, it is convenient to fix the location of its center of mass in all frames. For example, Additional file 2: Video S2 shows the growth of a gold crystal. During the video, due to drift adjustments of the beam location, the relative position of the crystal has changed. Here, we present a means for the translating a single object to fix the location of its center of mass in all the frames. As part of this process, we track the location of the object’s centroid as a function of time. The code can be extended to systems of multiple dynamic objects via particle tracking algorithms such as the methods developed by Crocker and Grier [40] that are available as open source code for several computation platforms. This software relates the positions of entities between frames to statistically identify the displacement of individual entities and establish their track (path in time). For simplicity, here, we will only extract the positions of particles subject to translation.

We consider the nanocrystal growth from Fig. 2 and the corresponding video sequence (Additional file 2: Video S2). Once the binary image is generated (Fig. 2g), the centroid of the foreground pixels is identified for each frame according to the formula

$$\left( {x_{c,i} ,y_{c,i} } \right)_{\Upomega } = \left( {\frac{1}{n}\sum\limits_{\Upomega } {x_{i} } ,\frac{1}{n}\sum\limits_{\Upomega } {y_{i} } } \right),$$
(2)

where Ω is the region of pixels identifying the object (white pixels in Fig. 2g), n is the number of pixels in Ω, the (x, y) pair denotes the pixel location, and i is the frame number. By shifting the location of the particle in each frame using the linear transformation

$$I_{\text{Mapped}} \left( {x,y,t} \right) = I\left( {x - x_{c} ,y - y_{c} ,t} \right),$$
(3)

we end up with the centers of mass in each frame located at {0, 0}.

Growth front tracking during electrodeposition

The first application of our image processing algorithm is to extract the growth front morphology during copper electrodeposition. Copper electrodeposition experiments were carried out with our custom-made liquid cell, the nanoaquarium [41], operating in a three-terminal configuration with Pt electrodes controlled by a Gamry potentiostat. The liquid cell was filled with an aqueous solution of 0.1 M CuSO4 + 0.18 M H2SO4, prepared in doubly deionized water and not deaerated before use. The interface morphology evolution was recorded using bright-field imaging conditions during galvanostatic (constant current) deposition. Data were recorded at video rate (30 frames per second) in a Hitachi H-9000 TEM at 300 kV. In these types of experiments, liquid cell electron microscopy provides the surface evolution in real time and with nanoscale resolution [11, 12, 14, 19] and the aim of the quantitative analysis is to obtain insight into the mechanisms leading to stable and unstable growth fronts under different conditions.

The algorithm described here robustly extracts quantitative measures of the growth front morphology. Figure 3a and supplemental Additional file 3: Video S3 show the data recorded at selected times during galvanostatic deposition under conditions (high current density) that resulted in layer growth with a diffusive instability. Once we have extracted the edge, we can post-process quantitative data to obtain various figures of merit describing the growth. The simplest metrics are the average layer height, the maximum height, and the minimum height as functions of time (Fig. 3b). We see an approximately linear advancement of the average height (after an initial transient), as expected from galvanostatic conditions where the amount of material deposited per second should be constant. We also see a transition in growth mode around Frame 12. Here, the minimum height becomes constant (no additional material is being deposited at the base of the valleys) and the maximum height grows at a higher rate to maintain the average growth speed mandated by the constant current process.

Fig. 3
figure 3

Bright-field image of Cu electrodeposition in the nanoaquarium using the same electrochemical configuration (400 nA current), electrolyte, beam conditions and liquid thickness as Fig. 1. a An extracted growth edge from copper electrodeposition. Selected frames from Video S3 are shown. b Maximum, minimum, and average growth height as functions of frame number. c RMS roughness of the growth height as a function of time. d Contours of the autocorrelation function of the surface roughness as a function of spatial shift and frame number. e A heat map of the normal growth speed

We can further describe the interface by its amplitude and wavelength. The RMS roughness,

$$w\left( {t,L} \right) = \sqrt {\frac{1}{L}\sum\limits_{i = 1}^{L} {\left[ {h\left( {i,t} \right) - \bar{h}\left( t \right)} \right]^{2} } } ,$$
(4)

is a parameter used in kinetic roughening theory [42, 43] to characterize surface morphology. In the above, L is the length of the selected domain, h (i, t) is the height at location i and time t, and \(\bar{h}\left( t \right)\) is the average height at time t. Figure 3c depicts \(w\left( {t, L} \right)\) as a function of t when L ~1 μm is equal to the size of the observation window.

The lateral wavelength of the interface can also be extracted as a function of time. In this particular case, the wavelengths observed are all small (because they must be on the order of the window size). We use the cross-correlation of the interface with itself

$$R\left( {t,u} \right) = \frac{{\sum\nolimits_{i = 1}^{L - u} {\left( {h\left( {i,t} \right) - \bar{h}\left( t \right)} \right)} \left( {h\left( {i + u,t} \right) - \bar{h}\left( t \right)} \right)}}{{\sum\nolimits_{i = 1}^{L} {\left[ {h\left( {i,t} \right) - \bar{h}\left( t \right)} \right]^{2} } }}$$
(5)

to identify the dominant wavelengths, if any. In the above equation, u is the shift in pixels, 0 < u < L, and L is the image width. R (t, u) will always have a peak of 1 when u = 0. If additional peaks are present, say at u = λ 1, λ 2, …, we conclude that the \(\lambda_{i}\) are wavelengths. Figure 3d depicts contours of the cross-correlation R (t, u) as a function of the shift u and the frame number t. In all the frames, R (t, u) decays monotonically as u increases from zero, indicating the absence of periodic spatial behavior. This behavior is consistent with previous observations of the evolution of electrodeposited surfaces [43].

The morphology during electrodeposition is directly controlled by the normal current density at any given position along the interface [44, 45], which is in turn, related to the normal local speed of the interface via a mass balance or Faraday law (with appropriate assumptions about the deposition efficiency). It is thus desirable to obtain the normal velocity of the extracted edge [46]

$$v_{N} \left( {i,t} \right) = \mathop {\lim }\limits_{\Updelta t \to 0} \left( {\frac{{S_{t + \Updelta t} \left( {i_{N} } \right) - S_{t} (i)}}{\Updelta t}} \right),$$
(6)

where v N (i, t) is the normal velocity at position i and time t along the temporally evolving interface, S t . The velocity is determined by the vector distance between the point of interest, i, and the point that lies along the normal to S t (i), called i N , on the interface \(S_{t + \varDelta t}\). Essentially, this problem is reduced to finding the distance from a point on the current surface and the point of intersection between the normal to S t and the surface after it has evolved by the time interval Δt. Figure 4a illustrates the process when the surfaces (curves) are described with continuous functions.

Fig. 4
figure 4

Illustration of normal velocity extraction for a a smooth continuous surface, b a discretized surface specified with pixels and approximated with a least square fitted curve, and c the minimum distance approach. Figure 3a and e presents an example of this analysis

When processing experimental data, the situation is more complicated. Instead of a smooth curve, we have a noisy 2D point cloud (Fig. 4b). To address the noisy data, we have implemented two methods for extracting the normal direction to the surface. The first method is based on a moving least square algorithm [47]. To construct a normal to the surface at point i, we fit the experimental data locally, around the point i, with the m-degree polynomial \(P_{m} = \sum\nolimits_{j = 0}^{m} a_{j} x^{j}\), where we maintain the laboratory frame of reference. The coefficients a i are calculated by minimizing the error

$$E^{2} = \sum_{{k = i - \frac{n - 1}{2}}}^{{i + \frac{n - 1}{2}}} {\left[ {S_{t} \left( k \right) - P_{m} \left( {x_{k} ,a_{0} ,a_{1} , \ldots ,a_{m} } \right)} \right]^{2} } ,$$
(7)

where n is an odd integer number of points along the extracted edge’s point cloud centered about i. This method works best for low noise data and when the frame rate is low (i.e., the interface moves more than a few pixels per frame).

An alternative method exploits the fact that the shortest distance from a point to a surface/curve lies along the normal to the curve. To construct a normal to the surface St that intersects point j on surface St+Δt, we seek point i * on St such that the distance between points i and j is minimized. Figure 4c illustrates this process. In fact, Mathematica has built-in routines to quickly find which point from a point cloud (ordered or not) is closest to a point of interest. This proved to be a very effective and fast means for the extraction of point-wise normal velocity, especially in the presence of noise.

We applied the velocity routine to the evolving interface of Additional file 3: Video S3. Figure 3e shows the resulting “heat map” (contour plot) of the normal speed in space (along the interface) as a function of time. Initially, we see uniform distribution of speeds and therefore uniform current density. As the roughness develops due to natural fluctuations, the current density becomes less uniform locally, leading to continued amplification of the roughness.

Facet-dependent growth and dissolution rates in nanoparticles

Size and morphology control of nanocrystals is essential to tailor the physical and chemical properties of the resulting material [7, 8]. Liquid phase synthesis is a convenient, low-cost and versatile technique for fabricating well-controlled nanocrystals, since solution chemistry can be tuned to give specific morphology. A fundamental understanding of the driving force and principles that govern morphologies during nucleation and growth is essential to optimize existing synthesis methods and to discover novel methods. Experiments on the dissolution and growth of gold nanorods were carried out with the nanoaquarium in an FEI Quanta 600 FEG Mark II SEM with a transmission detector. The aqueous solution contained gold nanorods [48] with a trace amounts of the surfactant cetrimonium bromide (CTAB) and HAuCl4 at pH ~7. The devices were filled under atmospheric conditions and the solution was not deaerated. Production of radiolytic species by the electron beam, as described in Schneider et al. [20], causes the Au nanorods to either grow or dissolve. During dissolution, we hypothesize that the radicals produced during water irradiation react with the CTAB, whose role is to stabilize the nanorods; in the absence of CTAB, nanorods are known to be unstable and dissolve [49, 50]. Analysis of morphology and growth dynamics at specific facets provides information about the atomic level processes that control shape [51].

Additional file 4: Video S4 shows beam-induced dissolution of gold nanorods. Focusing on a single nanorod undergoing dissolution, we use the vertices of the minimum bounding box to obtain both the particle’s aspect ratio and its orientation as functions of time. Figure 5a shows the image sequence of the nanorod after binarization and with the superposition of the minimum bounding box vertices. The smallest bounding box is used due to the nearly rectangular shape of the particle in projection. We use Toussaint’s rotating caliper method [52] to rotate the box around the object and extract the smallest bounding rectangle. Alternative geometric shapes and minimum area algorithms can be selected for other particle morphologies. Many of the convex hull algorithms, including the minimum bound box, are available in Mathematica. See the supplemental Code 1. Figure 5b, c, and d depicts, respectively, the particle’s aspect ratio, perimeter, and rotation from the initial orientation as functions of time. Interestingly, the etching is not isotropic, as is evident from the non-monotonic variation of the aspect ratio. The particle first etches at the long facets, increasing its aspect ratio, and only later do the tips etch and the particle becomes isotropic. This is likely related to the distribution of ligands (CTAB) that remain from the manufacturing process [48]. CTAB binds more strongly to the long facets. During growth, this effectively reduces access of reducible species to the long facets and encourages the formation of rods [53, 54]. Conversely, we hypothesize that as the beam creates an oxidizing environment and degrades the surface bound CTAB [20], the long edge is affected first, resulting in the observed anisotropic etching. This is somewhat analogous to growth studies [51] that suggest that the position and dynamics of ligands on a nanoparticle surface determine the growth rate for each facet. The local pH (driven toward acidic by the beam [20]) is also likely to play a role in the degree of anisotropy during the etching process [48]. We can track the perimeter of the particle (Fig. 5c) and its rotation (Fig. 5d). The rotation can be used along with the center of mass to extend the code to extract the point-wise normal etching (or growth) velocity similar to the analysis in [23].

Fig. 5
figure 5

Beam-mediated etching of gold nanorods (nominally 50 × 80 nm) suspended in water (pH ~7) with trace amounts of the surfactant cetrimonium bromide (CTAB) and chloroauric acid. Imaging was carried with a FEI Quanta 600 in STEM mode (30 keV) beam currents of 0.07 nA, beam radius of ~1 nm resulting in a time average dose rate of 7.4 × 106 (Gy/s) or 2 (e2-s). a An image sequence of a gold nanorod dissolution. The images are binarized and superposed with the minimum bounding box (green). The box vertices are shown in red. bd Particle aspect ratio, perimeter, and inclination angle from initial orientation, respectively, as a function of frame number

Bubble nucleation, growth, and migration

The dynamics of gas–liquid interfaces plays an important role in many physical processes such as high efficiency phase change heat transfer. In recent years, the existence and stability of nanobubbles have been debated [5558]. Recently, atomic force microscopy (AFM) measurements have confirmed the existence and unraveled the geometry of nanobubbles [55, 56, 58, 59]. However, such measurements have limited spatial resolution and there is an additional concern that the mechanical perturbation applied by the cantilever is too invasive. Electron microscopy imaging of nanobubbles provides information that complements AFM measurements.

Here, we consider the motion of radiolytically formed nanobubbles confined in a tapered conduit. Given the bubbles’ size, both the capillary and Bond numbers are essentially zero. In this fluid dynamical regime, motion and geometry of the nanobubbles are governed by three-phase contact line wetting and de-wetting. Identifying the interfaces of single bubbles and tracking their trajectories is essential in obtaining the quantitative information needed to model contact line dynamics. In this case, both the contact line velocities and trajectories of the centroids are of interest.

Additional file 5: Video S5 shows the nucleation, growth, and migration of a nanobubble. The bubble is shown at a particular instance in Fig. 6a (left). Figure 6b depicts the edge of the bubble at various times. We first extract the effective radius of the bubble by finding the circular area equivalent radius from the binarized image

$$r = \sqrt {\frac{{A_{{Binarized}} }}{\pi }} .$$
(9)

The area can be obtained by counting the number of pixels in the binary convex hull of the bubble object. Figure 6c depicts the bubble radius as a function of the frame number. To track the bubble’s location throughout the growth process, instead of taking the area centroid, we account for the three-dimensional shape of the bubble by taking the detected electron intensity-weighted centroid, where each pixel is multiplied by its intensity value. This serves to find the centroid most closely aligned with the center of mass of the vapor bubble, while using the two-dimensional projection.

Fig. 6
figure 6

Nanobubble formation in an aqueous solution of gold nanorods with a measured pH ~7 and trace amounts of the surfactant cetrimonium bromide (CTAB). Imaging was performed with at beam current ~1 nA and beam radius ~2 μm resulting in a dose rate of 7.5 × 107 (Gy/s) or 20 (e2-s). a Bubble radius as a function of frame. b Comparison of raw, binary, and masked images. c Bubble intensity weight centroid (squares) with bubble edge outline, color-coded in time

Data processing and analysis tutorial

For each of the three examples given above, i.e., growth front tracking during electrochemical deposition, nanoparticle dissolution, and bubble motion, an overview is described in the supplemental Additional file 6: Code S1. This is available on GitHub [25] with an accompanying video tutorial Additional file 1: Video S1. Fully automatic and unsupervised analysis of the three sample datasets presented in this paper is available in the supplemental Additional file 7: Code S2, Additional file 8: Code S3, Additional file 9: Code S4, also maintained on GitHub [25].

Conclusions

Liquid cell electron microscopy is a powerful technique with many applications to dynamic processes in liquid media. The extraction of quantitative measures is essential for proper interpretation of the data. However, typical experiments generate very large quantities of data, which can be a challenge to analyze quantitatively. We have presented a workflow that is suited to analyze typical data of the type collected during liquid cell electron microscopy. We chose Mathematica as the platform to implement the various procedures because of its ease of use, availability of numerous built-in algorithms for image processing, automation, and flexibility. We hope that the tools presented here and the tutorials provided in the supplement will be useful to the microscopy community for diverse applications of data analysis for dynamical processes both in liquid cell experiments and for in situ microscopy in general.