Abstract
Quantitative phase imaging (QPI) has emerged as a valuable tool for biomedical research thanks to its unique capabilities for quantifying optical thickness variation of living cells and tissues. Among many QPI methods, Fourier ptychographic microscopy (FPM) allows long-term label-free observation and quantitative analysis of large cell populations without compromising spatial and temporal resolution. However, high spatio-temporal resolution imaging over a long-time scale (from hours to days) remains a critical challenge: optically inhomogeneous structure of biological specimens as well as mechanical perturbations and thermal fluctuations of the microscope body all result in time-varying aberration and focus drifts, significantly degrading the imaging performance for long-term study. Moreover, the aberrations are sample- and environment-dependent, and cannot be compensated by a fixed optical design, thus necessitating rapid dynamic correction in the imaging process. Here, we report an adaptive optical QPI method based on annular illumination FPM. In this method, the annular matched illumination configuration (i.e., the illumination numerical aperture (NA) strictly equals to the objective NA), which is the key for recovering low-frequency phase information, is further utilized for the accurate imaging aberration characterization. By using only 6 low-resolution images captured with 6 different illumination angles matching the NA of a 10x, 0.4 NA objective, we recover high-resolution quantitative phase images (synthetic NA of 0.8) and characterize the aberrations in real time, restoring the optimum resolution of the system adaptively. Applying our method to live-cell imaging, we achieve diffraction-limited performance (full-pitch resolution of \(655\,nm\) at a wavelength of \(525\,nm\)) across a wide field of view (\(1.77\,mm^2\)) over an extended period of time.
Similar content being viewed by others
Introduction
Quantitative phase imaging (QPI) is a powerful optical tool for visualizing and quantifying optical thickness variation of unlabeled transparent biological samples without the need for specific exogenous contrast agents [1,2,3]. Various QPI techniques have been developed in the past decades and achieved great success in biomedical applications. These techniques include digital holography (DH) [4,5,6], phase shifting interferometry [7, 8], lateral shearing interferometry [9, 10], transport-of-intensity equation (TIE) [11,12,13], differential phase contrast (DPC) [14,15,16], Fourier ptychographic microscopy (FPM) [17,18,19,20], among others. Many of these techniques are performed using a modified microscope system. The imaging throughput follows the law of Lagrange invariant [21, 22], leading to the unavoidable trade-off between imaging resolution and field-of-view (FOV). Among them, FPM is a promising solution to achieve high-throughput QPI by synthesizing a wide-field, high-resolution complex image from multiple low-resolution images captured under angle-varied illumination. It has been demonstrated for long-term label-free observation and quantitative analysis of large cell populations without compromising the spatial and temporal resolution [19, 23,24,25,26]. Free from the adverse effects of staining reagents on cell viability and signaling, high-throughput QPI can find important applications in drug discovery, personalized genomics, cancer diagnosis, and drug development [27,28,29,30].
Nevertheless, high spatio-temporal resolution imaging over long-time scales (hours to days) and across a wide FOV remains a critical challenge [31,32,33], due to the spatially non-uniformity and temporally varying system aberrations. The contributing factors of this challenge include the optically inhomogeneous structure of biological specimens, mechanical instability of the microscope, and thermal fluctuations of the environment [34]. Their combined impact can significantly degrade the achieved resolution and quality of the imaging platform. Moreover, the aberrations are sample- and environment-dependent, and thus they cannot be compensated using a fixed optical design [35]. Therefore, an adaptive method for the dynamic correction of wavefront distortions over a large FOV is highly desired in QPI systems.
Adaptive optics (AO) was originally developed for astronomical telescopes [36, 37]. This technology can detect the aberrations introduced by the atmosphere and correct them using a deformable mirror. Nowadays, the AO-based technology has found extensive applications in optical imaging [38, 39], particularly in confocal and multiphoton microscopy, as it facilitates three-dimensional volumetric imaging of thick biospecimens [40, 41]. One major drawback of the AO technology is the need for additional hardware, such as Shack-Hartman wavefront sensor, spatial light modulator, and deformable mirror. In contrast, with the help of inherent data redundancy, FPM enables direct aberration recovery and correction post measurement. As a result, it can reconstruct a high-quality image without resorting to AO hardware [42,43,44], offering a solution as “computational” AO [45, 46]. However, the data redundancy requirement (generally dozens of images) in conventional FPM significantly limits its imaging speed, resulting in low temporal throughput. Moreover, the adaptive wavefront correction in FPM is generally designed for imaging absorptive samples. Their applicability to the QPI of transparent phase objects remains to be explored.
In this article, we report a computational AO method to tackle the temporally- and spatially-varying aberrations in live-cell imaging. Our technique, termed AO-QPI, acquires the intensity images under the annular NA-matched illumination configuration. Then, the imaging aberration and cell phase are simultaneously recovered with a ptychographic reconstruction algorithm. It should be noted that only six intensity images are required for the reconstruction, performing the high frame rate of imaging. To demonstrate the validity and practicability of AO-QPI, we performed long-term live-cell imaging using our prototype setup, achieving diffraction-limited performance across a FOV of \(1.77\,mm^2\), a time of \(51\,hours\) and a frame rate of \(16.66\,Hz\) (it can reach the camera speed of \(100\,Hz\) as the sliding window scheme is adopted). To the best of our knowledge, it is the first time to perform dynamic aberration correction on QPI to secure high-quality imaging performance over time, especially at such a high spatiotemporal bandwidth.
Materials and methods
Optical setup and principle
The reported AO-QPI system was built using a standard bright-field microscope [IX73, Olympus, Japan, see Fig. 1(a)] with a programmable LED array as the illumination source. During the image acquisition process, the LED elements were sequentially turned on to illuminate the specimen from different angles and the corresponding low-resolution images are acquired using a 10\(\times\), 0.4 NA objective lens (UPLSAPO10\(\times\), Olympus). The specimen phase as well as the wavefront aberrations were coupled in the captured intensity images, which need to be recovered through a nonlinear optimization algorithm. Figure 1(b) shows the workflow of the reconstruction process, which takes the cell specimen as an example for demonstration. The initial guess of the specimen is obtained by a coherent version of DPC deconvolution [15]. The captured images are then successively adopted to impose the intensity constraints for jointly updating the specimen and the pupil function. Here, the pupil function is characterized by the Zernike polynomials for an efficient and accurate recovery. When the convergence condition is met, the specimen phase and the aberration are recovered from the raw images. As is shown in Fig. 1(d), through the reconstruction process mentioned above, the intracellular details are resolved with high resolution after \(\sim\)50 iterations.
It seemed that the reconstruction process follows a similar framework to FPM. However, compared with traditional FPM methods which usually require dozens of frames as raw data, the AO-QPI has two significant improvements for enhancing the imaging efficiency, including the optimization of illumination strategy and aberration correction algorithm. By integrating the two innovations, AO-QPI is able to realize dynamic phase imaging and aberration correction with one order of magnitude fewer measurements.
Illumination mode optimization
In the traditional FPM imaging process, a dense sampling strategy in the Fourier domain is adopted to meet the requirement of the spectrum overlapping rate [47]. Here, to reduce the data acquisition time and improve the imaging speed, we adopted an annular NA-matched illumination scheme (i.e., the illumination NA strictly equals to the objective NA [23]). Such an illumination condition has been verified to be significant for phase retrieval in asymmetric-illumination-based QPI methods [16, 48, 49]. For the imaging system with non-negligible aberrations, considering that the object spectrum and imaging aberrations are coupled as two inseparable components in the captured images, the matched illumination condition is hence equally crucial for imaging aberration recovery, which has been validated in Supplementary Note 3. Moreover, based on the annular matched illumination, we further explored the minimum data redundancy required for aberration recovery (see Supplementary Note 4). It is found that only six images could satisfy a successful ptychographic reconstruction and more raw images do not significantly improve the imaging quality. Therefore, we carefully adjusted the height of the LED array to make the six selected LED elements (see Fig. 1(c)) accurately satisfy the matched illumination condition. In the imaging process, we cyclically turn on the six LED elements to acquire the raw dataset for dynamic phase imaging. Benefiting from the successful reconstruction with such a small number of raw images, the adaptive correction is sustained over the high-speed imaging process to defy the temporal-varying aberrations in real time. Moreover, we also apply a sliding window along the whole captured image sequence, and thus the effective phase imaging frame rate is further raised to the shutter speed of the camera.
(a) The AO-QPI prototype platform where we adopt a programmable LED array to replace the conventional Köhler illuminator as the light source. (b) The workflow of the reconstruction process of AO-QPI. (c) Schematic diagram of the illumination system. The LED array is adjusted to an appropriate height so that 6 LED elements in the array provide the NA-matched illuminations. (d) The comparison of phase reconstruction with/without AO-QPI. The intensity cost function gradually reduces after enabling AO-QPI and finally converges after \(\sim \,100\) iterations
Aberration recovery algorithm
The common reconstruction algorithms of FPM [42, 50, 51] are derived from several versions of the ptychographic iterative engine (PIE) [52,53,54], which reconstruct the complex amplitude of specimens by alternating iterations in the spatial and frequency domains. The updating process for the specimen spectrum is given as follows,
where, O is the spectrum of the specimen, P is the pupil function of the optical system, \(\Phi _u\) and \(\Phi _e\) are the Fourier spectrum with and without the intensity constraint respectively, and \(\alpha\) is the updating step size. Moreover, benefiting from the redundancy of captured raw images, the pupil function can be reconstructed in a similar way as shown in Eq. 2, where the pupil function and the spectrum are reversed. The updating formula is first proposed in the method of EPRY [42], which is a modified version of the ePIE method [55] applied in FPM.
Here, we report a revised pupil function updating formula, as shown in Eq. 3. The term of normalized spectrum modulus \({|O|^{2}}/{|O|^{2}_{max}}\) is omitted in the new formula. For the extremely high energy of the DC term in the bright-field spectrum, the term \({|O|^{2}}/{|O|^{2}_{max}}\) is close to zero and will make the pupil function P hard to be effectively updated. A similar normalized term was adopted for the update of the probe function in PIE [52] and ePIE [55] to defy the detector noise. However, it is not applicable to FPM because the object function O in the update formula is transformed from the spatial domain to the frequency domain. The comparison of the reconstruction results using Eq. 2 and Eq. 3 are provided in Supplementary Note 1.
In practical implementations, Eq. (1) and Eq. (3) are applied to alternatively update the object function and the pupil function until the convergence condition is satisfied. However, the joint estimation of both object and pupil requires more raw datasets for obtaining the correct solution. In order to achieve aberration correction while ensuring imaging efficiency, the prior knowledge of the common aberrations is introduced to the reconstruction process. Here, we adopted the Zernike polynomials \(Z_n^m\) [44, 56, 57] as a suitable basis to express pupil aberration. Therefore, the degrees of freedom in solution space are reduced from a two-dimensional matrix to a small number of coefficients associated with the dominant Zernike modes, which is beneficial for the reconstruction with limited raw images. More details and improvements of this implementation are further discussed in Supplementary Note 4, which is adopted as a way to reduce the requirement for data redundancy.
Samples for testing
We tested our method on a standard microlens array, a phase resolution target and HeLa cell cultures in Petri dishes. The micro lens array (18-00036, SUSS, Microoptics, refractive index (RI) of 1.45) was placed on a cover glass for imaging. The curvature radius is 9670 \(\upmu\)m and the pitch is 130 \(\upmu\)m for this target. Its theoretical maximum height is 0.22 \(\upmu\)m, corresponding to a maximum phase of \(\sim\)1 rad. The adopted quantitative phase microscopy target (QPT\(^{TM}\), Benchmark Technologies Corporation, USA) provides an optical path difference for QPI by increasing the thickness of the bars on a glass plate. The bars are made of the acrylate polymer (The refractive index is 1.52) on Corning Eagle XG Glass including groups 6 - 10 with the feature height of 150 nm. The cells were cultured in a 37\(^{\circ }\)C incubator with 5\(\%\) CO2 for observation to further demonstrate the performance of the proposed method on biospecimens.
Computation platform used for AO-QPI
The AO-QPI reconstruction algorithm was performed based on MATLAB software (MATLAB R2018b) with a laptop equipped with a 2.60 GHz central processing unit (Intel Core i7-10750H) and 16 GB of random-access memory. As we mainly focused on the proof-of-concept, the image reconstruction speed was not optimized. The time required for the computation processes (including 50 iterations to solve the phase and aberration simultaneously) of the raw intensity images (2048 \(\times\) 2048 \(\times\) 6) is \(\sim\) 20 minutes. In the computation processes, we divided each full-FOV raw image (2048 \(\sim\) 2048 pixels) into 10 \(\times\) 10 sub-regions (256 \(\times\) 256 pixels each), with a 60-pixel overlap on each side of neighboring sub-regions. After completing the reconstruction of each sub-region, a full-FOV result is generated by using an alpha-blending stitching method. The consuming time for the reconstruction could be further reduced by implementing GPU acceleration.
Results and discussion
Optimal illumination scheme for AO-QPI
In this section, we presented our simulation study and experimental results to validate the illumination scheme of AO-QPI for accurate and efficient aberration characterization. The simulation parameters were chosen to realistically model the experimental platform. Figure 2(a) shows the input specimen phase and imaging aberration for simulation, where a micro lens is adopted as the simulated specimen and the aberration is the sum of several Zernike polynomials. In Supplementary Note 2, we have verified that the annular illumination could replace the traditional FPM illumination mode for an effective ptychographic reconstruction. Based on the annular illumination, we further optimize the illumination scheme through simulations in this section. Figure 2(b) demonstrates the necessity of the matched illumination condition for the accurate characterization of both phase and aberration. We first performed the reconstruction using 12 raw images with the matched illumination, and the obtained results are almost identical to the input, as shown in the first row of Fig. 2(b). For a fair comparison, we also adopted the same number of raw images for reconstruction, but the illumination NA is half of the objective NA in the second row of Fig. 2(b). In this case, the recovered aberration has a similar distribution but weaker amplitude compared to the input. It indicates that there are still uncorrected aberrations that could degrade the image recovery. The corresponding recovered phase shows a circular outline, which is a typical result obtained by high-pass filtering. This is due to the fact that the low-frequency information is lost in the imaging process without the matched illumination condition. More systematic analyses are provided in Supplementary Note 3. To further explore the minimum dataset required for a successful aberration characterization, we provide the convergence curves as well as the reconstructed pupil aberrations using a different number of raw images in Fig. 2(c) and 2(d). Here, we adopted the algorithm of Zernike mode constraint to improve the performance with limited number of raw images. For comparison, the comparable results with the method of EPRY are provided in Supplementary Note 4. As shown in Fig. 2(d), when only three raw images are adopted, the aberration error (RMSE) curve initially descends in the first several iterations but rebounds rapidly, and finally converges to a stable value with some fluctuations. The obtained result in Fig. 2(c) completely deviates from the input which can be attributed to the insufficiency of data redundancy. The increase in data redundancy could steer the algorithm to better convergence but the results are still unsatisfactory. As the number of raw images increases to more than six, the RMSE curve achieves a solid decline and finally converges to near zero, which indicates that we have successfully characterized the aberration.
Reconstructed phase and aberration under different illumination modes. (a) Simulated specimen phase (above) and aberration (below). (b) Reconstructed results with and without matched illumination, including illumination modes of the LED array (the red circle is NA of the objective lens), recovered aberrations, recovered phase and corresponding phase error. (c) Reconstructed results with 3, 6, 9 images. (d) Convergence curves of the reconstruction with different number of images. (e) Reconstructed phase of microlens in the full FOV. (e1) Experiment result of ROI using full 121 images. (e2) Experiment result of ROI using 6 images under the matched illumination. (e3) Experiment result of ROI using 37 images without the matched illumination
To further validate the forementioned conclusions, we performed an experiment on the micro lens array, which was deliberately placed at an out-of-focus position. The LED array was adjusted to an appropriate height so that the six LED elements in the array can exactly provide the annular matched illuminations for imaging (the coordinates of these LED elements in the array are (0, ±5), (3, ±4), (-3, ±4), in Fig. 2(e2)). We first sequentially lit 121 LED elements distributed in an array to acquire the raw dataset. In Fig. 2(e1), all raw images are utilized for a typical FPM reconstruction and the imaging aberration as well as the phase of a micro lens are successfully recovered. The obtained aberration is mainly composed of a spherical aberration caused by defocusing, and the remaining part represents the error of the imaging system. Then we excluded most of the raw images and only adopted six of them to satisfy the matched illumination condition for reconstruction and the results are provided in Fig. 2(e2). As we have demonstrated in simulations, the aberration and phase obtained in this way are almost identical to the standard result. Figure 2(e3) provides the reconstructed results using 37 raw images with the illumination angles smaller than the objective NA. As with the results in the simulation, the aberration and phase fail to converge to the correct solution in this case. It indicates that more raw datasets will not help the correct solution without the matched illumination. Based on the simulation and experiment results, it can be concluded that the matched illumination configuration provides an efficient way for the accurate reconstruction of both sample phase and pupil aberration, and at least six raw images could provide sufficient data redundancy for the correct solution under this condition.
AO-QPI on a quantitative phase microscopy target
A quantitative USAF phase resolution target was imaged to quantify the AO-QPI’s capacity to restore the diffraction-limited resolution in the presence of imaging aberration. To better demonstrate the aberration correction performance, the QPT was deliberately placed at an out-of-focus position to magnify the imaging aberrations. We first perform a standard FPM method without aberration correction to perform the quantitative phase reconstruction using a total of 121 raw images. However, due to the uncorrected imaging aberrations, significant defocus artifacts appear on the recovered phase (Fig. 3(a)), which severely degrade the obtained imaging resolution. Then, the method of EPRY was performed using the same set of raw images, which is a common algorithm for aberration correction in FPM. With the characterization of the imaging aberrations, the desired phase map is successfully obtained, as is shown in Fig. 3(b). The minimum center-to-center distance of the line profile across the bars demonstrate the conservatively estimated resolution of 690 nm, which is consistent with theoretical resolution of the imaging system (655 nm, 0.8 NA). The proposed AO-QPI was equally tested for the same recovery of both the imaging aberration and specimen phase as shown in Fig. 3(c). By integrating the annular matched illumination scheme and the aberration correction algorithm based on Zernike mode constraint, AO-QPI could provide the comparable result with only six raw images. Moreover, through the comparison of the reconstructed aberrations in Fig. 3(b) and 3(c), we can see that a real aberration could be appropriately characterized by the Zernike modes we adopted in the reconstruction algorithm. Through the experimental results above, we can see that the method of AO-QPI provides an efficient way to remove the aberrations and achieve the diffraction-limited performance of the imaging system.
The experimental results on a quantitative phase microscopy target. (a) The reconstructed results with full 121 images with the aberration correction, including the illumination mode of the LED array (the red circle is NA of the objective lens), recovered aberration, recovered phase and corresponding phase profile. (b) The reconstructed results with full 121 images using the EPRY method. (c) The reconstructed results with 6 images using the AO-QPI method
AO-QPI on HeLa cell cultures for a full-field imaging
In this section, the AO-QPI method was applied for imaging HeLa cell cultures across a full FOV of 1.77 \(mm^2\). Although a flat-field microscope objective was adopted in the experiment, the optical imaging tends to exhibit aberrations that vary across the field. To defy the spatially-varying aberrations, we divided the full FOV into 10 \(\times\) 10 sub-regions and performed the AO-QPI reconstruction on each region independently. In Fig. 4(b)-4(g), we provide the recovered cell phase of six selected regions, as well as the recovered aberrations. In the provided phase maps, the subcellular structures are revealed with high-quality imaging performance, which could be adopted to proof that the aberrations within each local FOV have been effectively characterized and corrected. To further demonstrate the necessity of our method, we also compared phase maps reconstructed with and without the aberration correction. Figures 4(f1)-4(g1) are the zoomed-in views of cells in the red and blue boxes of Fig. 4(f) and 4(g). With the successful characterization of the aberration, the punctate ribosome structures inside the cells can be clearly distinguished with the resolution approaching the theoretical value of the imaging system (see Fig. 4(f3) and 4(g3)). In Fig. 4(f2) and 4(g2), the comparable results are reconstructed without the aberration correction except for an auto-refocusing algorithm [58] to remove the effect of defocus. It can be seen that the subcellular structures with a size close to the diffraction limit have been completely smoothed out due to the uncorrected aberrations. These results demonstrate the capacity of our approach to sustaining high-quality imaging performance over a large field, suggesting that it provides a powerful tool for high-throughput biological analysis. More dynamic results to further verify this point can be seen in Visualization 1.
Full-FOV high-resolution reconstruction of living HeLa cultures based on AO-QPI. (a) Reconstructed full-FOV phase map. (b-g) Zoomed view of the reconstructed cell phase in 6 regions as well as the recovered aberrations. (f1)-(f2) (g1)-(g2)Comparison of recovered phase maps with and without aberration correction in red and blue boxes of (f) and (g). (f3) (g3) Profiles of the cell phase in (f1)-(f2) and (g1)-(g2)
AO-QPI on HeLa cell cultures over an extended period of time
The AO-QPI method was further performed for continuous observation of dynamic cell activities for up to dozens of hours, which is a challenging biomedical applications for microscopy. The adverse factors for long-term imaging can be attributed to thermal fluctuations of the environment [59], mechanical instability of the system [60], and evaporation from the cell media. All these factors will lead to the temporal-varying aberrations across the whole imaging process and significantly degrade the imaging quality. To solve these problems, we apply the AO-QPI method to provide the adaptive aberration correction and continuously decipher the dynamic cell activities with high quality across 51 hours. The obtained time-lapse movie is provided in Visualization 2. Figure 5(a) shows the full-FOV phase map stitched together from the two frames at hour 0 and hour 51. After the reproduction over dozens of hours, the cell populations went from being sparsely distributed to almost covering the full FOV. Figure 5(a1) to 5(a5) are the five frames of the zoomed-in cell phase maps extracted from the movie at a 12-hour interval. The corresponding recovered aberrations vary significantly across the whole imaging process and the main reason can be attributed to focus drift (see the line chart beside). Figures 5(b1)-5(b4) demonstrate the performance of AO-QPI to trace the organelle motions occurring on a short time scale and at a small length scale. In Fig. 5(c1)-5(c3), to further verify the necessity of the dynamic correction, we compare the cell phase maps reconstructed with the same six raw images but corrected by the aberrations recovered at different time points. It can be seen from the obtained phase maps that a fixed correction could not sustain the imaging quality for even a few hours, and the adaptive correction scheme is hence indispensable for long-term imaging. Benefiting from the long-term stable imaging capacity of AO-QPI, we further study the cell morphology during cell divisions covering three generations. The cell images at the key timings are given in Fig. 5(d1)-5(d6). The mother cell in Fig. 5(d1) underwent three mitoses across \(\sim\)27 hours and was eventually divided into four individual daughter cells, as shown in Fig. 5(d6). The provided experiment results reveal the capacity of the AO-QPI approach for correcting temporally varied aberrations and securing the imaging performance for a long-term longitudinal study.
Long-term live-cell phase imaging across 51 hours [see Visualization 2]. (a) The Full-FOV phase maps of the first and last frame in the movie. (a1)-(a5) The zoomed-in phase maps in the central region of (a) as well as the line chart indicating the out-of-focus distance variation over the 51 hours. (b1)-(b4) Dynamic high-resolution demonstration of cellular architecture across 4 hours. (c1)-(c3) Comparison of results with and without adaptive aberration correction as well as the profiles of the cell phase. (d1)-(d6) The 3-generation cell division process from 8h 6min to 35h 6min
Through the experimental results demonstrated in Fig. 5, we can see that the proposed AO-QPI could stably provide continuous analysis of single cells over several cell divisions, even though the microscopy hardware might have not been robust enough for long-term imaging. The long-term continuous imaging is indispensable to detect dynamic, rare, and heterogeneous cell responses, which could otherwise be missed by the single-shot imaging. Compared with the existing methods [60,61,62,63] for tracking focus drift in real time, our approach does not require any additional hardware and allows digital refocusing after the data has been acquired. In addition, the commonly used algorithm of gradient detection for autofocusing is problematic for cell culture dishes as the intensity image has the lowest contrast when the sample is placed at the in-focus position. The reported AO-QPI scheme can address this challenge and offer a turnkey solution for long-term in vitro imaging in a robust and stable manner.
Conclusions
In this article, we first apply an FPM-based computational AO method in QPI to secure the high-quality performance for long-term imaging across a wide field. The proposed AO-QPI approach is based on the annular matched illumination configuration, which has been verified to be essential for the recovery of low-frequency phase information [23, 49]. Here, through the simulations and experiments, we further reveal that such an illumination configuration is equally indispensable for the accurate characterization of the imaging aberrations. This can be attributed to the characteristic of the aberration which mainly consists of low-frequency components. To improve imaging efficiency, we also analyze the minimum data redundancy required for a successful recovery of both phase and aberration. We find that only six raw images could provide sufficient raw datasets for the ptychographic reconstruction, performing the real-time imaging of live cells. For a further demonstration of AO-QPI’s performance on biomedical samples, we apply the method to HeLa cell cultures for long-term and wide-field imaging. The diffraction-limited imaging performance is secured across a large FOV of 1.77 \(mm^2\), over a long period of 51 hours at a 16.66 Hz frame rate. Our demonstration indicates that the AO-QPI approach offers a powerful tool for joint spatiotemporal adaptive aberration correction.
The FPM-based aberration correction approach has been widely applied in many challenging fields to improve imaging performance [43, 64,65,66,67]. However, a limitation is that these works are mainly aimed at stained samples. In this article, we first apply a similar framework for quantitative phase imaging. It should be noted that due to the transfer response difference of the sample’s phase and absorption components in the FPM imaging process [23, 49], the aberration recovery for pure phase objects is more difficult to be achieved based on a normal FPM illumination scheme. The adopted annular matched illumination configuration could provide a reliable and efficient approach for aberration correction on the transparent biospecimen. Moreover, as an extension to FPM, the technology of 3D FPM [68,69,70,71] is developed to further reveal the 3D phase information for thick samples. The existence of aberrations could also affect the imaging performance of the 3D phase imaging. Therefore, applying AO-QPI to achieve high-quality depth-resolved imaging for thick phase objects is another important direction that requires further investigation.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Change history
14 November 2022
A Correction to this paper has been published: https://doi.org/10.1186/s43074-022-00073-1
Abbreviations
- QPI:
-
Quantitative phase imaging
- FPM:
-
Fourier ptychographic microscopy
- NA:
-
Numerical aperture
- DH:
-
Digital holography
- TIE:
-
Transport-of-intensity equation
- DPC:
-
Differential phase contrast
- AO:
-
Adaptive optics
- AI:
-
Annular illumination
- FOV:
-
Field of view
- LED:
-
Light-emitting diode
- PIE:
-
Ptychographic iterative engine
- ePIE:
-
Extended ptychographic iterative engine
- EPRY:
-
Embedded pupil function recovery
- QPT:
-
Quantitative phase microscopy target
- 3D:
-
Three-Dimensional
References
Mir M, Bhaduri B, Wang R, Zhu R, Popescu G. Quantitative phase imaging Prog Opt. 2012;57(133–37):217.
Park Y, Depeursinge C, Popescu G. Quantitative phase imaging in biomedicine. Nat Photonics. 2018;12(10):578–89.
Fan Y, Li J, Lu L, Sun J, Hu Y, Zhang J, Li Z, Shen Q, Wang B, Zhang R, Chen Q, Zuo C. Smart computational light microscopes (SCLMs of smart computational imaging laboratory (SCILab. PhotoniX. 2021;2(1):19.
Kim MK. Principles and techniques of digital holographic microscopy. SPIE Rev. 2010;1(1): 018005.
Zhao R, Huang L, Wang Y. Recent advances in multi-dimensional metasurfaces holographic technologies. PhotoniX. 2020;1(1):20.
Huang Z, Memmolo P, Ferraro P, Cao L. Dual-plane coupled phase retrieval for non-prior holographic imaging. PhotoniX. 2022;3(1):3.
Popescu G, Deflores LP, Vaughan JC, Badizadegan K, Iwai H, Dasari RR, Feld MS. Fourier phase microscopy for investigation of biological structures and dynamics. Opt Lett. 2004;29(21):2503.
Wang Z, Millet L, Mir M, Ding H, Unarunotai S, Rogers J, Gillette MU, Popescu G. Spatial light interference microscopy (slim). Opt Express. 2011;19(2):1016–26.
Chanteloup J-C. Multiple-wave lateral shearing interferometry for wave-front sensing. Appl Opt. 2005;44(9):1559.
Bon P, Maucort G, Wattellier B, Monneret S. Quadriwave lateral shearing interferometry for quantitative phase microscopy of living cells. Opt Express. 2009;17(15):13080.
Barty A, Nugent K, Paganin D, Roberts A. Quantitative optical phase microscopy. Opt Lett. 1998;23(11):817–9.
Gureyev T, Roberts A, Nugent K. Partially coherent fields, the transport-of-intensity equation, and phase uniqueness. JOSA A. 1995;12(9):1942–6.
Zuo C, Sun J, Li J, Zhang J, Asundi A, Chen Q. High-resolution transport-of-intensity quantitative phase microscopy with annular illumination. Sci Rep. 2017;7(1):7654.
Hamilton D, Sheppard C. Differential phase contrast in scanning optical microscopy. J Microsc. 1984;133(1):27–39.
Tian L, Waller L. Quantitative differential phase contrast imaging in an LED array microscope. Opt Express. 2015;23(9):11394.
Fan Y, Sun J, Chen Q, Pan X, Tian L, Zuo C. Optimal illumination scheme for isotropic quantitative differential phase contrast microscopy. Photon Res. 2019;7(8):890.
Zheng G, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat Photonics. 2013;7(9):739–45.
Ou X, Horstmeyer R, Yang C, Zheng G. Quantitative phase imaging via Fourier ptychographic microscopy. Opt Lett. 2013;38(22):4845.
Tian L, Liu Z, Yeh L-H, Chen M, Zhong J, Waller L. Computational illumination for high-speed in vitro Fourier ptychographic microscopy. Optica. 2015;2(10):904.
Zheng G, Shen C, Jiang S, Song P, Yang C. Concept, implementations and applications of fourier ptychography. Nat Rev Phys. 2021;3(3):207–23.
Greivenkamp JE. Field Guide to Geometrical Optics, vol. 1. Washington: SPIE Press Bellingham; 2004.
Park J, Brady DJ, Zheng G, Tian L, Gao L. Review of bio-optical imaging systems with a high space-bandwidth product. Adv Photonics. 2021;3(04). https://doi.org/10.1117/1.AP.3.4.044001.
Sun J, Zuo C, Zhang J, Fan Y, Chen Q. High-speed Fourier ptychographic microscopy based on programmable annular illuminations. Sci Rep. 2018;8(1):7669.
Fan Y, Sun J, Chen Q, Pan X, Trusiak M, Zuo C. Single-shot isotropic quantitative phase microscopy based on color-multiplexed differential phase contrast. APL Photon. 2019;4(12): 121301.
Li J, Matlock A, Li Y, Chen Q, Zuo C, Tian L. High-speed in vitro intensity diffraction tomography. Adv Photon. 2019;1(06):1.
Chang X, Bian L, Zhang J. Large-scale phase retrieval eLight. 2021;1(1):4.
Starkuviene V, Pepperkok R. The potential of high-content high-throughput microscopy in drug discovery. Br J Pharmacol. 2007;152(1):62–71.
Glory E, Murphy RF. Automated subcellular location determination and high-throughput microscopy. Dev Cell. 2007;12(1):7–16.
Park Y, Best CA, Badizadegan K, Dasari RR, Feld MS, Kuriabova T, Henle ML, Levine AJ, Popescu G. Measurement of red blood cell mechanics during morphological changes. Proc Natl Acad Sci. 2010;107(15):6731–6.
Li Y, Di J, Wang K, Wang S, Zhao J. Classification of cell morphology with quantitative phase microscopy and machine learning. Opt Express. 2020;28(16):23916–27.
Lukosz W. Optical Systems with Resolving Powers Exceeding the Classical Limit*. J Opt Soc Am. 1966;56(11):1463.
Schroeder T. Long-term single-cell imaging of mammalian stem cells. Nat Methods. 2011;8(S4):30–5.
Shirasaki Y, Yamagishi M, Suzuki N, Izawa K, Nakahara A, Mizuno J, Shoji S, Heike T, Harada Y, Nishikomori R, Ohara O. Real-time single-cell imaging of protein secretion. Sci Rep. 2015;4(1):4736.
Skylaki S, Hilsenbeck O, Schroeder T. Challenges in long-term imaging and quantification of single-cell dynamics. Nat Biotechnol. 2016;34(11):1137–44.
Adie SG, Graf BW, Ahmad A, Carney PS, Boppart SA. Computational adaptive optics for broadband optical interferometric tomography of biological tissue. Proc Natl Acad Sci. 2012;109(19):7175–80.
Davies R, Kasper M. Adaptive optics for astronomy. arXiv preprint arXiv:1201.5741. 2012.
Tyson RK, Frazier BW. Principles of Adaptive Optics, CRC Press, 2022.
Booth MJ. Adaptive optical microscopy: The ongoing quest for a perfect image. Light Sci Appl. 2014;3(4):165–165.
Guo Y, Zhong L, Min L, Wang J, Wu Y, Chen K, et al. Adaptive optics based on machine learning: a review. Opto Electron Adv. 2022;5(7):200082.
Tao X, Fernandez B, Azucena O, Fu M, Garcia D, Zuo Y, Chen DC, Kubby J. Adaptive optics confocal microscopy using direct wavefront sensing. Opt Lett. 2011;36(7):1062.
Gould TJ, Burke D, Bewersdorf J, Booth MJ. Adaptive optics enables 3D STED microscopy in aberrating specimens. Opt Express. 2012;20(19):20998.
Ou X, Zheng G, Yang C. Embedded pupil function recovery for Fourier ptychographic microscopy. Opt Express. 2014;22(5):4960.
Chung J, Kim J, Ou X, Horstmeyer R, Yang C. Wide field-of-view fluorescence image deconvolution with aberration-estimation from Fourier ptychography. Biomed Opt Express. 2016;7(2):352.
Song P, Jiang S, Zhang H, Huang X, Zhang Y, Zheng G. Full-field Fourier ptychography (FFP: Spatially varying pupil modeling and its application for rapid field-dependent aberration metrology. APL Photon. 2019;4(5): 050802.
Kam Z, Hanser B, Gustafsson MGL, Agard DA, Sedat JW. Computational adaptive optics for live three-dimensional biological imaging. Proceedings of the National Academy of Sciences. 2001;98(7):3790–5.
South FA, Liu Y-Z, Bower AJ, Xu Y, Carney PS, Boppart SA. Wavefront measurement using computational adaptive optics. J Opt Soc Am A. 2018;35(3):466.
Sun J, Chen Q, Zhang Y, Zuo C. Sampling criteria for Fourier ptychographic microscopy in object space and frequency space. Opt Express. 2016;24(14):15765.
Baek Y, Park Y. Intensity-based holographic imaging via space-domain Kramers-Kronig relations. Nat Photonics. 2021;15(5):354–60.
Li J, Zhou N, Sun J, Zhou S, Bai Z, Lu L, Chen Q, Zuo C. Transport of intensity diffraction tomography with non-interferometric synthetic aperture for three-dimensional label-free microscopy. Light Sci Appl. 2022;11(1):154.
Yeh L-H, Dong J, Zhong J, Tian L, Chen M, Tang G, Soltanolkotabi M, Waller L. Experimental robustness of Fourier ptychography phase retrieval algorithms. Opt Express. 2015;23(26):33214.
Bian L, Suo J, Zheng G, Guo K, Chen F, Dai Q. Fourier ptychographic reconstruction using wirtinger flow optimization. Opt Express. 2015;23(4):4856–66.
Rodenburg JM, Faulkner HML. A phase retrieval algorithm for shifting illumination. Appl Phys Lett. 2004;85(20):4795–7.
Guizar-Sicairos M, Fienup JR. Phase retrieval with transverse translation diversity: A nonlinear optimization approach. Opt Express. 2008;16(10):7264.
Maiden A, Johnson D, Li P. Further improvements to the ptychographical iterative engine. Optica. 2017;4(7):736.
Maiden AM, Rodenburg JM. An improved ptychographical phase retrieval algorithm for diffractive imaging. Ultramicroscopy. 2009;109(10):1256–62.
Wang JY, Silva DE. Wave-front interpretation with Zernike polynomials. Appl Opt. 1980;19(9):1510.
Lakshminarayanan V, Fleck A. Zernike polynomials: A guide. J Mod Opt. 2011;58(7):545–61.
Zhang S, Zhou G, Zheng C, Li T, Hu Y, Hao Q. Fast digital refocusing and depth of field extended fourier ptychography microscopy. Biomed Opt Express. 2021;12(9):5544–58.
Ho GH, Cheng A, Chen C-J, Fang C-K, Li MC, Chang I-C, Chu P, Chu Y, Shu K, Huang C, et al. Lens heating-induced focus drift of i-line step and scan: correction and control in a manufacturing environment. In: Metrology, Inspection, and Process Control for Microlithography XV, vol. 4344. SPIE; 2001. pp. 289–296.
Lee SH, Baday M, Tjioe M, Simonson PD, Zhang R, Cai E, Selvin PR. Using fixed fiduciary markers for stage drift correction. Opt Express. 2012;20(11):12177.
Cheng B-J, Liu H-C, Cui Y, Guo J. Improving image control by correcting the lens-heating focus drift. In: Optical Microlithography XIII, vol. 4000. SPIE; 2000. pp. 818–826.
Kreft M, Stenovec M, Zorec R. Focus-Drift Correction in Time-Lapse Confocal Imaging. Ann N Y Acad Sci. 2005;1048(1):321–30.
Grover G, Mohrman W, Piestun R. Real-time adaptive drift correction for super-resolution localization microscopy. Opt Express. 2015;23(18):23887.
Kamal T, Yang L, Lee WM. In situ retrieval and correction of aberrations in moldless lenses using Fourier ptychography. Opt Express. 2018;26(3):2708.
Konda PC, Taylor JM, Harvey AR. Multi-aperture fourier ptychographic microscopy, theory and validation. Opt Lasers Eng. 2021;138: 106410.
Aidukas T, Eckert R, Harvey AR, Waller L, Konda PC. Low-cost, sub-micron resolution, wide-field computational microscopy using opensource hardware. Sci Rep. 2019;9(1):7457.
Shen C, Chan ACS, Chung J, Williams DE, Hajimiri A, Yang C. Computational aberration correction of VIS-NIR multispectral imaging microscopy based on Fourier ptychography. Opt Express. 2019;27(18):24923.
Tian L, Waller L. 3d intensity and phase imaging from light field measurements in an led array microscope. Optica. 2015;2(2):104–11.
Horstmeyer R, Chung J, Ou X, Zheng G, Yang C. Diffraction tomography with fourier ptychography. Optica. 2016;3(8):827–35.
Zuo C, Sun J, Li J, Asundi A, Chen Q. Wide-field high-resolution 3D microscopy with Fourier ptychographic diffraction tomography. Opt Lasers Eng. 2020;128: 106003.
Zhou S, Li J, Sun J, Zhou N, Chen Q, Zuo C. Accelerated fourier ptychographic diffraction tomography with sparse annular led illuminations. J Biophotonics. 2022;15(3):202100272.
Acknowledgements
Not applicable.
Funding
This work was supported by the National Natural Science Foundation of China (61905115, 62105151, 62175109, U21B2033, 62105156), Leading Technology of Jiangsu Basic Research Plan (BK20192003), Youth Foundation of Jiangsu Province (BK20190445, BK20210338), Fundamental Research Funds for the Central Universities (30920032101), and Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense (JSGP202105, JSGP202201).
Author information
Authors and Affiliations
Contributions
C.Z. and J.S. provided the original idea. Y.S. and J.S. designed and performed the experiments. Y.S. and Y.F. analyzed the data. Y.S., N.Z., J.L. and R.Y. prepared the figures and supplementary movies. Y.S. and J.S. wrote the manuscript. C.Z., Q.C. and G.Z. supervised the overall projects. All the authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Ethics approval and consent to participate
There is no ethics issue for this paper.
Consent for publication
All authors agreed to publish this paper.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original version of this article was revised: the Additional file 2 was updated.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Shu, Y., Sun, J., Lyu, J. et al. Adaptive optical quantitative phase imaging based on annular illumination Fourier ptychographic microscopy. PhotoniX 3, 24 (2022). https://doi.org/10.1186/s43074-022-00071-3
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1186/s43074-022-00071-3