1 Introduction

In the last decade, Digital Holography (DH) has become one of the most important tools for the inspection and characterization of microsystems [1]. In particular, DH has been a fundamental optical metrology technique for the inspection and characterization of micro-electromechanical systems (MEMS) and microoptoelectromechanical systems (MOEMS) [2]. Meanwhile, impressive developments have been also made by DH for applications in biomedical science by lab on chip devices [3,4,5].

With the development of science and technology, the curiosity for the micro-world has been stimulating us to advance the microscopic imaging technologies. From the birth of the first microscope to super-resolution imaging, 400 years of technology accumulation have been stepped underfoot. Nowadays, we stand on the shoulders of giants and look at the future a variety of microscopic imaging technologies exert their abilities in different measurement scales. DH has been considered as a powerful candidate for the best imaging tool. Conventional holography was proposed by Dennis Gabor in 1964 [5]; then, in 1967, Goodman et al. introduced the idea of using a digital detector for recording the holograms, which built the base of modern DH. Owing to the development of laser technology, DH has become one of the fundamental optical metrology techniques for inspection and characterization of microsystem [1, 2] bio-samples [6, 8], polymer [7, 14], and so on. Recently, impressive developments have been also made by DH for applications in biomedical science by lab on a chip device [3]. Although this type of equipment is far away from the usual impression of microscopes, the flexibility of DH allows it to achieve longitudinal high-resolution imaging, not only in axial but also in lateral [9, 10].

Essentially, the digital hologram is an interference pattern which is created by interference between a reference beam and an object beam. The digital recording of the holograms permits to calculation of the complex wavefront, and then, the amplitude and phase of specimen can be extracted separately [11]. Owing to the use of microscope objective (MO), quantitative microscopic phase-contrast image can be obtained by DH; the lateral resolution can reach at submicron scale. Numerous applications have been exploited for DH in various contexts, such as particles and flow tracking [12], imaging of biological samples, and object recognition [13,14,15,16,17,18]. DH technique has the possibility to provide a full-field, non-invasive, dynamic, quantitative analysis of the sample. Once the hologram has been recorded, the phase map of the object can be reconstructed by means of numerical procedures; in particular, it is possible to operate a re-focusing procedure by means of the diffraction integral.

Unfortunately, DH setups are affected by phase aberrations mainly due to the off-axis recording, imaging lens, MOs, etc.

Over the years, many scholars have developed different strategies for obtaining high-quality holographic imaging system without aberrations. Optical setup optimization [19,20,21,22] and numerical algorithm [23,24,25] are considered to be effective means to improve the imaging quality of the system, especially for the aberrations that affect the overall imaging process. Recently, methods based on Artificial Intelligence (AI) have been demonstrated very effective for compensating for all types of aberration in DH.

In this paper, the strategies to compensate for the aberration in digital holographic microscopy have been reviewed and discussed. These methods can effectively improve the accuracy of holographic imaging and allow quantitative analysis of samples. In general, phase aberrations in DH imaging can be divided into lower order aberration and higher order aberration. The former is introduced by the off-axis angle and the lens [26], the latter, instead, is often introduced by the optical properties of the sample itself, e.g., lens effect of cell [27] or membrane [8]. It can also be distinguished by the order of the phase distortion factors during reconstruction. The factor less than the third order is a lower order aberration, and the factor higher than the third order is a higher order aberration [28]. Among the phase aberrations, the toughest ones are related to the higher order aberrations. Current methods mostly can compensate the lower order aberrations accurately, but have limitations in quantifying higher order aberrations [29]. Automatic aberration compensation based on a single hologram is necessary for preserving the temporal resolution of holographic microscopy. Furthermore, the ability of aberration compensation also allows us to improve particle characterization in DH [30].

In Sect. 2, we introduce the basic principle and conventional setups of DH. The influence of aberrations on DH imaging process is analyzed; the causes of different aberrations are carried out. In the experiments with high temporal resolution requirements, off-axis holography has been widely used. However, the off-axis geometry induces a wavefront tilt in the retrieved phase, so-called off-axis aberration. This is the most common aberration in conventional off-axis DH. Moreover, the aberration induced by MOs or lens also can affect the quality of DH imaging. More in general, all types and different polynomial order of aberrations need to be considered during holographic numerical reconstruction. Each type of aberration provokes a certain phase contribution that does not allow a correct reconstruction of phase. In addition, to extend the range of phase measurement in DH, a set of interferograms with different wavelengths is recorded based on a multiple-wavelength configuration. The synthetic phase map without \(2\pi\) ambiguity is obtained with a beat wavelength. Owing to the chromatic aberration of optical components, the in-focus image for each wavelength will be at different planes, which introduced multi-order aberrations. Consequently, the chromatic aberrations can strongly influence the retrieved phase map in synthetic wavelength.

In Sects. 3, the optical methods to compensate for aberrations in holographic microscopy are discussed, e.g., a background hologram, additional optical components, or special setup to compensate for phase aberrations. Numerical methods are discussed in Sect. 4; generally, this strategy is based on digital phase mask (DPM) approaches. A DPM could be modeled with parabolic function, standard polynomials, or Zernike polynomials. Hybrid method based on a fusion between optical and computational approaches is described in the Sect. 4; the low-pass filtering based on common-path DH microscopy, double fitting and background detecting, twice recording, and polynomials calculation have been discussed. In the last section, artificial intelligence (AI) procedures are discussed. In particular, CNN, U-net model, multivariate regression network, and deep convolutional neural network can be used to compensate for most types of aberration in an automatic way.

The optical, numerical, and AI-based methods to compensate for aberration, that we discussed in this review, allow us to ensure the axial and lateral resolution of DH. In general, a fast and automatic method for compensating all types of phase aberration is important for the future applications. We present a review of research and development in phase aberration compensation, where the optical, numerical, and AI methods are summarized. The advantages and limitations of different methods are analyzed. As a conclusion, we prospect the future needs and trends on aberration compensation.

Digital Holography in microscope configuration and inherent wavefront aberrations.

1.1 Digital holographic microscopy

DH is an emergent imaging technology. Digital holographic microscopy (DHM) is one of the most important research and application areas of DH; it could obtain quantitative phase-contrast images with lateral resolution at the submicron scale [31,32,33,34] and longitudinal resolution in sub-nanometer range [35]. DH is a non-invasive and quantitative approach to investigate sample. A single hologram acquisition includes the whole complex field of sample scattered light, and with respect to other microscopic technique, it allows subsequent focus propagation of recorded images. Hence, in-focus imaging and tracking of multiple objects in the same field of view (FoV) at different depths are granted and smart-tracking of fast particles is achievable [36, 37].

A basic DHM system includes a light source, an interferometer with MO or MOs, a digitizing camera, and a computer with the necessary software. Laser is the conventional light source for DHM, and its coherence properties allow to create interferogram. For multi-wavelength techniques, two or more of different lasers can be coupled into the single interferometric geometry; the tunable laser also can be employed. [7, 19]

In holography, the laser beam coming from the source is split into two beams, so-called object beam and reference beam. The object beam is reflected or transmitted through a sample; it will meet the reference beam at hologram plane. The interference of these two beams’ results in fringe patterns, that are recorded by a charge-coupled device (CCD) or a complementary metal–oxide–semiconductor (CMOS) camera. The hologram will be transferred to the computer in digital format. Its intensity represents the interference between the object wave O and the reference wave R

$$\begin{array}{c}{I}_{\mathrm{H}}\left(x,y\right)=\left({\varvec{R}}+{\varvec{O}}\right){\left({\varvec{R}}+{\varvec{O}}\right)}^{*}={\left|{\varvec{R}}\right|}^{2}+{\left|{\varvec{O}}\right|}^{2}+{{\varvec{R}}}^{*}O+R{{\varvec{O}}}^{*}.\end{array}.$$
(1)

The different terms of this relation represent the zeroth order of diffraction \(\left({\left|{\varvec{R}}\right|}^{2}+{\left|{\varvec{O}}\right|}^{2}\right)\), the virtual image \(({\varvec{R}}{{\varvec{O}}}^{\boldsymbol{*}})\), and the real image \(({{\varvec{R}}}^{\boldsymbol{*}}{\varvec{O}})\). Generally, these terms are numerically filtered producing a hologram which contains only the virtual or the real-image term.

Taking into account the pixelization of the camera, the snapshot is more rigorously described through the discrete intensity distribution

$$\begin{array}{c}{I}_{H}\left(r,l\right)=\sum_{r=1}^{{N}_{x}}\sum_{l=1}^{{N}_{y}}I\left(r\Delta {x}_{0}, l\Delta {y}_{0}\right) ,\end{array}$$
(2)

in which \(\left({N}_{x},{N}_{y}\right)\) are the number of pixels of the camera and \(\left(\Delta {x}_{0},\Delta {y}_{0}\right)\) is the pitch.

Although DH demands post-acquisition processing, to date, for digital holograms collected by off-axis arrangements, the reconstruction pipeline which includes demodulating [35], re-focusing [38, 39], denoising [40], and phase unwrapping [41] are very accessible and rapid thanks to AI and numerical methods [42, 43] advanced in the last years. While, iterative phase retrieval algorithms are required for in-line recording arrangements [44].

In a DH setup, the MO is inserted in the object and/or in the reference arm; in this way, it is possible to improve the lateral resolution and the magnification of the image in the hologram plane.

Basically, digital holographic setups are categorizable in two main classes. In-line arrangements where the light beam scattered by the object produces both the object beam and the reference beam. This configuration is much more simple and offers high stability and resolution, even if phase retrieval is quite cumbersome [45]. In off-axis arrangements, the reference beam is separated from object one; thus, it provides lower resolution, but simpler numerical reconstruction procedures. In addition, off-axis geometry induces an angle between object and reference beams which provides intrinsically a tilt phase aberration to the phase of hologram, i.e., an off-axis aberration. [35].

Off-axis DH has two conventional geometries, i.e., Michelson interferometer and the Mach–Zehnder interferometer, as shown in Fig. 1. There are many other configurations based on different arrangements. One special configuration is based on wavefront division. In this case, the object beam and the reference beams are obtained by dividing the wavefront. Different optical elements can be used to achieve this scheme, such as diffraction grating, pinhole, polarizing element, specular reflection, or refraction in an optical prism [35].

Fig. 1
figure 1

Interferometer configurations for digital holographic microscopy. a Michelson; b Mach–Zehnder. BS beam splitter, M Mirror, L lens, MO microscope objective

Full size image

Usually, a Michelson interferometer is used for reflective objects’ investigation. The radiation coming from the laser source is split into two beams by a beam splitter, and then, the reference beam hits a mirror and is reflected back, while the object beam hits the sample plane and is reflected back. Both light beams are recombined by the beam splitter and illuminate the camera screen.

The Mach–Zehnder interferometer can be used for both transmissive and reflective objects; it is the most common off-axis geometry. Mach–Zehnder interferometer is based on the two-beam interference by splitting of the incoming radiation laser. The two waves travel along different paths; sample is placed in object arm. In DHM, the sample is illuminated by the object beam and then the diffracted light of it will be collected by MO. The reference beam and the object beam are recombined by a beam splitter and the hologram is recorded by a camera. In general, the sample is illuminated by a plane wave and the magnification is ensured by the MO.

All the above-mentioned configurations can be modified, either it is possible omitting lenses in the object and reference arms obtaining Fresnel holography configuration. In alternate setup, it is possible to use a lensless Fourier configuration where a lens in the reference arm has the focal plane in a plane conjugate to the object plane.

Compared to conventional microscopy imaging methods, DH microscopy has a significant key feature, i.e., it is a label-free imaging. The interferometric imaging process makes DH very sensitive to the optical path difference (OPD) between object beam and reference beam [46, 47], and thus, wavefront aberrations can be strongly affected by phase aberrations and the final imaging. To enhance the real-time imaging capability of the holographic recording process, off-axis geometry is widely used in the imaging process of biological samples [47] where the most common aberration, so-called off-axis angle aberration is present. Moreover, the introduction of microscope objectives (MOs) for acquiring the higher resolution also increases the risk of further aberrations. For holographic phase imaging, phase aberration suppresses the phase contrast of the recorded object and further affects the final imaging accuracy. This is the reason why compensating or eliminating the phase aberrations is of fundamental importance in reconstructing digital holograms.

1.2 Wavefront aberrations in DH

Over the years, numerous approaches have been developed to compensate for the presence of aberrations and imperfections of the optical systems in DH both from a physical and a numerical point of view. In fact, the wavefront curvature arose from the presence of MOs and lenses can be easily removed and/or compensated [48, 49], as well as astigmatism [20, 50], anamorphism [50], spherical aberrations [21, 22], and chromatic aberrations [51]. Aberration is a typical issue in optical studies. For example, the effect of wave front aberration in a simple glass lens causes light to be spread out over some region of space rather than perfectly focused to a single point [52]. Monochromatic aberrations affect the light that is reflected or refracted through a mirror or a lens and also occur in the case of monochromatic light. Chromatic aberrations are due to dispersion, i.e., the variation of a lens’s refractive index with wavelength, different wavelengths of a light beam come to focus on different points. If we consider a point image generated by a perfect optical system, its wavefront is hemispherical. The aberration of an optical system is described in terms of the departure of the geometric wavefront from a spherical shape centered on the paraxial image, as shown in Fig. 2.

Fig. 2
figure 2

a Reference sphere and actual wavefront are shown in the exit pupil of a lens. Also, the optical path difference and the ray intercept in the image plane are shown. b Reference beam and object beam in case of off-axis interference; θ is the off-axis tilt

Full size image

Zernike circular polynomials are used to fit the surface produced by the ray trace, allowing the characterization of the departure of the geometrical wavefront from the reference sphere. They have been widely used owing to the mathematical properties [37, 38]. The wave aberration function \(\Delta\) is the distance between the reference sphere and the actual wavefront, in the exit pupil. This function defines the OPD between two surfaces; it is measured along the radius of the reference sphere. In an optical system, we can describe the functional dependence of \(\Delta\) from scalar quantities, such as \(\rho\), \(\rho h\mathrm{cos}\theta\), and \({h}^{2}\). Herrin, \(\Delta\) can be expand as a power series in these scalar variables, to obtain a description of the aberrations in the optical system [55]. \(\rho\) is the pupil radius; \(\theta\) is the azimuthal angle around the pupil with\(0\le \theta \le 2\pi\). \(h\) is the distance of the object from the optical axis. The Zernike coefficients are linearly independent; thus, individual aberrations’ contributions to an overall wavefront may be isolated and quantified separately. Generally, for sake of simplicity, aberrations are considered until to third order; however, wavefront aberrations produced by propagation through atmospheric turbulence or aerodynamic flow fields are not well modeled by Zernike polynomials; in this case, one may employ other fitting methods [56].

1.2.1 Spherical aberration (lens aberration)

In the optical imaging process, one of the main aberrations caused by lenses is called spherical aberration, which is a monochromatic aberration. Owing to the use of spherical lens, spherical aberration usually appears as third-order aberration. Ideally, the intersection of single-wavelength light beams passing through a lens and the optical axis should be a single point, as shown in Fig. 3a. However, spherical aberration is difficult to avoid in practical situations as often the lenses are used also for non-paraxial rays, and thus, the intersection points mainly depend on the distance between the point, where rays strike the spherical surface of the lens, and the center of it [57]. If the rays are further away from the optical axis, (i.e., far from the paraxial approximation), they are focused on points closer to the lens, as shown in Fig. 3b [29].

Fig. 3
figure 3

a A depiction of a perfect lens without spherical aberration: all incoming rays are focused in the focal point. b A real lens with spherical surfaces, which produces spherical aberration: The different rays do not meet after the lens in one focal point. The further the rays are from the optical axis, the closer to the lens they intersect the optical axis (positive spherical aberration) (Drawing is exaggerated.)

Full size image

Whan a quantitative phase map is retrieved in DH, the aberration introduced by the lens produces a perturbing phase factor during the numerical reconstruction, that is superimposed to the correct phase map of the object under investigation. In fact, the interference between the object wavefront and the reference wavefront gives rise to a digital hologram in which appears a fringe pattern similar to the well-known Newton ring. This pattern is owing mainly to the interference between the defocus (or parabolic) phase factor, contained in the object wave, and the reference beam. This superimposed ring pattern increases the complexity of the numerical reconstruction process and can produce important distortion in the reconstructed phase maps [21, 22]. Of course, if there are also other aberrations such a spherical aberration, the object phase map will be affected both by defocus and also spherical wavefront aberration.

1.2.2 Astigmatism

A lens that shows a different focal length for the tangential and sagittal rays is affected by astigmatism. In particular, a spherical lens with astigmatism behaves, for off-axis rays, as a cylindrical lens is in contact with the spherical lens. As shown in Fig. 4a–d, the digital reconstruction of a hologram affected by astigmatism shows a rectangular shape. A point light source is reconstructed in a different plane; the astigmatic wavefield focuses on a line image corresponding to the tangential focal line [20]. In Fig. 4a, the two distributions are superposed owing to the spherical symmetry of the wavefront, whereas in Fig. 4b, they are different due to the astigmatism. To compare the numerically reconstructed phase at different planes, Grilli et al. plotted in Fig. 4e, f the unwrapped phase distributions for different reconstruction distances, under two considered cases.

Fig. 4
figure 4

Reprinted and adapted with permission from ref [20] © The Optical Society

a, c Reconstruction of a hologram of a wavefront without astigmatism in a plane at distance \({\varvec{d}}\) from the hologram plane (a) and at the focal plane (c). b, d Reconstruction of a hologram of a wavefront affected by astigmatism at distance \({\varvec{d}}\) from the hologram plane (b) and at the focal plane (d). e, f One-dimensional representation of the unwrapped phase values along the x-horizontal (straight line) and y-vertical (dashed line) directions for reconstruction distances \({\varvec{d}}{^{\prime}}\) ranging from 160 to 220 mm, step size of 10 mm: e phase reconstructions of the simulated spherical wave front; f phase reconstructions of the astigmatic wave front. The scale of the horizontal axis is determined by the pixel size of the reconstructed image which does not change in the convolution-based reconstruction method. The vertical axis is the z propagation axis along which the various phase distributions are evaluated.

Full size image

1.2.3 Chromatic aberration

Commonly, the refractive indexes of an optical lens depend on wavelength [44]. For this reason, an optical system, which uses different light sources or a tunable laser, can be affected by chromatic aberrations that result in different focal length. In the case of multi-wavelength hologram recording, the same optical path will be used to create holograms of different wavelengths. The chromatic aberration will occur once multi-wavelength shares the same imaging plane. The correctness of the phase map, obtained by subtracting two phases corresponding to two different wavelengths, will be incorrect, since one of them will result out-of-focus. The final quantitative phase map (QPM) calculated with the synthetic wavelength will result in erroneous. In DH, the object wave front, scattered by the sample, if it is recorded out-of-focus, the in-focus image can be obtained by a numerical reconstruction of the digital interferogram at the right focus image plane. The flexibility, intrinsically embedded in the DH that consists in the numerical re-focusing process offers a very important and useful opportunity to compensate aberrations and remove the errors in the QPM. In this case, the mechanical adjustment or wavefront correction by means of active devices can be abandoned.[59]. It is important to note that, owing to the chromatic aberrations, the non-negligible longitudinal image shift will occur in the numerical reconstruction process. The well-reconstructed images can be obtained after properly resized; this process is related to the different reconstruction distances and the recording wavelength.

2 Optimizing the optical setup to compensate aberrations in digital holographic microscopy

Over the years, numerous crafts have been developed to compensate for phase aberrations in DH. In this section, the optimization of optical setups and related numerical processes to compensate for the various types of aberrations in the context of DH will be discussed. The use of an MO in DH enables the object wave to have a spherical wavefront, which could further introduce hemispherical wavefront phase distortion. In fact, the MO induces intrinsically a parabolic phase factor in the recorded hologram. Thus, to recover the phase information of the field scattered by the object, it is necessary to remove this phase contribution. A lot of techniques in past decades have been employed to compensate for this curvature of the wavefront, both numerically and physically.

One of the most conventional and classic strategies proposed by Sànchez-Ortiga et al. [19] is to compensate for the residual parabolic phase distortion by means of a telecentric arrangement in a Mach–Zehnder interferometer. The telecentric arrangement is used to replace the MO for achieving the magnification imaging, as shown in Fig. 5a. A second lens [the tube lens (TL)] of higher focal length and smaller NA is used to form an intermediate image near the back focal plane. In this way, a plane object wave could be obtained and the interference with the reference wave provides a hologram without quadratic phase distortion, so the phase factor introduced by the objective lens is avoided.

Fig. 5
figure 5

a Schematic of DHM telecentric arrangement. b Schematic of the DHM where two identical MOs are inserted in the two arms

Full size image

Another widely used strategy that could solve this problem was presented by Mann et al. [60] and depicted in Fig. 5b. The microscope objective \({\mathrm{MO}}_{2}\) is inserted in the reference arm to compensate for the phase curvature caused by the \({\mathrm{MO}}_{1}\) in the object arm. In general, these two MOs have the same focal length and are placed at the same distance from the digital recording camera. However, in some specific cases, MOs with different focal lengths could also be used [48], but the placement position needs to be adjusted to ensure that the spherical waves are approximately equal in the hologram plane. In this way, the spherical aberration will be fixed by matching the spherical waves curvature of the reference and object waves.

Although the telecentric arrangement in a Mach–Zehnder interferometer could effectively suppress the phase aberration caused by MO, in actual biological holographic imaging experiments, the influence of other aberration, e.g., sample, environment vibration, and off-axis configuration, also needs to be considered. In 2011, Liu et al.[61] proposed an improved DH microscope for long-term quantitative phase-contrast imaging of living cells, as shown in Fig. 6. In this work, the optical configuration is arranged in a telecentric structure and optimized in the form of a free-space-fiber hybrid system, which compensates the quadratic phase aberrations and promotes the flexibility of imaging in the complex or semi-enclosed experimental environment. Herein, the phase aberrations introduced by the imperfect placement of biological culture medium were also taken into account and compensated with digital phase mask in the numerical reconstruction of hologram. The core of this processing is a posteriori surface fitting method based on Zernike polynomials [62]. It could effectively eliminate the residual defocus aberration as well as other primary aberrations.

Fig. 6
figure 6

Reprinted and rearranged by permission from [61]: Laser Physics, Springer Nature, Copyright 2011

At left schematic of DH microscopy system for long-term phase-contrast imaging of living cells in the form of free-spatial-fiber; a hologram; b interference fringe of a labeled by the white rectangle; c wrapped phase image of the cells; d unwrapped phase image of the cells.

Full size image

The above discussions focus on Mach–Zehnder structure-based holographic setup, Qu et al. [63], in their work, proposed a quasi-physical phase compensation method in Michelson interferometric DH configuration. This strategy could work on both reflection-mode and the transmission-mode holographic recording system by means of an adjustable lens. In the reflection-mode setup (Fig. 7a), an adjustable lens is placed in front of laser source and will affect both the reference and object beams, so it works as condenser lens of the MO. In the case of object beam, the shape of illuminating wavefront depends on the position of the lens, and furthermore, the phase curvature produced by the MO will change. Furthermore, the reference beam is reflected by a mirror before entering the beam splitter, its wavefront curvature will be affected by the position of lens and mirror. Therefore, by adjusting the relative position between the lens and the mirror, the authors demonstrated that it is possible to find a correct position to obtain the same spherical curvature for both object and reference beams on the camera plane.

Fig. 7
figure 7

a Schematic of the reflection-mode DH microscopy setup. b Schematic of transmission-mode DH microscopy setup. BS beam splitter, L lens, MO microscope objective, M mirror

Full size image

In the transmission mode (Fig. 7b), optical fiber is used to guide the beam from one light source into two parts. One of them is used as reference beam, which passes through a lens in front of beam splitter; other is used as object beam, which illuminates directly on the sample. In the object beam, the location of the light source is chosen to match the numerical aperture of the MO used. In the reference beam, an adjustable lens is used for the phase curvature control to ensure that the reference beam reaches the CCD plane with the same spherical phase curvature as the object wavefront.

The conventional telecentric structures are mainly working for single-wavelength DH setups, if considering the multi-wavelength situation, an additional algorithm process is needed to assist in phase compensation. Nguyen et al. [64] proposed multiple-wavelength telecentric DH microscopy (MW-TDHM) in reflection and transmission modes. In this work, the parabolic phase distortion caused by the MO in MW-DHM is optically removed, eliminating the need for a second reference hologram to subtract the two-phase maps. Besides, two polynomial models using 2D surface fitting are employed to compensate digitally for chromatic aberration (in the multi-wavelength case) and higher order phase aberrations. The digital automatic aberration compensation is easier to apply compared to non-telecentric configurations.

The above strategies are based on the conventional optical devices, the phase distortion could be eliminated by adjusting the relative position and relationship between the devices. With the development of electronic technology, the emergence of modulated electro-optical devices has brought new possibilities for phase distortion calibration. One strategy is to induce the spatial light modulator for correcting the phase aberrations in DH. Balasubramani et al. [65] and Deng et. al [66] proposed a novel adaptive wavefront correction (AWC) technique implemented by designing and displaying a series of computer-generated holograms (CGH). These CGH are composed of blazed grating with phase Fresnel lens on a phase-only spatial light modulator (SLM). In their work, the aberrations of the optical system are sensed by digital holograms and are used to design the CGH-based AWC to compensate for the phase aberrations of the imaging system, as shown in Fig. 8. This is a closed-loop aberration compensation method, which can effectively suppress the mixed aberration of the recording system.

Fig. 8
figure 8

Reprinted and rearranged by permission from [66]: Optics and Laser Engineering, Elsevier, Copyright 2020

Calibration measurement procedure showing the steps to be followed to achieve the wavefront aberration compensated imaging.

Full size image

New methods in the last years based on adaptive optics have been developed [67,68,69,70,71,72]. Adaptive optics is a scientific and engineering discipline whereby the performance of an optical signal is improved using information about the environment through which it passes. The most common technique to compensate for the various aberrations is based on the principle of the phase conjugation. Let be \(\left|E\right|{e}^{-i\phi }\) the aberrated field; it could be corrected multiplying it by a field which is proportional to itself but with opposite sign, i.e., its complex conjugated \(\left|E\right|{e}^{i\phi }\).

In 2015, Doblas et al. [72] enveloped a method based on the electrically tunable liquid lens (LL) for obtaining accurate phase measurements in the DH system. Therefore, the numerical post-processing to correct the parabolic phase factor is not necessary. The sample is illuminated by a converging spherical wave, such that its focus is conjugated with the front focus of the TL placed after the MO (Fig. 9a). Herein, the spherical wave curvature allows them to compensate for the quadratic phase caused by MO. By adjusting the voltage of the LL, it is possible to produce spherical illumination adapted to any of the interchangeable MOs. Furthermore, the comparison between the results for the telecentric DH with the one for the non-telecentric DH with physical compensation was carried out, noting that the proposed approach is comparable with that of telecentric DH and superior to the non-telecentric one.

Fig. 9
figure 9

a Configuration of physical compensation method in the traditional transmission off-axis DHM. BE beam expander, BS beam splitter cubes; M1 and M2, mirrors; MO, microscope objective; LL, liquid lens; b Illustration of physical method for canceling the phase perturbations in non-telecentric DHMs. The specimen is illuminated by a converging spherical wavefront. ETL electronically tunable lens

Full size image

Moreover, in the application of DH microscopy, it is often called the requirement of using different MO to observe the biological samples; in this case, the system has to face the challenge of different spherical phase factors. Therefore, the LL placed in the reference arm could be a more flexible strategy. Deng et al.[73] proposed a study for this circumstance. The electrically tunable lens (ETL) is placed in the reference arm, as is depicted in Fig. 9b, to produce a spherical wavefront that matched the object wavefront. As mentioned above, depending on the supplied voltage, the ETL generates wavefront with different curvatures without complex mechanical movements or additional numerical post-processing and keeping the system compact. The interference between object and reference wave results that the spherical phase is suppressed and leaves the tilt and the constant phase. The proposed method has been implemented for multiple Mos, demonstrating the correctness and the agreement with the theoretical analysis.

The aberration suppression and compensation in dual-wavelength DH need to consider additional phase distortion factors, which are due to different refractive indices of wavelengths. In the conventional dual-wavelength single-shot structure [74], the beams of two wavelengths share the same object arm, but use different reference arms to generate off-axis interference angles orthogonally. This configuration will cause the holograms of the two wavelengths to have different wavefront phase distortions and increase the complexity of the de-aberration process. One possible strategy is to use a common-path structure to suppress multi-wavelength wavefront aberrations, which was proposed by Tahara et al. [75]. In this study, authors set beams of different wavelengths to be transmitted in the same off-axis interferometer structure, and then use the difference in refractive index to separate the carrier frequency information of different wavelengths in the spectrum. By giving the accurate analysis of the carrier frequency in the spectrum, the matching of the multi-wavelength wavefront aberration was realized, and the problem of multi-wavelength aberration was downgraded to the single-wavelength level. Furthermore, Wang et al. [76] proposed a common-path polarization-multiplexing setup to simplify the processing of dual-wavelength aberrations. In this recording system, a single camera screen was divided into two parallel areas to record holograms of two wavelengths, respectively, and then an interferometer-like structure was used to adjust the wavefront aberration of the two reference beams before the camera. In this way, the generation of multi-wavelength off-axis aberrations is more effectively suppressed during the recording process.

In addition to the conventional Mach–Zehnder interferometer structure, lateral shear interferometry (LSI) can also be combined with the common-path idea to achieve aberration suppression in dual-wavelength imaging. In 2016, Di et al.[77] showed a method based on the combination of dual-wavelength common-path digital holography and LSI. The two laser beams were coupled into an optical fiber, passes through a lens, and then through the sample in LSI structure. This is a more concise and efficient dual-wavelength hologram acquisition strategy, which uses fewer optical devices and therefore produces fewer higher order aberrations compared to conventional methods. Although the effective space bandwidth is suppressed, but the LSI structure can produce a smaller off-axis interference angle, which will allow the use of a smaller pixel size camera without complex system adjustment.

3 Numerical methods remove or compensate wavefront aberrations from phase-contrast maps

As one of the state-of-the-art imaging technologies, the numerical reconstruction of digital holograms, recorded by a CCD or a CMOS camera, has been investigated in the last decades. In DH, it is possible studying simultaneously the phase and the amplitude of the complex field numerically. It also favors adjusting the complex field of the recorded wavefront. Thus, DH suggests new possibilities in optical metrology by making the phase of the field available, the possibility of managing phase in all DH acquisition makes possible the removing of aberrations. There are lots of studies which proved that wave front curvature introduced by optical instruments can be successfully removed and/or compensated numerically [23,24,25, 49, 78,79,80,81,82]. In this section, compensation of aberrations by numerical algorithms in the conventional DH setups will be discussed. The significance of these studies is that for holographic systems with limited space or limited imaging environments, numerical aberration compensation can take the place of system optimization and complete phase calibration.

Generally, after the conventional process of numerical reconstruction, including phase unwrapping, the unwrapped phase could be written in this form

$$\begin{array}{c}\phi \left(x,y\right)={\phi }_{\mathrm{S}}\left(x,y\right)+{\phi }_{\mathrm{T}}\left(x,y\right)+{\phi }_{\mathrm{P}}\left(x,y\right),\end{array}$$
(3)

where \({\phi }_{\mathrm{S}}(x,y)\) is the sample phase, \({\phi }_{\mathrm{T}}\left(x,y\right)={k}_{x}x+{k}_{y}y\) is the contribution off-axis angle, where \({k}_{x}\) and \({k}_{y}\) are the carrier frequencies in \(x\) and \(y\) directions, and \({\phi }_{\mathrm{P}}(x,y)\) is the phase curvature due to the optical elements of DH microscopy setup.

In the case of phase tilt error, the common solution to solve it is spectrum centering [83], and it consists of shifting the carrier frequency of the virtual image to the center of spectrum, but its precision is always limited to one pixel which needs further sub-tilt compensation. For this reason, other numerical methods were developed which can remove off-axis tilt and compensate for first-order aberrations. They are based on digital phase mask (DPM) either in the reconstruction plane or in hologram plane [78, 84], double exposure[49], 2D fitting procedures using standard spherical surface or Zernike polynomials [85], principal component analysis [86], and on the extraction of the tilt information from the interference pattern in the phase image [87] both in monochromatic and in multi-wavelength configuration [88, 89].

In 2003, a method to compensate phase tilt error and wave front curvature aberrations has been proposed by Ferraro et al. [49]. This method could be used at any reconstructed image plane by means of a phase mask. First, the circular fringe carrier from the recorded hologram was removed and the correct wave front in the hologram plane was found. Then, a nonlinear fit of the unwrapped phase was performed, so that the phases of the numerically reconstructed wave fronts, relating to the recorded hologram and correcting wave front, could be evaluated at distance \(d\). Meanwhile, a method based on double exposure was also proposed in this study. The first acquired hologram \({h}_{1}(x,y)\) is made of the sample, whereas the second one \({h}_{2}(x,y)\) is made of a flat reference in proximity of the sample. The second acquisition is a hologram which contains information only about the wave front and all the aberrations introduced by the optical devices of the setup. Thanks to the manipulation of two holograms acquired, it is possible to remove most of the wavefront aberrations. Both the holograms were reconstructed at a certain distance \(d\), calculating numerically the diffraction integral in the Fresnel approximation, obtaining the two complex wave field, then the phase difference between the two was calculated as

$$\begin{array}{c}{\phi }_{\mathrm{o}}\left(\nu ,\mu \right)=\mathrm{arg}\left[\frac{{O}_{1}\left(\nu ,\mu \right)}{{O}_{2}\left(\nu ,\mu \right)}\right],\end{array},$$
(4)

where \({\phi }_{\mathrm{o}}(\nu ,\mu )\) are the coordinates in the constructed image plane, \({O}_{1}(\nu ,\mu )\) and \({O}_{2}(\nu ,\mu )\) are, respectively, the two complex wave fields calculated numerically, and \(arg\) states for the argument of the complex function \(\frac{{O}_{1}\left(\nu ,\mu \right)}{{O}_{2}(\nu ,\mu )}\). This method compensates for the inherent wavefront curvature completely, but it needs an additional hologram recording without the sample.

Other methods based on the use of a phase mask in either the reconstruction plane or hologram plane have been developed. In 2006, Colomb et al. [84] presented an automatic procedure to compensate for all phase aberrations in DH by computing a polynomial digital phase mask (DPM). The DPM is crested directly from the hologram. Its parameters are calculated automatically and without knowing of physical values of setup such as wave vectors, focal lengths, or distances. They extract automatically the parameters which define the DPM from the hologram by means of curve-fitting procedures applied to phase data. In particular, they analyze the line profiles located in areas where the sample contributions were considered constant. The curve-fitting procedures are implemented to reconstruct phase data. Then, by studying the line profiles from fitting, the DPM of the phase could be defined automatically. Herein, the DMP could be written as a polynomial complex function:

$$\begin{array}{c}{\Gamma }{{^{\prime}}}\left(m,n\right)=\mathrm{exp}\left(-i\sum_{\alpha =0}^{H}\sum_{\beta =0}^{V}{P}_{\alpha ,\beta }{m}^{\alpha }{n}^{\beta }\right),\end{array}$$
(5)

where \(m\) and \(n\) are integers \(\left(-\frac{N}{2}\le m,n\le \frac{N}{2}\right)\), \({P}_{\alpha ,\beta }\) define a set of reconstruction parameters, and \(H\) and \(V\) represent the polynomial orders in the horizontal and vertical directions, respectively. \(\Gamma {^{\prime}}(m,n)\) has constant amplitude and a phase which is defined by a 2D polynomial as reported in \((5)\). The estimations of \({P}_{\alpha ,\beta }\) parameters provide information about the off-axis geometry and the experimental setup, and could be obtained by fitting polynomial curves along the selected area. Where the contribution of the sample to the phase can be considered constant. Once obtained the \({P}_{\alpha ,\beta }\) parameters, it is possible to construct the phase mask \(\Gamma {^{\prime}}(m,n)\) and multiply it by the reconstructed wavefront. Finally, the corrected wavefront and then the phase distribution of the sample could be obtained.

Moreover, Colomb et al. [78] defined, in another work, a numerical parametric lens (NPL) placed in the hologram plane and/or in the image plane that provides a complete compensation for aberrations. The NPL used is based on Zernike polynomials models whose parameters are obtained by a 2D fitting procedure as described previously. This approach provides the correction of the tilt in the hologram plane and the complete compensation for most of the aberrations obtained could be implemented for any reconstruction plane. In the procedure, it assumed that the sample did not introduce aberration but only a phase shift \({\phi }_{\mathrm{S}}(x,y)\) and the aberrations not influenced the amplitude of the waves. For this reason, it was possible to represent the object and the reference wave as

$$\begin{array}{c}O\left(x,y\right)=\left|{\varvec{O}}\right|\mathrm{exp}\left[i{\phi }_{S}\left(x,y\right)\right]\mathrm{exp}\left[i{W}_{\mathbf{O}}\left(x,y\right)\right],\end{array}$$
(6)
$$\begin{array}{c}R\left(x,y\right)=\left|{\varvec{R}}\right|\mathrm{exp}\left[i\left({k}_{x}x+{k}_{y}y\right)\right]\mathrm{exp}\left[i{W}_{\mathbf{R}}\left(x,y\right)\right],\end{array}$$
(7)

where \({W}_{\mathbf{O}}(x,y)\) and \({W}_{\mathbf{R}}\left(x,y\right)\) are the phase aberration terms of object and reference wave, respectively. Then, the authors considered the filtered hologram, i.e., the hologram that contains only the virtual image term \({I}_{\mathrm{H}}^{\mathrm{F}}={{\varvec{R}}}^{\boldsymbol{*}}{\varvec{O}}\), whose phase contains the aberration term \(W={W}_{\mathbf{O}}-{W}_{\mathbf{R}}\) which must be suppressed by applying the fitting procedure to the filtered hologram phase. It is worth noting that NPL is not applied in the image plane but in hologram plane. The correction in the hologram plane avoids the use of NPL in the image plane, because it compensates for most of the aberrations at the same time. In addition, NPL acts as a sort of a physical lens, which placed in the hologram plane, and thus provides a magnification of the image, it could also compensate for the astigmatism induced by a Cylindrical Lens.

Miccio et al. [85] showed a procedure limited to the special case of thin object, simple but effective. It does not require twice recording of holograms and the selection of flat areas to perform the fitting. Once recorded a single hologram and performed the numerical reconstruction of the phase, the unwrapped phase distribution at distance \(d\) from the hologram plane could be obtained. The phase distribution \(\phi \left(\nu ,\mu \right)\) is the sum of the contributions

$$\begin{array}{c}\phi \left(\nu ,\mu \right)={\phi }_{\mathrm{S}}\left(\nu ,\mu \right)+{\phi }_{\mathrm{D}}\left(\nu ,\mu \right)+W\left(\mu ,\nu \right),\end{array}$$
(8)

where \({\phi }_{\mathrm{D}}\left(\nu ,\mu \right)\) is the quadratic contribution of the defocus aberration and \(W\left(\nu ,\mu \right)\) represents a higher order aberration contribution. In the case of a very thin object, the sample phase \({\phi }_{\mathrm{S}}\left(\nu ,\mu \right)\) is a small perturbation of the aberration contribution. For this reason, a fitting procedure of the \(\phi \left(\nu ,\mu \right)\) is a good approximation of the aberration contribution and \({\phi }_{\mathrm{S}}\left(\nu ,\mu \right)=\phi \left(\nu ,\mu \right)-{\phi }_{\mathrm{appr},a}(\nu ,\mu )\), where \({\phi }_{\mathrm{appr},a}(\nu ,\mu )\) is the approximation of the phase distribution at distance \(d\) by the hologram plane. A 2D nonlinear fitting procedure based on a linear combination of the Zernike polynomials is employed and showed results obtained for different terms used to fit the surface. This simple technique could be useful for the inspection and investigation of the very thin object such as biological sample.

Ferraro et al. [90] showed a method based on the concept of LSI applying a numerical shift in the image plane; in this way, an interferometric shearogram could be produced to suppress the aberrations. It was possible to retrieve the phase map of the sample without spherical aberrations. Furthermore, Coppola et al. [91] presented a similar method based on wavefront folding. They generated a numerical replica of the original complex wave field. Selecting and shifting an area of the wavefront where the contribution of the observed sample was null. Then, they used this area as a reference phase, because it was a flat region where the phase should be uniform and constant. The difference between the reference phase and the original one provided a linear phase term that was easily removed by a linear 2D fitting procedure. Both of the above two methods can be classified as a "self-calibration" process; compared with other numerical processing methods, their aberration suppression process is simpler and can achieve better results through multiple iterations.

The techniques described above usually require some special conditions for implementing, e.g., knowledge of the setup, manual operation, or only work for a particular type of sample. In 2013, a method based on principal component analysis (PCA) was enveloped [86]. The approach was applied to a DH system based on a Michelson interferometer in transmission configuration, but it could also be extended to the other configurations. One can write the recorded hologram as relation (1) and can extract the filtered virtual image whose phase has two contributions, the first due to the sample and the second to the aberrations due to the off-axis geometry and to spherical curvature

$$\begin{array}{l}{I}_{\mathrm{H}}^{\mathrm{F}}={{\varvec{R}}}^{*}O=\left|R\right|\left|O\right|\mathrm{exp}\left[i{\phi }_{\mathrm{S}}\left(x,y\right)\right]\mathrm{exp}\left[i\left({\phi }_{\mathrm{T}}\left(x,y\right)+{\phi }_{\mathrm{P}}\left(x,y\right)\right)\right].\end{array}$$
(9)

Let be the phase aberration matrix (PAM)

$$\begin{array}{l}Q\left(x,y\right)=\mathrm{exp}\left[i\left({\phi }_{\mathrm{T}}\left(x,y\right)+{\phi }_{\mathrm{P}}\left(x,y\right)\right)\right]=\mathrm{exp}\left[i\left({k}_{x}x+{k}_{y}y\right)\right]\mathrm{exp}\left[i\left({l}_{x}{x}^{2}+{l}_{y}{y}^{2}\right)\right].\end{array}$$
(10)

According to the authors, the ideal model which represents \(Q\left(x,y\right)\) is a rank one matrix. Two vectors are defined: \(p\left(x\right)=\mathrm{exp}\left[i\left({k}_{x}x+{l}_{x}{x}^{2}\right)\right]\) and \(q\left(y\right)=\mathrm{exp}\left[i\left({k}_{y}y+{l}_{y}{y}^{2}\right)\right]\) and the authors write the PAM as \(Q\left(x,y\right)=p{q}^{\mathrm{H}}\), i.e., the product between \(p\) and the complex conjugate of \(q\). The role of the PCA is to reduce the number of variables, limiting as possible the loss of information. In this case, the PAM should have just one principal component. Once the coefficients of \(Q(x,y)\) have been determined separately, by means of least-squares fitting, one can calculate its conjugate \({Q}^{\mathrm{H}}\left(x,y\right)\), multiply it for the filtered hologram \({I}_{\mathrm{H}}^{\mathrm{F}}\) and get a virtual image without aberrations. As shown in Fig. 10, the method was applied to an experiment on human macrophage cells, and then, various stages of the cells have been shown. Figure 10a presents the reconstructed phase by rank one approximation from the first dominant singular vectors, Fig. 10b, c shows the reconstructed phase with the contribution of the second and third dominant components, respectively. After the fitting procedures in Fig. 10f is presented the final estimated phase aberration and then by multiplying \({Q}^{\mathrm{H}}\) with the sub-sampled hologram and substituting the original spectrum with the reconstructed one. The spectrum obtained in Fig. 10g presents a compensation for all aberrations. In sum, the authors have proposed a method based on PCA that can be made automatic and enable a fast processing of the data thanks to the removal of the redundant data. In addition, the separating aberration terms to two singular vectors make possible the reduction of the computational complexity, which makes the fitting procedure a one-dimensional problem.

Fig. 10
figure 10

Reprinted with permission from ref. [86] © The Optical Society

Phase aberration compensation using the proposed PCA algorithm. a The rank one phase aberration approximation formed from the dominant singular vectors. b, c Phase reconstructed from the first two and three sets of the dominant singular vectors, respectively. d, e The unwrapped left and right dominant singular vectors and their corresponding quadratic fitted ones. f The obtained phase aberration map. g Fourier spectrum after aberration compensation. h Reconstructed phase map.

Full size image

In fact, the combination of some conventional methods can also create new possibilities and achieve better aberration suppression. Trujillo et al. [92] proposed an automatic method that fully compensates the quantitative phase measurements in off-axis DH microscopy, where the curvature phase flaw introduced by the MO is avoided by the use of an optimized telecentric imaging system for the recording of the holograms and the remaining phase perturbation due to the tilt of the reference wave is removed by the automatic computation of a digital compensating reference wave. The core of this strategy is a more precise processing of spectrum positioning algorithm; it will allow faster aberration suppression. In 2020, Leiping et al. [93] proposed an automatic method to compensate for the residual phase aberrations in telecentric DH microscopy by combining background automatic detection and Zernike polynomial fitting. They use a simple threshold operation to automatically detect the specimen-free regions via a digital synthetic differential interference image, which is generated by shearing the wavefront of the object wave along with multiple directions. Then, the residual phase aberrations are fully compensated by subtracting the digital phase mask estimated from the phase extracted by the binary mask. This is an effective numerical strategy in the practical application of DHM, which allows a quick aberration correction process.

With the deep integration of holographic technology and other traditional imaging technologies, some emerging methods have also begun to be paid attention to, phase-contrast tomography (PCT) is one of them. It allows 3D microscopic imaging at the subcellular level. In the conventional PCT imaging process, since the sample needs to be scanned by the laser beam, a new type of phase aberration will be introduced due to different illumination angles. Baek et al. [94] proposed a method to get pupil information by utilizing the cross-spectral density between optical fields at different incident angles demonstrating an aberration-free quantitative phase imaging in two and three dimensions. This strategy will effectively suppress the extra aberrations caused by multi-angle MO illumination during PCT recording.

Some purely numerical methods have been proposed to compensate for most of phase aberrations of any order in an automatic way and without specimen-free zones necessary [95]. Among these in ref [96], an inverse problem approach has been used to fit the typical aberration surface to compensate for all aberration and solved an \({\mathcal{l}}_{1}\)-norm-based optimization. It was enveloped a procedure [79] that first for compensating aberrations carries out least-squares fit procedure. Then, the flat phase background of the image is retrieved by means of phase gradient and segmentation methods. Some widely used methods are applied to quantitatively evaluate the needed compensation [97]; also, the minimizing total standard deviation of the phase map could be used to confirm [98]. In addition, geometrical transformation methods for unwrapping phase are used to compensate for aberrations automatically and effectively without complex numerical fitting procedures acquiring different phase-shifted holograms [99] or applying geometrical transformation to a single recorded hologram [23, 100].

Finally, Goud et al.[95] in their work proposed a simple method for compensation of spherical aberrations induced by MOs. They integrated DH in a commercial pathology microscope, implementing an off-axis interferometric configuration, and proposed a defocus hologram method. They acquire two holograms: one is acquired with object in the correct focus plane. Another hologram is acquired with an object afar from the correct in-focus position. This procedure allows to retrieve object complex field at the image plane in perfect in-focus condition and in complete defocus condition. The former includes both specimen phase contribution and phase distortion. The latter includes only phase distortion due to MO. Thus, it is possible to retrieve the phase of specimen without phase distortion multiplying the two complex object fields one with the conjugation of the other and considering the argument of the result. This method is simple and particularly useful when it is not possible to acquire specimen-free region.

In summary, phase aberrations in DH microscopy can be compensated by either the optical method or numerical method. Obviously, averting phase aberrations in hologram recording process by optical method would certainly relax the computation of the reconstruction of hologram, making the task of real-time reconstruction of hologram easier. However, to obtain satisfactory reconstructions using the optical method, the optical system should be in strict alignment, which is a sophisticated operation. Different with optical method, numerical method compensates phase aberrations in a flexible way, thus is potential to remove most kinds of aberrations. The numerical method surely increases the consumption of time and computation of hologram reconstruction and may behave unreliably when the aberrations are severe and the signal-to-noise of hologram is low. Nowadays, a better phase aberration suppression strategy is achieved by artificial intelligence.

4 Aberration calibration by artificial intelligence

Artificial intelligence (AI), also called machine intelligence, has dramatically changed the landscape of science, industry, defense, and medicine in the last several years. With active advancements in computer and computational technologies, the field of AI has shifted from mostly theoretical studies in the discipline of computer science to real-life applications spanning many areas. Especially, as a subfield of AI, deep learning has already assisted researchers to make rapid predictions based on available databases rather than by time-consuming theoretical calculations or costly and labor-intensive experiments. In recent years, this technology has developed rapidly, and very significant achievements have been made in areas such as autonomous driving, natural language processing, computer vision, and many more. Currently, this powerful tool has also been shown to benefit the DH imaging, and it has already been applied to various important tasks in DH, such as phase recovery [18, 101], framework of holographic reconstruction [102, 103], autofocusing [104] and depth of field enhancement [105], and so on.

In recent years, AI methods also have been applied in aberration compensation in DH. They are generally enabled by supervised optimization of deep convolutional neural networks (CNNs) using labeled images, such as object-background segmentation images [106, 107] or aberration-free images [108, 109], which is the ground truth for the training. The two unique structures of CNNs are convolutional layers and pooling layers. The function of the convolution layer is to extract features from the aberrated image, which contains multiple convolution kernels. Each element of the convolution kernels corresponds to a weight coefficient and a bias vector. Through the process of training, the weights and biases of the neural network are adjusted. The convolution layer also contains nonlinear activation functions to help express complex features and further to fit arbitrary functions, in this case, object segmentation and aberration fitting. The pooling layer down-samples the extracted feature maps to reduce parameters and expand the perception field. The error between the network output and the target labels could be minimized, in terms of a user-defined cost or loss function. After the CNN is trained, the prediction process typically takes only a fraction of a second without the need for any iterations and manual intervention.

As shown in Fig. 11, Nguyen et al. [106] proposed a supervised background detection CNN combined with Zernike polynomial fitting (ZPF) to automatically obtain aberration-free phase image in DHM. It is based on U-net model to compensate all the phase aberrations without manual cropping. To prepare the training data set for CNN, the holograms were filtered in the Fourier domain for the + 1 order component, and then, inverse Fourier transform and phase unwrapping were used to obtain the unwrapped phase images (1024 × 1024). The input of CNN is the sub-sampled phase aberration images (256 × 256) and their corresponding ground truth is the binary segmentation images of the object and background. Then, the predicted background of the trained CNN model is fed into the ZPF to calculate the conjugated phase aberration (256 × 256). The aberration is compensated in the spatial domain by multiplying the Inverse Fourier Transform of the cropped + 1 spectrum order with the complex exponential term of the conjugated phase aberration. The Fourier Transform of the compensated spectrum is centered and zero-padded to the original image size of 1024 × 1024. Finally, the angular spectrum method is used to reconstruct the full-sized, aberration-free phase image of hologram. The authors claimed that the deep learning CNN model for image segmentation is more robust of the overwhelming phase aberration image in DHM.

Fig. 11
figure 11

Reprinted with permission of ref. [106] © The Optical Society

Image reconstruction strategy with phase aberration compensation based on background detection CNN + ZPF.

Full size image

Ma et al. also proposed a CNN-ZPF numerical method to compensate off-axis phase aberration, especially for microstructures with small area small background, or dense phase distribution [107]. In this method, the sample and background area are recognized and segmented directly from the hologram instead of the reconstructed phase image. It can eliminate the influence of the measured object on the phase aberration compensation. Then, ZPF is executed on the background area of phase image to implement the whole process of phase aberration compensation. The training dataset consists of 1000 holograms with several samples that had specific morphological characteristics. The ground-truth were the binary images of object and background area segmented manually. Then, a U-net architecture is trained to automatically segment the holograms. The experimental results of the trained CNN model and ZPF show that the total phase aberrations can be compensated only by requiring a simple hologram without manual intervention. Compared with the previous process, this is a streamlined strategy to improve overall efficiency. The entire processing can be accelerated by splitting the image in advance.

In 2020, Xiao et al. [108] proposed a CNN network for multivariate regression based on the resnet-50 model structure to cope with the phase aberration compensation task. As shown in Fig. 12, they transform the problem of estimating the polynomial coefficients for constructing a phase aberration map to a regression problem. The phase images with aberrations are reconstructed from the holograms and zoomed out to 256 × 256. Then, these aberrated images are fed into the CNN and the labels are the polynomial coefficients calculated by fitting the object-free area of the original phase image. Herein, the aberration map is built by the output coefficients of trained CNN with polynomial fitting. The size of the fitting aberration map is 1024 × 1024 just as the original hologram. As the result, the aberrations could be removed directly by subtracting the aberration map from the reconstructed phase image. This method allows the phase compensation task to complete without detecting background regions and knowing any physical parameters about the setup. It is more suitable for the phase maps which do not have much phase fluctuation, etc., biological samples.

Fig. 12
figure 12

Reprint with permission from [108] Lab on a Chip, Copyright 2021

The strategy of aberration compensation based on multivariate regression CNN.

Full size image

In the above two deep learning strategies, the inputs of CNN are both aberrated phase images and deep learning method is used to optimize a part of the phase reconstruction process. In 2019, Ren et al. [109] proposed an end-to-end deep learning framework to reconstruct noise-free phase images from raw holograms. As shown in Fig. 13, the recording holograms of different sectional samples are directly fed into the proposed holographic reconstruction network (HRNet) and it can automatically learn internal representations of the necessary processing steps in holographic reconstruction, such as spectrum filtering, aberration compensation, and phase unwrapping. Then, the output of HRNet is the quantitative phase image with high quality and aberration-free. Besides, the authors also tested the performance of the trained network with different incident angles and distances of holograms. The reconstructed images output from the network is still in good quality, demonstrating that the deep learning network can learn the underlying characteristics of digital holograms.

Fig. 13
figure 13

Schematic of the end-to-end deep learning framework HRNet[109]. Request sent waiting for permission

Full size image

5 Conclusions

DH setups may be affected by numerous optical aberrations due to diverse sources such as optical lens and MOs; for this reason, in the years, numerous scholars have investigated aberrations and enveloped methods to compensate for induced aberrations and to make DH an investigating optical technique applicable in any field with high-resolution imaging results, not only in axial but also lateral. In this work ,we reviewed numerous methods, starting from milestones up to the most recent ones, to compensate for all type of aberrations based on different approaches and sorting them into four categories: (i) optical methods, (ii) numerical methods, (iii) hybrid methods, and (iv) methods based on AI and highlighting their advantage and defects. Table 1 shows a side-by-side comparison of different strategies; it can provide a reference for experiments that require aberration suppression.

Table 1 Comparison of different aberration compensation strategies
Full size table

For the first category, we have mostly analyzed the optical setups to compensate for the type of aberrations introduced by MOs based on telecentric arrangements. The optical method is a conventional strategy for aberration suppression; it is simple and easy to implement. Although it is difficult to effectively compensate all kinds of aberrations using only optical calibration, but for experiments with high temporal resolution requirements, optical method is the best strategy. The numerical method is the mainstream aberration suppression strategy for aberrations such as off-axis aberration and lens aberration. In the measurements with relatively simple optical systems, e.g., DH for lab on a chip, it is a reliable way of dealing with aberrations. The hybrid methods are wise to combine both optical method and numerical method to tackle phase aberration compensation problem, since they show strong complementarity to save computational time and to avoid the use of complicated and expensive optical setups. The AI method is a state-of-the-art strategy; it can provide efficient aberration suppression for all types of aberration. For the repeated experiments or the experiments pursuing ultimate imaging accuracy, the AI method shows great potential for the best aberration method. It has been widely applied in the field of DH to solve various tasks such as phase recovery, hologram reconstruction, autofocusing, and so on, as we reported.

All these efforts that we have reported, allowed DH to provide higher and higher resolution imaging and to obtain more and more precise information in retrieved quantitative phase imaging. In addition, we also found that the most recent approaches are focusing on automatic compensation methods to minimize the researchers' work during the post-acquisitions’ work.