Introduction

Imaging through scattering is one of the biggest challenges facing optical technologies, which plays crucial roles in many fields including biomedical, and astronomy applications [1,2,3]. In the past decades, there have been increasing interests in coherent and incoherent imaging through scattering media. To enable imaging through this challenging condition, it is an emerging topic to extract spatial information through scattering by employing the active control of light with the aid of prior knowledge of objects or media, or combined with post-imaging recovery processes, i.e., computational imaging of hidden objects [4,5,6,7,8,9,10,11,12]. Many powerful computational imaging approaches have been reported, utilizing interference (such as holography [13]) and interference-less methods [14], transmission matrix [15, 16], active wavefront shaping techniques [17, 18] for precise light control and smart computational imaging through scattering media [19,20,21,22,23,24]. In these techniques, strict stability of operation with complex optical setups are required. To overcome these technical challenges, speckle-autocorrelation imaging (SAI) [25], deconvolution [26,27,28] and cross correlation [29,30,31] approaches were developed. In particular, the SAI method received extensive attention due to their advantages of single-shot, easy-operation, low-cost, and calibration-free properties. However, although the object’s image was recovered using SAI, its orientation remained unknown due to the random-initial guess and indeterministic iterations in phase retrieval (PR) [25, 32]. Based on SAI, various recovery algorithms [30, 33] and engineering of the point-spread-function (PSF) for imaging are still under active investigation for multitargets imaging [34], depth resolved [35] and three-dimensional (3D) imaging [36], and multispectral imaging [37]. However, all aforementioned imaging technologies through scattering media require sophisticated optical setups (e.g. [4,5,6,7,8,9,10,11,12,13,14,15,16]) or time-consuming post computational reconstruction with carefully adjusted parameters for the pre-processing of the speckle and/or the autocorrelation (e.g. [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37]). In those previously reported strategies (e.g. [25,26,27,28,29,30,31,32,33,34,35,36,37]), directly obtained random speckle patterns have never been used for imaging due to severe degradation and multiple scattering of photons. Speckles were always considered as scrambled patterns with unseen chaos.

In this work, we report a ‘counterintuitive’ strategy to directly observe self-imaging units of small objects using speckles through diffusers. By carefully examining this speckle (which was considered as noise or chaotic patterns in previously reported imaging strategies), clear self-imaging phenomena can be directly obtained from a single shot of the speckle image, releasing the needs in time-consuming image acquisition and computational reconstruction. As a result, directly seeing through random diffusers with the naked eye and real-time video imaging of multitargets are realized. The orientation of the object can be directly seen and traced in real-time. Intriguingly, this method requires no complex equipment for the active control of light, nor prior knowledge of medium or pre-calibration of the system, with no need of iterations and parameter adjustments. Our results provide new perspective of diffuser-imaging systems, and suggest novel imaging applications through scattering diffusers.

Results

Figure 1(a) shows the diagram of a simple optical system for incoherent imaging through a thin-diffuser scattering medium (similar to those used in ref. [25,26,27,28,29,30,31,32,33,34,35,36,37]). By moving the camera along the Z direction, different slices of the volume-speckles can be captured at different depths of z. In this experiment, a standard USAF resolution test object (1951USAF, Edmund,1.2 mm in length) was illuminated by a collimated LED light source (M625L3, Thorlabs, with the central wavelength of λ = 625 nm and the bandwidth of ~ 16 nm) with an object-distance u = 50 cm to the diffuser. The speckle through a glass diffuser with 220-grit (DG10-220-MD Thorlabs) was recorded by a camera (Zyla 5.5, Andor) placed at a distance of v = 10 cm to the diffuser (see supplement information (SI) Note 1 and Fig. S1 for more experimental details). Figure 1(b) and (c) show the captured original raw speckles (first column) of different isolated numbers ‘1’ (Fig. 1b) and ‘3’ (Fig. 1c), respectively. One can see typical grainy and scrambled patterns which were widely observed in previous reports (e.g. [25,26,27,28,29,30,31,32,33,34,35,36,37]). However, in those previously reported imaging strategies (e.g. [4, 25,26,27,28,29,30,31,32,33,34,35,36,37]), digital-filtering and computational reconstructions with carefully tuned parameters were applied to process their raw speckles. Surprisingly, by analyzing enlarged regions in the speckle in our experiment, we recognized that those extra computational efforts are unnecessary! Self-reproduction of the objects can be observed directly (see black-rectangle-boxes in Fig. 1(b, c)). As shown in the second and third columns of Fig. 1(b-c), the self-reproduction of the object with perfect profiles can be identified from the speckle. In particular, the right column of Fig. 1(b-c) highlights the clear lineament. Although their contrasts are not ideal, self-reproduction of the object was directly observed in the chaotic speckle, which has never been reported.

Fig. 1
figure 1

Direct observation of self-imaging in raw speckles. a The schematic of the diffuser imaging system. b-d Experimental observation of self-imaging in speckles. Camera captured original raw speckles of isolated objects of (b) digit ‘1’, (c) digit ‘3’, and (d) multi-objects ‘2’ and ‘6’

Full size image

To confirm this self-imaging effect, we further recorded the speckle of multi-objects: As shown in Fig. 1(d), self-imaging patterns of two digits of ‘2’ and ‘6’ were identified. By analyzing zoomed-in regions, one can see that random and chaotic speckle patterns contain rich self-imaging units. These units were generated by different extents of degenerated, out-of-focus, and incomplete self-imaging units with random position-shifts and scaled transmission intensity. As shown in Fig. S2, S3, S4 and S5 in the supporting information, we recorded self-imaging patterns of varied objects through diffusers at different wavelengths. Intriguingly, by carefully analyzing previously reported experimental data (e.g. [25,26,27,28,29,30,31,32,33,34,35,36,37]), this type of self-imaging patterns was also recorded but mostly omitted! The key question is: what is the mechanism of this self-imaging phenomenon?

Physical mechanism of self-imaging in speckle

To reveal the physical mechanism of the self-imaging phenomenon, here we explore the model for the incoherent diffuser imaging system. As shown in Fig. 2, we compare the incoherent imaging model of lens-system and a diffuser-system. A classic lens-imaging system will result in a point-to-point mapping (Fig. 2a) or object-to-image mapping (Fig. 2b). In contrast, the output of a diffuser-imaging system is a speckle for both a point source (Fig. 2c) or an object source (Fig. 2d). In this speckle, many ‘hot-spot’-like grains are obtained due to the constructive interference of incident beams from different optical paths (see Fig. 2c). This speckle can be simplified as a superposition of shift-and-scaled object’s self-reproduction images with degeneration, aberrations, and vignetting. Similar to the conventional optical lens imaging system (Fig. 2a and b), the diffuser imaging system can also be presented as a convolution. However, due to the random phase bumps and distributions of the diffuser, the ‘hot-spot’ (Fig. 2c) and the random self-imaging speckles (Fig. 2d) were induced by the interference of multiple scattered light. Next, we develop an incoherent-imaging model to interpret the diffuser imaging system.

Fig. 2
figure 2

Operational principles of diffuser-lens imaging systems. The schematic of (a) the PSF for (b) the lens imaging system. (c) The ‘hot-spot’ effect in the diffuser imaging speckle, and (d) the imaging model and experimental setup for incoherent illumination. (e-f) Random phase screen model (e) without features of phase distribution, and (f) with lenslet array phase features. The speckle in (e) is generated by calculating the light-propagation through phase-screen [38]. The speckle in (f) is generated considering the lenslet-phase with incoherent-imaging model using Eq. 1

Full size image

By illuminating a transmission-type point or object O using a spatial incoherent light, the scrambled speckle I after the diffuser can be described as [25]:

$${I}_{\lambda }={O}_{\lambda}\ast {PSF}_{\lambda }$$
(1)

where’*denotes a convolution operation. No matter how many objects are within a single optical memory effect (OME) zone, they share a common PSF [25, 29, 33,34,35,36,37]. Using the SAI model and the convolution theory in SAI method [25], we get

$${I}_{\lambda}\bigstar {I}_{\lambda }=\left({O}_{\lambda}\bigstar{O}_{\lambda}\right)\ast \left({PSF}_{\lambda}\bigstar {PSF}_{\lambda}\right)=\left({O}_{\lambda}\bigstar {O}_{\lambda}\right)+C,$$
(2)

where’★‘denotes the autocorrelation operation, and the ‘C’ represents the background term. The object’s diffraction-limited image can then be reconstructed from speckle’s autocorrelation by the SAI method using an iterative phase-retrieval algorithm [25, 39, 40]. Here we introduce a modification of the SAI imaging with the self-imaging model to investigate the self-imaging formed speckle and speckle’s autocorrelation. An array of identical objects O are introduced at varied random positions (xn’, yn,’ z) to represent the pattern of the speckle I′. The self-imaging modeled speckle can then be represented as:

$${I_{\lambda}}^{\prime}\left({x}^{\prime },{y}^{\prime },z\right)=\sum_{n=1}^N{O}_{\lambda}\left({x}^{\prime },{y}^{\prime },z\right)\ast \delta \left({x}^{\prime }-{x}_n^{\prime },{y}^{\prime }-{y}_n^{\prime },{z}^{\prime }-z\right)$$
(3)

In this case, one can easily show that the self-imaging modeled speckle’s autocorrelation gives

$${I_{\lambda}}^{\prime}\bigstar {I_{\lambda}}^{\prime }={F}^{-1}{\left|F\left({I_{\lambda}}^{\prime}\left({x}^{\prime },{y}^{\prime },z\right)\right)\right|}^2={F}^{-1}{\left|F\left({O}_{\lambda}\right)F\left(\sum\limits_{n=1}^N\delta \left({x}^{\prime }-{x}_n^{\prime },{y}^{\prime }-{y}_n^{\prime },z\right)\right)\right|}^2$$
(4)

where F represents the Fourier transformation operation, and F−1 is its inverse. Using Wiener-Khinchin theorem [25, 31,32,33,34], this equation can be simplified as

$${I_{\lambda}}^{\prime}\bigstar {I_{\lambda}}^{\prime}\cong {C}^{\prime}\bullet \left({O}_{\lambda}\bigstar {O}_{\lambda}\right),$$
(5)

where, ‘C″ is a constant coefficient. By comparing Eqs. 2 and 5, one can see that the speckle’s autocorrelation is exactly the original object’s autocorrelation (Eq. 5) with intensity-stretch but no background term nor cross-talks (Eq. 2). This relation between the object and its speckle provides clues to model the diffuser as a collection of randomly distributed lenses with a random phase-mask. As a result, the speckle can be treated as an accumulated random self-reproduction (Eq. 3) of the diffuser-modulated self-imaging of the object. With these self-reproduced images, degenerated and dimmed partial self-image units, the accumulation of the overlapped units forms the final scrambled speckle. It is therefore feasible to find partial or complete images (i.e. hot spots) after the phase-modulation by the diffuser: i.e., the emergence of clear self-imaging of the object (i.e. hot spots) in the scrambled speckle background can easily be identified at different depths of the field.

Intriguingly, based on the proposed incoherent self-imaging model, the broadband illuminated speckles can be represented as an incoherent sum of speckles with self-imaging units at each single wavelength, which cannot be obtained using conventional methods [25, 29, 32,33,34,35]. According to previous studies on spectral-correlation of speckles [29, 41,42,43,44], the correlations and the contrast of broadband speckles are significantly reduced, leading to a severe dispersive blur in speckles. Based on conventional view of the diffuser scattering [25, 29], the diffuser was treated as a random complex Gaussian distribution phase-screen [38, 45]. Using this method, a fully-developed speckle is obtained as shown in Fig. 2(e) (see details in Note 6 in the supplementary material). One can see that the self-imaging is unpredicted for this single wavelength case, let alone the situation of broadband speckle with drastically increased blur. In contrast, inspired by our proposed new view of the diffuser, the self-imaging phenomenon in speckle can be predicted using Eq. 1, as shown in Fig. 2(f). Although this prediction was widely obtained in conventional imaging systems (e.g., using the system in Fig. 2(d) with the aid of pre-calibrated PSF) [14, 26, 27, 33, 35, 36], the self-imaging in speckle has never been predicted since the PSF itself was a chaotic pattern with no describable features (e.g. [14, 26, 27, 33, 35, 36]). Due to multiple-scattering and chaos in PSF’s speckle, increased blur in the final speckle can be predicated as a sum of speckle from every point of the object. But the self-imaging in broadband white-light speckle was considered infeasible (i.e., post processing is required as suggested by [26, 29, 33]). In contrast, here we treat the hot-spot in the speckle as the image of the point. As a result, calibration-free self-imaging in broadband speckle can be predicted using our proposed model, similar to the lens-imaging system for broadband illuminations. The resolution of the diffuser speckle-imaging system then is given as δx = λu/D = λu/(λv/Δx) = Δx/M, where M = v/u represents the magnification of the imaging system, and D represents the effective diameter of any limiting aperture stop placed between the scattering medium and the camera. In a nutshell, the resolution of the proposed system is limited by the resolution of the detector and the magnification of the system (see details in Note 5 in the SI). In addition, these hot-spots are strong in intensity, which can easily be resolved using regular cameras with no need of expensive scientific ones, as will be verified below.

Directly seeing through diffusers

In order to verify the theoretical and numerical predictions, here we collected speckles using a white light LED source (MCWHL7, Thorlabs) and an inexpensive color camera (CS165CU, Thorlabs, see experimental details in SI, Note 1). The raw image of a sample “S” is shown in the left column of Fig. 3(a) with no obvious information at the first glance using the naked eye. This is one of the major reasons that this phenomenon was missed in previously reported works. However, by zooming into a randomly selected region-of-interest (ROI), the self-imaging of the object can be observed! ROIs 1 and 2 are depicted in the central and right columns in Fig. 3(a), respectively. One can see many blurred self-imaging units in the speckle (see yellow boxes): i.e., the chaotic pattern of speckle is composed of random superpositions of these blurred self-imaging units and multi-scattering as the background. Intriguingly, this broadband self-imaging phenomena can also be observed directly for different objects and varied grits of diffusers (see more results in Note 2 in the SI). This experimental observation shows intuitive link between the object space and speckle pattern’s spatial information which was largely overlooked in precious studies. However, these self-imaging units are fairly blur (see details in Note 3 in SI), which is undesired for practical imaging applications. The next important question is how to enhance the contrast of these self-imaging units.

Fig. 3
figure 3

Self-imaging in white-light/single-wavelength speckles. (a) Raw speckle captured using broadband white light LED source. (b-c) Experimental results of imaging of different objects (b) digit ‘2’, and (c) digit ‘3’, through thin diffuser with narrow-band filter for high-contrast speckles. The ground truth object is shown in the bottom-left corner. Clearer view and abundant self-imaging units in speckle are directly observed by the naked eye, with no filtering pre-processing or post-processing utilizing computational reconstruction

Full size image

According to incoherent imaging theory and speckle-spectral-correlation properties [25, 29, 32,33,34], high-contrast speckles can be obtained using narrow-band light source. This strategy is also applicable to our direct self-imaging system (see Note 2, and 3 in the SI). Remarkably, spectral imaging using speckles recorded at different filtered wavelengths (channels) can be efficiently obtained using self-imaging with no need of encoding-introduction and post-reconstruction (see Note 3 in the SI). Here we introduced a narrow-band LED through a narrowband filter (Andover, 633FS02-50, with the central wavelength of λ = 633 nm and the bandwidth of ~ 1 nm) in front of the camera. Two objects of digit ‘2’ and ‘3’ with the dimension of ~ 0.35 mm on the resolution test target were characterized with u = 60 cm and v = 10 cm, respectively. As shown in Fig. 3(b-c), self-imaging units with improved contrast were directly observed in the raw speckles. As shown in those zoom-in ROIs (last column in Fig. 3(b-c)), the composition of the speckle manifests itself as a random accumulation of self-imaging units. Compared with the results in Fig. 3(a), one can see more self-imaging units with clearer details due to the improved contrast of speckles. Intriguingly, this direct observation strategy with no need of post computational data process will enable real-time imaging of the targets through diffusers.

To demonstrate real-time imaging, we introduced a group of individual letters one by one (small objects with the dimension of ~ 0.2 mm, see the upper panel in Fig. 4(a)) and shoot a video by an ordinary color camera within the OME region (~ 0.84 mm, see video 1 in SI, and details for experimental setup). One self-imaging unit in the speckles is highlighted by the yellow circle in the central panel of Fig. 4(a) showing clear profile of the object. Intriguingly, for a given diffuser, the positions of self-imaging units for different objects are unchanged (see the lower panel in Fig. 4(a) and videos 1, and 2), since isoplanatic linear space-invariant PSF is maintained within the OME region (see Eq. 1). To further demonstrate real-time self-imaging of larger targets in the OME range, a letter ‘A’ with the dimension of ~ 0.6 mm was tested. To erase the salt-point-noise, denoised speckles are shown in Fig. 4(b) (using 2 × 2 pixels median-filtering of the raw-speckle). One can see self-imaging units of isolated letter ‘A’ with obviously high contrast. Furthermore, a target with three letters ‘EDU’ (with the total lateral dimension of ~ 0.75 mm) were also tested. For this relatively complicated target, incoherent superposition of isolated objects’ speckle is expected. As shown in the yellow box in Fig. 4(c), the complete self-imaging of multi-targets is observable. Importantly, since prior information of the diffuser-medium and the objects is unnecessary, one can obtain the self-imaging patterns of unknown objects in the scrambled speckle with no need of post computational reconstruction nor active wave control or precise modulation devices. Moreover, the object’s orientation information is directly seen and instantly tracked (see video 3 in SI for dynamic tracking results), which was unknown when reconstructed using phase retrieval (PR) (e.g. [19, 25, 34]). Further efficient processing of speckles to extract the image of the target can be realized using Gaussian filtering and/or homomorphic filtering, which is a robust way for image extraction (see details in Note 7 in SI). The quality of the directly observed images is even comparable to a majority of reconstruction results processed by SAI (e.g. [25, 30]), but was not better than the reconstruction with pre-calibrations or multi-frames-related techniques [28, 32,33,34, 46]. Therefore, post imaging processing is still useful to further enhance the quality of these self-imaging strategy, as will be explored below.

Fig. 4
figure 4

Real-time video imaging through a diffuser. (a) Real-time imaging of moving objects through random diffusers. (b) Imaging of isolated object of letter ‘A’, and (c) multitargets of letters ‘EDU’. Clear self-imaging images of multitargets are observed directly and instantly. The insets in the lower right inset shows the object, in which the red-dashed circles depict the OME region. The upper right inset shows the observed self-imaging unit. Object-distance and image-distance are u = v = 10 cm

Full size image

To improve the quality of reconstructed images, refined structures of speckle’s autocorrelation are desired for PR processing [25, 30,31,32,33,34]. By refining the detailed structure of speckle’s autocorrelation (i.e., the detailed object’s autocorrelation, see Eq.5), refined detail information of the object in the frequency domain is therefore predicted according to the Wiener-Khinchin theorem [25, 38]. As a result, improved profiles of reconstructed images are expected using PR iteration. Here we propose a modified SAI method to reveal this potential (see details in Note 8 in the SI). In the speckle, clear units can be treated as in-focus images of the object, while other dimmed units are treated as the out-of-focus images of the object. By introducing intensity-extension with typical homomorphic filtering (HF) of the speckle, the information encoded in the high-intensity in-focus units and dimmed out-of-focus units are both considered, contributing to the refined structure of speckle’s autocorrelation. To verify this improved methodology, we tested the experimental speckles of ‘3’ obtained in Fig. 3(c) and compared the results of traditional SAI and the modified SAI method. First, the raw speckle of digit ‘3’ and its autocorrelation result are shown in Fig. 5(a) and (b), respectively. For the reconstruction process, a typical hybrid PR algorithm is used: i.e., a hybrid input-output (HIO) algorithm with a decreasing factor from 2 to 0 with a step of 0.02 and a loop of 50 iterations in each trial followed with classic error-reduction (ER) algorithm for another 50 iterations [25, 39, 40]. As a result, the image of the object is retrieved from a single-shot speckle using conventional SAI as shown in Fig. 5(c). In comparison, the imaging results processed using our modified SAI with HF are shown in Fig. 5(d-f). Specifically, Fig. 5(d) presents the speckle of Fig. 5(a) after HF. As shown in the zoomed-in ROI, besides those original high-intensity self-imaging units, more self-imaging units with lower-intensity in Fig. 5(a) are emerging in the speckle with HF. Remarkably, the autocorrelation of this new speckles (Fig. 5(e)) shows obviously more information than Fig. 5(b). Therefore, the reconstructed image in Fig. 5(f) shows sharper profiles of the object “3”. To quantitatively reveal the improved imaging quality of our modified SAI method, in Fig. 5(g, h), we directly compared the cross-line plots at the central vertical lines of the images in Fig. 5(b & e) and (c & f), respectively. One can see more details in speckle’s autocorrelation and sharper reconstruction profile, demonstrating the better performance of our modified SAI. Similarly, the comparison of the digit “2” obtained in Fig. 5(c) is shown in Note 8, Fig. S12 in SI.

Fig. 5
figure 5

Modified SAI reconstruction inspired by self-imaging. Experimental results of imaging through a thin diffuser using (a-c) classic SAI (Ref. [25]) and (d-f) improved SAI reconstruction methods. (a) Speckle for reconstruction. (b) Autocorrelations of speckle in (a). (c) The recovered image using the direct SAI method. (d) Image of speckle in (a) after homomorphic filtering processing. (e) Autocorrelations of speckle in (d). (f) The reconstructed results using (d) and (e). Scale bars in (c, f): 10 pixels. (g, h) The cross-line plots of the intensity of direct SAI (first row) and modified SAI (second row) at the central vertical line position (shown as the colored-dashed vertical line in (b-c) and (e-f)). AC: autocorrelation. HF: homomorphic filtering

Full size image

Discussion

These results provide further understanding of the autocorrelation properties of speckle, and the features of diffuser-imaging system, representing a new efficient way for imaging recovery using speckles. A more detailed comparison in pros and cons with previously reported methods is listed in Table S1 in the supplementary information. In a nutshell, this proposed method shows fascinating capabilities with new potentials for imaging through scattering diffuser due to the narrowband/broadband, single-shot, real-time, calibration-free, computation-free, low-cost, and high-efficiency features using simple apparatus. In conclusion, this work reported a direct observation strategy through scattering diffusers using self-imaging effect that was largely overlooked by previous works. By reexamining inherent nature of speckles, a self-imaging speckle model was developed and validated using experiment, enabling visual understanding of the speckles and the statistic properties of speckle. Remarkably, a reconstruction-free strategy for real-time imaging of static and moving objects with their actual orientation information was demonstrated. This new technique requires no complicated or expensive devices for detection nor post computational reconstruction. The visibility of directly observed imaging patterns is equivalent to those processed with direct SAI (e.g. [25]). Furthermore, using a simply modified SAI method with efficient joint-filtering, the achieved imaging quality and resolution are comparable to the best results processed by previously reported computational reconstruction methods (e.g. [30,31,32,33,34,35,36,37, 47]). These results processed by the modified SAI show that the reconstructed images beyond the diffraction-limit can be achieved, which is obviously better than conventional SAI methods. Intriguingly, this directly observed self-imaging phenomenon indicates new potentials for emerging deep-learning-assisted imaging, design of physics-based or physics-aware intelligent technologies for computational imaging [48, 49], design of customized speckle [50], and will create new opportunities for bio-imaging (e.g., [2, 24, 25]), super-resolution imaging through thin scattering layers, and other novel imaging methodologies [51].

Methods

Experimental setup

The object in the USAF test target (1951USAF, Edmund) is used. Objects of letters and symbols are fabricated using laser etching of metallic masks. The target is illuminated from the back at varied distance u, by a collimated LED light source (Thorlabs, see details in the SI, for different sources used). The ground glass diffuser of a 220-grit (DG10-220-MD, Thorlabs) and a 600-grit (DG10-600-MD, Thorlabs) are used in the experiment. The speckle is recorded by a scientific camera (Zyla 5.5, Andor). As a comparison, a color camera (1440 × 1080 pixels, Pixel Size 3.45 μm, CS165CU CMOS Cameras, ThorLabs) is used to record the results of color speckles. Self-imaging observations in speckles are directly observed, for both narrow band and broadband illuminations. For further details of experimental setups and additional supporting results, see the Supplementary information.