Interest in systems of obtaining and forming optical ghost images has grown significantly (see, e.g., [1–19]) because they have a number of advantages over conventional images, particularly for biological objects, which require a gentle irradiation regime. For these applications, the endoscopy of viscera and cavities of a human being is promising, e.g., to study blood vessels. Conventional endoscopy requires the introduction of a probe equipped with a video camera to obtain images of the inner surface of an organ, e.g., stomach, bronchi, or internal walls of vessels. The ghost imaging principle eliminates the necessity of the introduction of the video camera in an object under study; for this reason, ghost fiber optic endoscopy seems very attractive because it allows one to extremely reduce the thickness of the probe.

The ghost imaging method is based on single-pixel imaging [1–18]. Single-pixel imaging requires only one detector to reconstruct a full image [1, 2]. For single-pixel imaging, an object under investigation is successively irradiated by light with known specially prepared spatial structures [20]. The total intensity of light scattered by the object corresponding to each of these structures is collected by a single multimode fiber or by a bundle of fibers and is detected by a detector without spatial resolution. The image of the object can be decoded with various image reconstruction algorithms [3–5, 21–23] such as the traditional correlation method, compressed sensing method [24], and measurement reduction method [25]. Advantages of the single-pixel imaging technique are a significant reduction of the cost of a sensing camera and the extension of the studied spectral range [3–5].

The single-pixel imaging technique has been applied in recent years in biomedical visualization [6, 7], infrared spectroscopy [8], formation of holographic images [5, 9, 10], microscopy [11, 12], and hyperspectral studies [14, 15]. This technique made it possible to create ghost polarimetry [26–29], to provide the encoding of information [18, 19], and to obtain three-dimensional (3D) images [16, 17].

In this work, we propose an original method of fiber optic 3D endoscopy with femtosecond light pulses to probe a partially transparent scattering medium.

The proposed technique implies obtaining images by the ghost imaging principle using the correlation method. According to this principle, an object under study is illuminated by light with a random spatial intensity profile through an optical fiber or bundle of such fibers, and radiation scattered by the object is detected separately.

A possible layout of an endoscope implementing the proposed 3D endoscopy technique is presented in Fig. 1. The object under study is illuminated by light transmitted through the fiber bundle (shown in red in Fig. 1). Several multimode fibers 62.5 μm in diameter, which are used to collect radiation scattered from the object, are present inside the bundle of randomly distributed single-mode fibers 9 μm in diameter with an average distance of 72 μm between their centers. The diameter of the bundle is approximately 1 mm. If necessary, the number of fibers can be multiplied.

Fig. 1.
figure 1

(Color online) Layout of the femtosecond ghost 3D endoscope: (1) illuminated object under study, (2) one of the ends of the bundle with the lens and collecting fibers, (3) planar part of the bundle irradiated by the spatially modulated femtosecond pulse, (4) nonlinear crystal for the sum frequency generation irradiated by the same laser through the delay line, (5) collecting detector for scattered radiation, and (6) CCD camera with the objective to detect the radiation intensity profile at the output of the fiber bundle.

Full size image

Radiation illuminating the object is introduced transversely in the middle of the fiber bundle. The bundle is simultaneously an illuminating and detecting device. The material of fibers in the middle of the bundle is modified in order to make them scattering. This part of the bundle is illuminated by femtosecond laser pulses formed by a cylindrical lens in the linear light beam with a spatially modulated intensity profile. For each fiber to receive its individual portion of light, the illuminated part of the bundle is expanded in a planar structure and is then again assembled in the bundle. The independent illumination of individual fibers is thus implemented. The intensity is modulated by a spatial light modulator, which forms spatially random pseudothermal light [20] with the correlation radius that is approximately equal to or better less than the diameter of the fiber.

Pulses scattered in fibers propagate in each of the single-mode fibers in opposite directions. The spatially modulated beam at both edges of the bundle forms light with a random spatial modulation; the spatial structures of light at both outputs are correlated, which allows one to obtain ghost images (see, e.g., [21, 23, 30, 31]).

The independent scattering of light in individual fibers forms sources of radiation with random amplitudes and phases at the output of the bundle, thus creating a field with Gaussian statistics. To ensure the spatial randomness of the field and the statistical independence of radiation in nonoverlapping spatial regions, radiation from a single fiber should not illuminate a region illuminated by neighboring fibers in the bundle. Since radiation leaving single-mode fibers diverges rapidly, this requirement imposes a strong upper bound on the probing depth. To solve this problem, it is possible to use a symmetric aspherical biconvex lens to form an image of the end of the bundle on the object under investigation. Aspherical surfaces allow the complete exclusion of spherical aberration. At a linear magnification of –1, odd field aberrations—coma, astigmatism, and distortion—are automatically excluded owing to the total symmetry of the system (see, e.g., [32] and references therein). Only the curvature of the field remains, which does not damage but instead improves the situation when studying round objects from the interior because sections in penetrating depth will be slightly curved rather than flat. This systematic error is easily corrected in the subsequent computer processing and reconstruction of the 3D structure. Thus, such a single-lens projection system will operate almost ideally with an accuracy of fabrication.

Light scattered by the object enters multimode fibers of the bundle and propagates in them to the collecting detector. The bundle at the measuring end is split into two parts: one part consists of single-mode fibers packed in the same order and the second part includes multimode fibers. The arrangements of fibers in the object and measuring ends of the fiber bundle are schematically shown in Fig. 2. The image of the measuring end of the bundle is projected by a micro-objective on the matrix of a CCD camera, whereas radiation scattered from the object and transmitted through multimode collecting fibers is focused by another micro-objective on the nonlinear crystal, where sum frequency generation, in our case, second harmonic generation, occurs. The transformed light beam is shown in blue in Fig. 1. Conversion to the sum frequency is necessary to separate the fraction of the total scattered light pulse that comes from a given probing depth of the object. Thus, the image becomes three-dimensional. To this end, a time-delayed fraction of the same femtosecond pulse illuminating the fiber bundle is used as pump. As a result, only a signal scattered from the object that coincides in time with the delayed pulse is transformed. This ensures the depth resolution of the endoscope, which is determined by the duration of the femtosecond pulse. The depth resolution at a duration of 30 fs will be about 10 μm. Radiation at the sum frequency is recorded integrally by a single detector.

Fig. 2.
figure 2

(Color online) Geometries of the parts of the (а) output object and (b, c) measuring ends of the fiber bundle. Thin fibers shown in white transfer randomly modulated radiation and thick fiber shown in green collect scattered radiation.

Full size image

A plane conjugate to the surface of the object is projected on the nonlinear crystal upon measurements. This guarantees the coincidence of the spatial structures of rays on the surface of the object and in the measurement channels.

A ghost image is physically constructed from N measurements, which are carried out as follows. An amplitude–phase modulation mask, which transforms a femtosecond pulse, is formed on the spatial light modulator. Radiation is scattered on the object and is simultaneously detected by a CCD matrix. Scattered radiation is collected by multimode fibers, is transformed to the sum frequency in the required time window, and is detected. As a result, the intensity profile of the randomly modulated field at the output of the fiber bundle and the detected intensity of sum frequency radiation at a given focusing depth are obtained after each measurement. After N measurements, the ghost image is constructed by the correlation method. A set of ghost images obtained at different focusing depths using the algorithm described below forms a three-dimensional scattering map of the object.

It is convenient to mathematically describe the formation of a ghost 3D image in the discrete form by virtually separating the object into several successive finite-thickness layers. We first describe the formation of the ghost image of a two-dimensional object using the discrete approach. In this approach, the surface of the object is covered with a virtual grid, where each cell (virtual pixel) is specified by two indices i and j. In the two-dimensional case, the problem is reduced to the calculation of the covariance function of the intensities

$${{G}_{{ij}}} = \left\langle {{{I}_{{ij}}}{{I}_{B}}} \right\rangle - \left\langle {{{I}_{{ij}}}} \right\rangle \left\langle {{{I}_{B}}} \right\rangle .$$
(1)

Here, \({{I}_{{ij}}}\) is the intensity of light in the virtual pixel with the indices i and j; \({{I}_{B}} = \sum\nolimits_{k,l} {{R}_{{kl}}}{{I}_{{kl}}}\) is the total intensity of light after the interaction with the object, and \({{R}_{{kl}}}\) is the scattering coefficient of the object in the pixel with the indices k and l. Then,

$$\begin{gathered} {{G}_{{ij}}} = {{R}_{{ij}}}\left\langle {I_{{ij}}^{2}} \right\rangle + \sum\limits_{k,l} \left\langle {{{R}_{{kl}}}{{I}_{{kl}}}{{I}_{{ij}}}} \right\rangle \\ - \;{{R}_{{ij}}}{{\left\langle {{{I}_{{ij}}}} \right\rangle }^{2}} - \left\langle {{{I}_{{ij}}}} \right\rangle \sum\limits_{k,l} \left\langle {{{R}_{{kl}}}{{I}_{{kl}}}} \right\rangle . \\ \end{gathered} $$
(2)

If the used source is δ-correlated, the intensities measured in different pixels are statistically independent; i.e., \(\left\langle {{{I}_{{ij}}}{{I}_{{kl}}}} \right\rangle = \left\langle {{{I}_{{ij}}}} \right\rangle \left\langle {{{I}_{{kl}}}} \right\rangle \) if \(i,j \ne k,l\). Then,

$${{G}_{{ij}}} = {{R}_{{ij}}}\left\langle {I_{{ij}}^{2}} \right\rangle - {{R}_{{ij}}}{{\left\langle {{{I}_{{ij}}}} \right\rangle }^{2}} = {{R}_{{ij}}}{{D}_{{ij}}}.$$
(3)

Hence, the scattering coefficient of the object in the \(\left\{ {i,j} \right\}\) pixel is

$${{R}_{{ij}}} = \frac{{{{G}_{{ij}}}}}{{{{D}_{{ij}}}}}$$
(4)

and, correspondingly, the profile of the scattering object is determined.

Further, we consider a two-layer object with the layer scattering coefficients \(R_{{ij}}^{{(1)}},R_{{ij}}^{{(2)}}\). For simplicity, we consider that the scattering coefficients are independent of the radiation propagation direction, the layers are sufficiently thin, and, therefore, radiation transmitted through a layer entirely reaches the next layer. Let the intensity of radiation incident on the first layer be \({{I}_{{ij}}}\). Then, \(R_{{ij}}^{{(1)}}\) is obviously calculated by Eq. (4). To calculate \(R_{{ij}}^{{(2)}}\), it is necessary to take into account that radiation transmitted through the first layer reaches the second layer, whereas the detector detects radiation transmitted through the first layer in the opposite direction. The transmission coefficient of the first layer is 1 – \(R_{{ij}}^{{(1)}}\). Thus, the intensity of light incident on the detector is \({{I}_{B}} = \sum\nolimits_{k,l} {{\left( {1 - R_{{kl}}^{{(1)}}} \right)}^{2}}R_{{kl}}^{{(2)}}{{I}_{{kl}}}\) and, according to Eq. (4),

$${{\left( {1 - R_{{ij}}^{{(1)}}} \right)}^{2}}R_{{ij}}^{{(2)}} = \frac{{{{G}_{{ij}}}}}{{{{D}_{{ij}}}}}.$$
(5)

The coefficient \(R_{{ij}}^{{(2)}}\) can be reconstructed in terms of the distribution of \(R_{{ij}}^{{(1)}}\):

$$R_{{ij}}^{{(2)}} = \frac{{{{G}_{{ij}}}}}{{{{D}_{{ij}}}{{{\left( {1 - R_{{ij}}^{{(1)}}} \right)}}^{2}}}}.$$
(6)

By induction,

$$R_{{ij}}^{{(n)}} = \frac{{{{G}_{{ij}}}}}{{{{D}_{{ij}}}\prod\limits_{m = 1}^{n - 1} {{{\left( {1 - R_{{ij}}^{{(m)}}} \right)}}^{2}}}}.$$
(7)

Formula (4) obviously provides a recursive algorithm to reconstruct the three-dimensional image of the object by its layer-by-layer scanning and using information obtained in the preceding layer to calculate the next layer. The calculation of the ghost image from experimental data provides the coefficients

$$R_{{ij}}^{{'(n)}} = R_{{ij}}^{{(n)}}\prod\limits_{m = 1}^{n - 1} {{\left( {1 - R_{{ij}}^{{(m)}}} \right)}^{2}},$$
(8)

which are used to calculate the actual scattering coefficient by Eq. (7).

To test the described approach, we performed a numerical experiment using our ghost_images library [33]. A test object was an octahedron separated into nine layers (see Fig. 3). The layers were parallelepipeds each with a thickness of 1/9 of the height of the octahedron with the thickness-integrated scattering coefficients R = 0.01, 0.02, 0.03, 0.04, 0.05, 0.04, 0.03, 0.02, 0.01 and sides of 128, 256, 384, 512, 640, 512, 384, 256, and 128 μm in the 1024 × 1024 μm computational region including 64 × 64 points. The intensity profiles were simulated using a pseudorandom number generator with a uniform distribution.

Fig. 3.
figure 3

(Color online) (Top left) Test object separated into finite-thickness layers and its ghost images reconstructed by Eq. (7) from (from left to right and from top to bottom) 1024, 2048, 4096, 8192, 16 384, 32 768, 49 152, and 65 536 measurements. Pixel numbers of the computational grid are indicated on the x and y axes.

Full size image

Figure 3 shows the initial 3D scattering profile of the object and 3D scattering profiles calculated by Eq. (7) from 1024, 2048, 4096, 8192, 16 384, 32 768, 49 152, and 65 536 measurements. It is seen that the ghost image of the object approaches its structure with the growth of the sample and distortions caused by scattering on preceding layers are completely compensated by Eq. (7).

To summarize, it has been shown that the three-dimensional scattering profile of a three-dimensional object can be reconstructed using the described algorithm and illuminating the object by femtosecond radiation with of a random spatial profile.

CONCLUSIONS

An important practical feature of ghost fiber optic 3D endoscopy is that all its elements can be placed in a single unit spaced from a patient. The only element that is introduced in the body of the patient is an optical probe with a thickness less than 1 mm, which makes the method minimally invasive and nontraumatic. Since the organs of children are small, the application of such an endoscope is particularly relevant in pediatrics not only in diagnostic but also in online monitoring of surgical procedures.

No obstacles for the implementation of the proposed endoscope in the photon counting mode are seen because sum frequency generation can occur at the single-photon level. To this end, femtosecond radiation should be weakened to a few-photon level, and matrix photon counters, e.g., a recently fabricated Hamamatsu ORCA-Quest qCMOS camera C15550-20UP, should be used instead of light detectors. This regime of operation will ensure the maximally gentle action on diagnosed tissues.