1 Introduction

Single-pixel imaging (SPI), as a representative example of compressive sampling (CS), enables image acquisition with spatially unresolved detectors (Candès et al., 2006; Donoho, 2006; Candès and Wakin, 2008; Duarte et al., 2008). The acquired measurements for image reconstruction can be far fewer than the mage pixels. In contrast with laser-scanning imaging (e.g., time-of-flight (TOF) imaging), SPI can shorten image acquisition time significantly with only a slight degradation of image quality. Consequently, SPI has been widely applied to magnetic resonance imaging (MRI) and aerospace remote sensing in recent years (Lustig et al., 2007). In addition, SPI enables image acquisition in the invisible spectral region, such as infrared or ultraviolet, where traditional imaging methods require expensive image sensors. SPI is, therefore, an attractive prospect, as it allows a considerable cost reduction in present camera fabrication, and increases the range of spectral imaging. The first prototype of a single-pixel camera was implemented by Takhar et al. (2006), and a digital micromirror device (DMD) is used to generate structured light patterns for illumination. However, the response time of DMDs (∼50 µs) and the number of acquired measurements limit the temporal resolution of single-pixel cameras significantly. To increase their imaging speed dramatically, a mechanical-scan-free method is a necessity for the generation of structured light patterns. In recent years, an impressive ultrafast optical imaging technique called ‘serial time-encoded amplified microscopy (STEAM)’ has been proposed, overcoming the compromise between sensitivity and frame rate and achieving real-time observation of fast dynamic phenomena (Goda et al., 2009; 2012; Goda and Jalali, 2013; Chen et al., 2015; Lau et al., 2016; Lei et al., 2016). The STEAM technology uses a pulsed laser as the light source and establishes a bridge between the space and time domains using a photonic time stretch (PTS) technique together with spatially dispersive elements. It offers us a new strategy that generates structured light patterns through temporal modulation (namely, spectral shaping of optical pulses) instead of spatial modulation (Chan et al., 2015). By using pulsed lasers with high repetition rates (tens of MHz) and electro-optical modulators with large bandwidths (tens of GHz), the resultant switching rate of illumination patterns is approximately three orders of magnitude higher than that achieved in traditional single-pixel cameras. This strategy for structured-illumination pattern generation will accelerate SPI and give it the ability to observe fast dynamic phenomena. Such a high-speed single-pixel imaging device, called the ‘time-stretch-based single-pixel imaging (TSSPI) system’, is applicable to fast continuous imaging (e.g., flow cytometry) because it enables a notable reduction of data produced by high-throughput instruments (from tens of gigabytes per second to tens of megabytes per second), greatly relieving the pressure on data acquisition, transmission, and storage. Currently, an imaging flow cytometer based on this technique has been demonstrated, achieving a high throughput, a high spatial resolution, and efficient data compression simultaneously (Bosworth and Foster, 2014; Guo et al., 2014; 2015; Bosworth et al., 2015).

2 Single-pixel imaging

2.1 Principles of single-pixel imaging

SPI relies on two fundamental premises: sparsity and incoherence (Donoho, 2006). Natural images have concise representations, most of whose coefficients are small, and the relatively few large coefficients capture most of the information. This property is called the ‘sparsity of images’. Let x be a 2D image of size \(\sqrt N \cdot \sqrt N\), and ψ ∈ ℝN×N is an orthonormal basis, e.g., discrete cosine transform (DCT) or wavelet basis functions, where x can be represented sparsely as

$$x = \psi \theta ,$$
((1))

where θ is a sparse vector with at most K nonzero entries (K≪N). Then the sampling process is modeled by interacting x with a specific matrix A ∈ ℝN×N (M<N) as

$$y = Ax + e,$$
((2))

where y is a measurement vector and e is a noise term bounded by a known amount ∥e∥2≤σ. Matrix A must be incoherent with orthonormal basis ψ, so that sparse vector θ∲ can be reconstructed faithfully with only M=O(K·log(N/K)) measurements by seeking approximate solutions to the problem:

$$\min\limits_{\theta \prime} \left( {{1 \over 2}\left\Vert {y - A\Psi \theta \prime} \right\Vert _2^2 + \tau {{\left\Vert {\theta \prime} \right\Vert}_1}} \right){.}$$
((3))

Finally an estimated image vector x′ can be obtained by

$$x\prime = \psi \theta \prime{.}$$
((4))

An important parameter, compression ratio R, which characterizes the amount of compression, is expressed as

$$R = M/N{.}$$
((5))

2.2 Architecture of the single-pixel camera

A single-pixel camera is composed of a DMD, two lenses, a single-photon detector, and an analog-to-digital converter (ADC). With the help of a lens, the desired image is projected on DMD which consists of an array of electrostatically actuated micro-mirrors with two stable states (+12° and −12° from the horizontal). Thus, light falling on DMD may be reflected in two directions depending on the orientations of the mirrors. Then light reflected from one direction (namely, the image encoded by a random pattern) is collected by a second lens, and is focused onto the photodiode. The output of the photodetector is amplified by an electronic amplifier and then digitized by ADC. Repeating this process M times, M measurements are captured. A sparse approximation to the desired image can be reconstructed from the M measurements and the corresponding random patterns.

2.3 Limitations of single-pixel imaging

The major disadvantage of SPI is its low temporal resolution, which is limited by the response time of DMD and the number of captured measurements. Let rmax be the maximum refresh rate of DMD and M the number of measurements acquired for image reconstruction. The maximum frame rate of singlepixel camera fmax satisfies

$${r_{\max}} = {f_{\max}} \cdot M,$$
((6))

where rmax is about 22 kHz. For example, a 256×256 image of a black-and-white R is reconstructed from 1300 measurements, resulting in a frame rate of 17 frames/s, which prevents SPI from observing fast dynamic phenomena. Therefore, it is difficult to apply SPI technology to microscopy, because high-temporal resolution is particularly important for microscopy (even a slow event appears to occur as a ‘fast’ one in a small field of view).

3 Serial time-encoded amplified microscopy

3.1 Principles of serial time-encoded amplified microscopy technology

STEAM is an ultrafast optical imaging technology applied to microscopy. In contrast with conventional image sensors, such as charge-coupled devices (CCDs) and complementary metal-oxide-semiconductors (CMOSs), the STEAM technology overcomes the fundamental compromise between sensitivity and frame rate, achieving frame rates that are at least 1000 times larger than those of conventional CCDs. The key principle of the STEAM technology is the conversion of the spatial information of objects to temporal waveforms. A pulsed laser with a high repetition rate is used as the light source, and a diffraction grating separates spatially the spectral components of optical pulses. When the resultant dispersed pulse illuminates the object, the spatial information of the object is encoded onto the pulse spectrum. Then, a temporally dispersive medium (e.g., dispersion compensation fiber) is used to convert the pulse spectrum to a temporal waveform, which can be detected by high-speed photodiodes instead of relatively slow devices like CCD or CMOS cameras. The STEAM technology offers a real-time image acquisition at millions of frames per second and sub-nanosecond shutter speed; hence, it is applied suitably to imaging of particles flowing at high speeds, because it is capable of eliminating motion blur without resolution loss during the high-speed flow. A high-throughput single-microparticle imaging flow analyzer based on STEAM has been demonstrated, enabling high-throughput image-based screening of specific cells in blood with an unprecedented throughput of 100 000 particles per second and a record false positive rate of one in a million (Goda et al., 2009; 2012; Goda and Jalali, 2013).

3.2 Limitations of serial time-encoded amplified microscopy technology

STEAM is an ultrafast continuous imaging technology. Hence, a massive amount of digital data will be produced by this high-throughput instrument. For example, the STEAM camera acquires a data stream of 0.8 Tb/s. Detecting rare events such as cancer cells requires that data should be recorded continuously for a long time. The acquired data stream overwhelms data acquisition, storage, and processing operations, making the STEAM technology impractical for applications. Therefore, image compression technology should be adopted to solve this problem. Non-uniform time-stretch transformation that takes advantage of sparsity in physical signals has been proposed (Chen et al., 2015), achieving both bandwidth compression and reduction in the temporal length. However, ADCs with high sampling rates and large bandwidths (e.g., a sampling rate of 20 GS/s and a bandwidth of 7 GHz) are still required, putting great pressure on bank-end digitizers and digital processors. In addition, the amount of compression is about only three times, which cannot solve the big data problem fundamentally. Therefore, a more efficient method should be explored.

4 Time-stretch-based single-pixel imaging

The STEAM technology provides a space-to-time mapping strategy, which can be used to accelerate the structured illumination pattern generation. The illumination pattern is generated flexibly at a high speed by spectral shaping together with wavelength-to-space mapping. Such a mechanical-scan-free method has increased the pattern generation rate by three orders of magnitude, and can be used to increase the imaging speed of single-pixel imaging or structured illumination microscopy. By integrating the STEAM technology, single-pixel imaging technology, and signal processing technology, a novel technique called ‘time-stretch-based single-pixel imaging (TSSPI)’ was proposed by Guo et al. (2015), achieving high frame rates and enabling observation of fast dynamic phenomena. This can be employed widely in both industrial and biological applications. Herein, we give an introduction to the principle, architecture, and applications of TSSPI.

4.1 Principle of time-stretch-based single-pixel imaging

A TSSPI system is an infrared imaging device. The key feature of TSSPI is high-speed illumination pattern generation, whose basic principle is shown in Fig. 1. This is achieved by first dispersing a broadband optical pulse using a dispersive element (e.g., dispersion compensating fibers) temporally. Then the stretched pulse is shaped spectrally through time-encoding with pre-designed patterns. A spatially dispersive medium can be used to perform wavelength-to-space mapping to generate illumination patterns. Here, diffraction grating is used to generate 1D illumination patterns, and a pair of orthogonally oriented spatial dispersers is used to generate 2D ones. In this way, the pattern generation rate can reach tens of MHz, which is three orders of magnitude larger than that achieved with DMDs.

Fig. 1
figure 1

The basic principle of high-speed illumination pattern generation

Full size image

For SPI, the pre-designed patterns encoding the pulse spectrum are random, and adjacent patterns should be orthogonal to each other according to the theory of CS. Herein, the Hadamard matrices are used to provide structured light patterns. Then the generated structured light patterns illuminate the object and the reflected light (namely, the inner product between the pattern and the object) is detected by a single-pixel photodiode. To remove the sources of noise such as ambient light fluctuations, differential signals can be obtained by displaying each Hadamard pattern, followed by its inverse counterpart, and taking the difference between the measured intensities. The desired image is obtained from the acquired measurements (namely, the difference signals) via a reconstruction algorithm, such as the iterative shrinkage/thresholding algorithm (TWIST) or gradient projection for sparse reconstruction (GPSR) (Bioucas-Dias and Figueiredo, 2007; Figueiredo et al., 2007; Tropp and Gilbert, 2007; Blumensath and Davies, 2009; Needell and Tropp, 2009).

4.2 Architecture of the time-stretch-based single-pixel imaging system

The detailed architecture of the TSSPI system is presented in Fig. 2. The light source is a mode-locked laser (MLL) emitting a pulse train with a repetition rate of 50 MHz, a center wavelength of 1550 nm, and a pulse width of 150 fs. The optical pulses pass through a section of dispersion compensating fiber (DCF) with a group velocity dispersion (GVD) of −1368 ps/nm, and are stretched temporally to around 20 ns. Then, a high-power erbium-doped fiber amplifier (EDFA) is used to compensate for the optical loss induced by DCF. Then an arbitrary waveform generator (AWG) synchronized to MLL is used to generate a 6.25 Gb/s pseudo-random binary sequence (PRBS) signal for shaping spectrally the stretched pulses through a 12.5 Gb/s Mach-Zehnder modulator (MZM). An optical circulator directs the spectrally encoded pulses to a diffraction grating with a groove density of 1200 lines per millimeter. Because of the angular dispersion of the diffraction grating, 1D illumination patterns are generated via wavelength-to-space mapping. Collected by an objective lens with a focal length of 10 mm, the resultant 1D illumination patterns are focused on the sample, such as flowing cells. Then the spectrum of each pulse is encoded with the spatial information of the sample. The pulses reflected from the sample re-enter the diffraction grating followed by the circulator. A section of SMF with a complementary GVD of 1360 ps/(nm·km) is used to perform optical pulse compression. The energy of compressed pulses is detected by a 1.2 GHz photodiode and sampled by an ADC synchronized to MLL. Image reconstruction is performed offline.

Fig. 2
figure 2

Experimental setup of the time-stretch-based single-pixel imaging system

MLL: mode-locked laser; DCF: dispersion compensating fiber; EDFA: erbium-doped fiber amplifier; MZM: Mach-Zehnder modulator; AWG: arbitrary waveform generator; PRBS: pseudo-random binary sequence; Cir: circulator; SMF: single mode fiber; PD: photo-detector; DSP: digital signal processor

Full size image

4.3 Applications of time-stretch-based single-pixel imaging

4.3.1 Surface inspection

The frame rate of the TSSPI system is equal to the ratio between the pulse repetition rate and the number of measurements acquired for image reconstruction. As the pulse repetition rate is tens of MHz, the resultant frame rate is hundreds of kHz (assuming that the number of captured measurements is 512). Consequently, the TSSPI system can observe fast moving objects.

Surface defect and morphology inspection is one of the significant applications in industry. First, the TSSPI system has been used to perform high-speed surface inspection for detecting micrometer-sized surface defects that travel at a speed as high as several meters per second. The original image of a quick response (QR) code and its reconstructed images at different compression ratios (25.0%, 15.0%, 10.0%, 7.5%, and 5.0%) are presented in Figs. 3a–3f. To illustrate the effect of the compression ratio on reconstruction accuracy, Fig. 3g shows the peak signal-to-noise ratio (PSNR) as a function of the compression ratio, indicating that more measurements are required for a better reconstruction result.

Fig. 3
figure 3

Surface inspection: (a) original image of a quick response code; (b)–(f) reconstructed images at compression ratios of 25.0%, 15.0%, 10.0%, 7.5%, and 5.0%, respectively; (g) peak signal-to-noise ratio versus compression ratio curve

Full size image

The above-mentioned surface inspector can provide a scan rate of hundreds of kilo frames per second or even several giga frames per second, which is 1000 times larger than that of conventional single-pixel cameras. This inspector is expected to improve the cost-effectiveness and performance of organic light-emitting diode displays for next-generation smart phones.

4.3.2 Imaging flow cytometry

The imaging flow analyzer based on STEAM faces the ‘big data’ problem. As TSSPI has a data compression capability, it can be integrated with a flow cytometer to solve this problem. Here, a microfluidic device is required and cells are controlled to flow at a uniform velocity, focused and ordered in the microfluidic channel by intrinsic inertial lift force. The imaging flow cytometer based on TSSPI is demonstrated to enable high-throughput screening of cells with an unprecedented throughput of 30 000 cells/s. Fig. 4 shows the captured snapshots of flowing cells reconstructed from different numbers of measurements which correspond to different compression ratios (63%, 47%, 31%, 23%, 20%, and 12%).

Fig. 4
figure 4

Imaging flow cytometry based on time-stretch-based single-pixel imaging

Some snapshots of flowing cells reconstructed from different numbers of measurements are presented

Full size image

This method has greatly relieved the pressure on data acquisition and storage, because the sampling rates of back-end digitizers are only several hundred MS/s. It is effective for circulating tumor cell (CTC) detection because of its ability to significantly increase the detection speed. Therefore it is promising for early, noninvasive, and low-cost detection of cancer, and evaluation of chemotherapy.

5 Conclusions

In summary, we have reviewed the key principles of the SPI, STEAM, and TSSPI technology. The efficient data compression and fast image acquisition of the TSSPI system were highlighted. The applications of TSSPI in industry and biomedicine were introduced, demonstrating that TSSPI is a promising method for high-speed surface inspection and high-throughput imaging flow cytometry.

There are still some limitations of the TSSPI technology. Though TSSPI can relieve the pressure on data acquisition and storage, image reconstruction is a greatly computation-consuming operation. This prevents TSSPI from working in real time. Therefore, more efficient reconstruction algorithms should be explored to solve this problem. In addition, TSSPI can operate in only the near- and mid-infrared range due to the limit of electro-optical modulators. This challenge may be overcome if new modulators with a larger operational wavelength range emerge.