1 Introduction

To comprehensively characterize a satellite for space situational awareness (SSA), we need to know its type, location, orientation, purpose, and health (i.e., level of deterioration and functionality). Obtaining this information is challenging, especially for satellites that are faint due to their size or distance away (or both). For example, we have limited surveillance capability for both the smallest objects in Low Earth Orbit (LEO) and satellites in the 4\(\pi\)-steradian volume of space from Geosynchronous Equatorial Orbit (GEO) out to cislunar orbit: a region of increasing concern. Currently, there is no way to methodically monitor satellites’ health, even the largest and closest ones.

Objects are characterized by their spatial, spectral, and temporal signatures. The ultimate goal for an object’s characterization is a series of hyperspectral images that capture the object’s full 3D surface area in fine spatial and spectral detail. The latter data allow us to identify the object’s materials, shedding light on the object’s capability. However, we are currently far from this ideal situation. Although instruments for hyperspectral imaging exist, they typically require scanning in wavelength or one of the spatial dimensions. This limits the usefulness of these instruments to targets that are not rapidly changing pose. To the best of our knowledge, “snapshot” hyperspectral imaging of resolved targets, i.e., simultaneous acquisition of the spatial and spectral information, is still in its infancy, and there are no techniques in routine use for SSA. Here we investigate the potential for a novel snapshot hyperspectral imaging technique that leverages the imaging instrument’s aperture’s diffractive properties and the dispersive properties of the atmosphere. Here we define snapshot to be an exposure time that is shorter than, or commensurate with, the atmospheric coherence time [10].

Diffraction and dispersion both cause light at different wavelengths from a point source to land on spatially different locations in the image. That is, they produce a spectrally smeared signal. If we can unscramble the spectral information on the target that is inherently encoded in a broadband image, then we have a way to perform snapshot hyperspectral imaging.

In addition to the benefits discussed above, we note that snapshot hyperspectral imaging over a wide wavelength range offers the improved capability for detecting a faint object in proximity to a much brighter object. Accurate knowledge of the speckle structure in a short-exposure image due to the brighter object, requires knowledge of the target’s spectrum and how the light at different wavelengths is redistributed across the image. We can remove the bright object’s spectral contribution to the overall signal to facilitate the fainter object’s detection. We note that in this context, bright means in relation to the faint object. The absolute brightness of the primary object can still be faint. As an example, we may want to look for debris near a high-value asset in GEO. Here the brighter of the two objects is still dim.

In this paper, we focus on a proof-of-concept for the proposed atmosphere-instrument spectrometer concept. Is it even feasible to decode the target’s spectral information from a series of broadband speckle images? To answer this question, we investigate the relatively simple problem of two closely-spaced point-source objects of different brightness. With this type of object, the signal’s spectral spreading is not as smeared as it is for an extended object. The latter is a significantly more challenging problem to solve. In this feasibility study, we ignore photon noise and camera read noise.

2 Simulation Setup

2.1 Phase Screen Generator

We simulate the turbulence in the Earth’s atmosphere using Kolmogorov phase screens and the split-step beam propagation method [12]. This approach models the distributed turbulence along the propagation path utilizing 15 discrete, infinitely thin, phase screens equally spaced over the path. The variation in the phase across each screen represents the fluctuations induced by changes of the refractive index in the atmosphere over the volume of space half-way to adjacent screens.

The phase screen code [12] takes the initial wavefront and propagates it through a vacuum until it reaches a phase screen and repeats this process for the entire propagation path until every phase screen has perturbed the wavefront. The propagation between the phase screens also gives rise to amplitude fluctuations in the wavefront. The turbulence in each phase screen is defined by the refractive index structure parameter (\(C_n^2\)) which we model using the Hufnagel-Valley approximation [9]

$$\begin{aligned} C^2_n (z) = \mathbf {a}_{\mathbf {1}}~\exp (-z/H_1) + \mathbf {a}_{\mathbf {2}}~\exp (-z/H_2) + \mathbf {a}_{\mathbf {3}}~z^{10}\, \exp (-z/H_3)~, \end{aligned}$$
(1)

where the \(a_1\) coefficient represents the strength of the ground layer of the atmosphere, and \(H_1\) is the height of its 1/e decay. \(a_2\) and \(H_2\) are similarly defined for the turbulence in the troposphere, and \(a_3\) and \(H_3\) are related to the turbulence peak located at the tropopause. We select the values of the coefficients such that the \(C_N^2\) profile simulates the conditions at Mount Haleakala on Maui. We then make minor adjustments to the coefficients to obtain the desired Fried parameter [11],

$$\begin{aligned} r_0 = \left[ 0.423~\left( \frac{2\pi }{\lambda }\right) ^2 \sec \zeta \int _{0}^L C_n^2(z)~dz\right] ^{-3/5}~. \end{aligned}$$
(2)

The resulting turbulence and physical parameters are listed in Table 1. At the end of the propagation process, the wavefront is in the optical system’s aperture plane, and the values are wrapped between \(\pi\) and \(-\pi\). To unwrap the phase, we use the Goldstein branch cut phase unwrapping algorithm [2]. We note that we have ignored the variation of the refractive index (n) on wavelength, i.e., dispersion [8]. The effects of dispersion will be included in future studies.

Table 1 Table of simulation parameters
Full size table

2.2 Image Generation

In the case of incoherent light, the model for the noise-free intensity of an image \(g(x_i,y_i)\) is [3]

$$\begin{aligned} g(x_i,y_i) = \iint f(x_0,y_0)h(x_0,y_0;x_i,y_i)dx_0dy_0 \end{aligned}$$
(3)

where \((x_0,y_0)\) and \((x_i,y_i)\) are the coordinates in the object plane and the image plane respectively, \(f(x_0,y_0)\) is the object intensity distribution and \(h(x_0,y_0;x_i,y_i)\) is the space-varying point spread function (PSF) of the imaging system. For our simulations we assume that the field-of-view for our images is less than the size of the isoplanatic angle (\(\theta _0\)) for the atmosphere during the observations, where

$$\begin{aligned} \theta _0 = ~\left[ 2.91~\left( \frac{2\pi }{\lambda }\right) ^2 \sec \zeta \int _{0}^L C_n^2(z)~z^{5/3}~dz\right] ^{-3/5}~. \end{aligned}$$
(4)

In this case the PSF becomes spatially invariant over the field-of-view and Eq. 3 simplifies to a convolution [8]. Invoking the linear superposition principle for electromagnetic fields [3], which allows us to model a broadband image as a summation of monochromatic images, we then generate an observed broadband image using

$$\begin{aligned} g_{x,y} = \sum \limits _{\lambda _1}^{ \lambda _2} \varDelta \lambda _N\left( f_\lambda \circledast h_{\lambda }\right) _{x,y}, \end{aligned}$$
(5)

where \(\circledast\) denotes convolution and \(\lambda _2 - \lambda _1\) is the spectral bandwidth for the observation. The spectral bandwidth is sampled at N discrete wavelengths such that \(\varDelta \lambda _N = (\lambda _2-\lambda _1)/N\). We use \(\lambda _1\) = 400 nm and \(\lambda _2\) = 1000 nm. This is the wavelength range over which silicon-based cameras have sensitivity. We set \(\varDelta \lambda _N\) = 15 nm and generate monochromatic images for \(N=40\) color planes. This sampling represents the change in wavelength needed to provide a subtle visible difference in the speckle morphology of the PSFs created using the wavefronts generated at two adjacent wavelengths near 400 nm. For the object, we use point sources to simulate two unresolved, closely spaced objects. For simplicity, these sources are given by Blackbody spectra with different temperatures. We then generate a range of objects for our validation tests by varying the separation and difference in the objects’ brightness. We model the monochromatic point spread functions using

$$\begin{aligned} h_{x,y,\lambda } = | FT^{-1}\left( A(u,v) \exp {-i\phi _\lambda (u,v)} \right) |^2 \end{aligned}$$
(6)

where \(FT^{-1}\) denotes the inverse Fourier transform operator. For this work we use wavefronts that represent turbulence conditions of \(D/r_0=33\). This simulates observations with the AEOS telescope on Mount Haleakala through median daytime seeing conditions during the Fall (see Table 1 of [1]).

Figure 1 shows example images of a turbulence degraded binary star object at selected monochromatic color planes of 400 nm, 700 nm, and 1000 nm, along with the ultra-broadband integrated over 40 color planes, covering 400 nm to 1000 nm (left to right), for four different realizations of the atmosphere (top to bottom). The contrast in the secondary component’s brightness to that of the binary’s primary component is \(10^{-2}\). The images’ pixel scale is set so that the data at 400 nm are Nyquist sampled (i.e. 0.014 arcsec/pixel).

Fig. 1
figure 1

Examples of simulated monochromatic and broadband images for four different realizations of atmospheric turbulence represented by each row. The first three columns are monochromatic speckle images at 1000, 700, and 400 nm respectively. The final column is the resulting broadband image integrated over 40 color planes

Full size image

2.3 Image Reconstruction

For our initial investigation into the feasibility of recovering the original object from a set of broadband images, we assume we have a high-quality measurement of the PSF and that we only need to recover the object from a time series of observed broadband images. We then minimize the least-squares cost function

$$\begin{aligned} \epsilon = \sum \limits _k \sum \limits _{x,y} \left( g_{k,x,y}- \hat{g}_{k,x,y}\right) ^2 = \sum \limits _k \sum \limits _{x,y} r_{k,x,y}^2 ~, \end{aligned}$$
(7)

where

$$\begin{aligned} \hat{g}_{k,x,y} = \sum \limits _{\lambda _1}^{\lambda _2} \varDelta \lambda _M \left( \hat{f}_\lambda \circledast \hat{h}_{k,\lambda }\right) _{x,y} ~, \end{aligned}$$
(8)

and r is the residual difference between the observed and modeled images. Here \(\varDelta \lambda _M=(\lambda _2 - \lambda _1)/M.\) with \(M < N\). That is, we estimate the object at fewer color planes than were used to generate the broadband images. The pixels in the M color planes of \(\hat{f}\) are the variables in the minimization. We note the “monochromatic” PSF in Eq. 8 represents an integral of N/M of the monochromatic PSFs used in the generation of the broadband images.

We minimize the cost function using the variable metric limited memory optimization algorithm. This is a gradient-based algorithm and we calculate the gradient of the cost function with respect to the variables using

$$\begin{aligned} \frac{d \epsilon }{d\hat{f}}_{x,y,\lambda } = -2 \sum \limits _k \left( r_k \star \hat{h}_{k,\lambda }\right) _{x,y}~, \end{aligned}$$
(9)

where \(\star\) denotes correlation.

For our validation tests, we generate broadband data sets for two separations between the two unresolved objects (0.69 arc-seconds and 0.14 arc-seconds) and two contrast ratios for each separation (the brightness of the secondary component is \(10^{-2}\) and \(10^{-3}\) of the brightness of the primary component). For all the reconstructions presented in the paper, we use \(k=11\) broadband images and allow 1,000 iterations in the minimization routine. The number of iterations used was chosen to minimize computation time.

3 Results

For our initial proof-of-concept tests, we assume we have perfect knowledge of the wavefront for each of the M color planes of every broadband image. That is, \(\hat{h}_\lambda = h_\lambda\). The initial guess for our object is a shift-and-add of our 11 data frames. Our first two data sets represent the two unresolved objects at 0.69 arc-second separation. The spectral recoveries for the two components of the object are shown in Fig. 2. The top row of the figure shows the results for the bright (primary) object, and the bottom row shows the results for the dim (secondary) object. The left-hand column represents a secondary:primary contrast in brightness of \(10^{-2}\) and the right column represents a contrast of \(10^{-3}\). In both cases, the spectrum of the bright component is well recovered. This validates our hypothesis that spectral information on a target can be recovered from a series of ultra-broadband speckle images. Interestingly, the secondary component’s recovered spectrum is a reasonable representation of the truth spectrum when the contrast is \(10^{-2}\). This suggests the spectral recovery is robust over a useful dynamic range. When the brightness of the secondary decreases to \(10^{-3}\) of the primary object, the recovered spectrum for the former has little similarity to the truth spectrum. If we are just interested in detecting the faint companion, however, then we can integrate the recovered object over all the color planes. In this case, we find for both contrast scenarios that 99% of the counts for the primary component are recovered, while the recovery of the secondary components varies slightly between the two contrast ratios: at \(10^{-2}\) contrast, 90% of the counts are recovered while at \(10^{-3}\), 88% are recovered. That is, the secondary component is clearly seen in the restored images, even at the lowest contrast of \(10^{-3}\).

Fig. 2
figure 2

The recovered (solid symbols) and truth (solid line) spectra for a binary system with primary and secondary objects separated by 0.69 arc-seconds, \(10^{-2}\) contrast ratio (Left), and \(10^{-3}\) contrast ratio (Right)

Full size image

To assess the applicability of ultra-broadband speckle imaging for high-resolution, high-contrast imaging, we ran some additional tests where we reduced the separation between the two unresolved objects to 0.14 arc-seconds. The results are shown in Fig. 3 with the same format as in Fig. 2. Figure 3 shows that the primary component is recovered with similar accuracy to the earlier results for the 0.69 arc-second separation. However, the secondary component loses some accuracy compared to its counterpart in Fig. 2. Despite this, 95% of the total counts are still recovered. Finally, at \(10^{-3}\) contrast, although the secondary component’s recovered spectrum is a poor representation of the truth spectrum, the secondary component’s integrated flux still contains 67% of the true flux. Looking at the spectrally integrated recovered object in Fig. 4, which is shown in fourth root scale to emphasize the dimmer secondary component, we see the secondary component start to become visible to the right of the primary, at redder wavelengths. In the final image, which is integrated over the broadband range, the center of mass of the object becomes more clear. This suggests the potential for high-contrast, high-resolution imaging.

Fig. 3
figure 3

The recovered (solid symbols) and truth (solid line) spectra for a binary system with primary and secondary objects separated by 0.14 arc-seconds, \(10^{-2}\) contrast (Left), and \(10^{-3}\) contrast (Right)

Full size image
Fig. 4
figure 4

The recovered image of two point-sources separated by 0.14 arc-seconds with \(10^{-3}\) contrast. The three images, from left to right, are images of the recovered object at individual wavelength planes at 400, 700, and 1000 nm respectively. The final image on the right is the integrated object over the entire broadband range

Full size image

For our last test, we relax the requirement on the fidelity of the PSF. Again, we use a binary target, this time with a separation of 0.69 arc-seconds and a contrast of \(10^{-1}\) and \(10^{-2}\) between the two components. We replace the truth PSFs at each wavelength with PSFs generated from wavefronts with a root-mean-square phase error of 0.2 radians with respect to the truth wavefronts. We did this by smoothing the truth phases with a Gaussian kernel, which removes the wavefront’s high spatial frequencies. This level of wavefront error at \(D/r_0=33\) is commensurate with the performance obtained with an imaging Shack-Hartmann wavefront sensor, and multi-aperture phase retrieval of the WFS data [4, 6]. We note the impact of the missing high-spatial frequency information on the morphology of the PSFs will increase as the wavelength decreases. We also ignore variations in the wavefront amplitude when generating \(\hat{h}_{k,\lambda }\). The results in Fig. 5, which has similar format to Fig. 2, are similar to the results obtained with perfect knowledge of the PSFs, where the most deviation is found in the continuum of the secondary component.This suggests that the restoration’s quality is not highly sensitive to the fidelity of the PSF estimates (\(h_{k,\lambda }\)). This characteristic offers hope for the practical realization of ultra-broadband speckle imaging. We also note that with the addition of photon noise we would require more data images to compensate.

Fig. 5
figure 5

The recovered (solid symbols) and truth (solid line) spectra for a binary system with primary and secondary objects separated by 0.69 arc-seconds, \(10^{-1}\) contrast (Left), and \(10^{-2}\) contrast (Right), using degraded PSFs

Full size image

4 Conclusion

Our research has provided two advances. First, we have shown that for the idealized case of an unresolved binary object observed through \(D/r_0=33\) turbulence, without photon and read noise, it is feasible to recover the spectral information on the target, on a pixel-by-pixel basis, from a series of ultra-broadband images when we have high-quality estimates of the atmospheric PSFs and their variation with wavelength. Fortunately, recent advances in wavefront sensing provide such estimates. In addition, we can improve any measured estimates of the wavefronts using the blind deconvolution technique during the image restoration process. In this case, we note that broadband images are inherently a source of wavelength diverse information. As the wavefront phases at different wavelengths are related to the optical path difference via

$$\begin{aligned} \phi _\lambda (u,v) = \frac{2\pi }{\lambda } W(u,v)~, \end{aligned}$$
(10)

this inherent wavelength diversity provides a strong lever for the blind deconvolution process. We only need to estimate the values of W in the pupil in order to estimate all \(h_{k,\lambda }\) for a given frame k.

The second advance from our research is that we have demonstrated that ultra-broadband speckle imaging has the potential to better resolve and detect closely spaced objects when one object is significantly brighter than the other, over what can be achieved using narrow-band speckle imaging.

We note that all of the above results were obtained using 11 image frames. In practice, hundreds of frames will be available. Increasing the number of photons available for the restoration, by increasing the spectral bandwidth or increasing the number of frames, will improve the recoveries’ signal-to-noise ratio [7] and allow us to reach high contrast values. As an example, [5] showed that using 2,000 frames of monochromatic speckle data with \(\varDelta \lambda\) = 40 nm obtained on an 8-meter telescope at a cadence of 1 kHz, they can detect a secondary object whose brightness is \(10^{-4}\) of the brightness of the primary object. This is the current state-of-the-art for speckle imaging. Extending the spectral bandpass to \(\varDelta \lambda\) = 400 nm, as used in our simulations, offers the potential to improve the contrast to \(3.3\times10^{-5}\).

In closing, we emphasize that although these initial studies are incredibly encouraging, we need to perform additional research on ultra-broadband speckle imaging before claiming it is a viable approach for snapshot hyper-spectral imaging for SSA. Objects in GEO would be the obvious first step as these would be point-source like objects while more adjustments would need to be made for observing extended objects in LEO.