Skip to main content

Real Time Deconvolution of Adaptive Optics Ground Based Telescope Imagery

Abstract

This article presents new developments in a multiframe blind deconvolution algorithm for imaging low earth orbiting objects during flyover. The foundational aspects of the algorithm rely on the constrained maximum likelihood (ML) formulation in the presence of Poisson noise, previously developed in Schulz et al. (2018). The new algorithm achieves real time evaluation capability at over two frames per second, which is allowed by two novel aspects. First, the prototype algorithms are transferred to highly parallelized computations on graphical processing units (GPUs) with CUDA implementation that reduce the computational time of a single iteration by a factor of up to 100. Second, new numerical optimization strategies are developed and demonstrated to accelerate the convergence of the algorithm by a factor of 5 to 10. Several other new capabilities are also demonstrated in this article. We derive and implement a modified variation of the algorithm that achieves subpixel resolution, which is shown effective on real and simulated data. Finally, a new post-processing visual enhancement technique is proposed with several examples, which in part helps deal with the dynamic range degradation due to glint.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Code Availability

The software used in this article is available for purchase through Lickenbrock Technologies, Inc.

Notes

  1. In some instances, the optimal step length can be determined with an analytical expression.

  2. The images shown in Fig. 2 are slightly cropped for for closer examination.

  3. The examples were tested in prototype MATLAB code on standard CPU hardware, therefore they do not show the real time speed.

  4. Observe here that dk(x) is listed over pixels in which it is not measured, but in these instances it is multiplied by m(x) = 0, so it can be considered to take any arbitrary values here.

References

  1. Abijaoude, M., Schulz, T., Gray, D., Ostroot, M., Pineros, M., Holmes, T.: Practical aspects of multiframe blind deconvolution for ground based telescopes. In: Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing 2018, vol. 10772, p. 1077212. International Society for Optics and Photonics (2018)

  2. Bertero, M., Boccacci, P., Desiderà, G., Vicidomini, G.: Image deblurring with Poisson data: from cells to galaxies. Inv. Probl. 25(12), 123006 (2009)

    Article  MathSciNet  Google Scholar 

  3. Biggs, D. S., Andrews, M.: Acceleration of iterative image restoration algorithms. Appl. Opt. 36(8), 1766–1775 (1997)

    Article  Google Scholar 

  4. Castleman, K. R.: Digital Image Processing CUMINCAD (1993)

  5. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  6. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image restoration by sparse 3D transform-domain collaborative filtering. In: Image Processing: Algorithms and Systems VI, vol. 6812, p. 681207. International Society for Optics and Photonics (2008)

  7. Fish, D., Brinicombe, A., Pike, E., Walker, J.: Blind deconvolution by means of the Richardson–Lucy algorithm. JOSA A 12(1), 58–65 (1995)

    Article  Google Scholar 

  8. Gonzalez, R. C., Woods, R. E., Eddins, S. L.: Digital Image Processing Using MATLAB. Pearson Education India, Bengaluru (2004)

    Google Scholar 

  9. Hardy, J. W.: Adaptive Optics for Astronomical Telescopes, vol. 16. Oxford University Press on Demand, Oxford (1998)

    Google Scholar 

  10. Holmes, T. J.: Blind deconvolution of quantum-limited incoherent imagery: maximum-likelihood approach. JOSA A 9(7), 1052–1061 (1992)

    Article  Google Scholar 

  11. Holmes, T. J., Liu, Y. H.: Acceleration of maximum-likelihood image restoration for fluorescence microscopy and other noncoherent imagery. JOSA A 8 (6), 893–907 (1991)

    Article  Google Scholar 

  12. Jefferies, S. M., Christou, J. C.: Restoration of astronomical images by iterative blind deconvolution. Astrophys. J. 415, 862 (1993)

    Article  Google Scholar 

  13. Lucy, L. B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745 (1974)

    Article  Google Scholar 

  14. Matson, C. L., Borelli, K., Jefferies, S., Beckner, C.C., Jr, Hege, E. K., Lloyd-Hart, M.: Fast and optimal multiframe blind deconvolution algorithm for high-resolution ground-based imaging of space objects. Appl. Opt. 48 (1), A75–A92 (2009)

    Article  Google Scholar 

  15. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer Science & Business Media, Berlin (2013)

    Google Scholar 

  16. Nesterov, Y.: A method of solving a convex programming problem with convergence rate O(1/k2). Sov. math., Dokl. 27, 372–376 (2013)

    Google Scholar 

  17. Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J. B., Zuiderveld, K.: Adaptive histogram equalization and its variations. Comput. Gr. Image Process. 39(3), 355–368 (1987)

    Article  Google Scholar 

  18. Ramos, A. A., de la Cruz Rodríguez, J., Yabar, A.P.: Real-time, multiframe, blind deconvolution of solar images. Astron. Astrophys. 620, A73 (2018)

    Article  Google Scholar 

  19. Richardson, W. H.: Bayesian-based iterative method of image restoration. JoSA 62(1), 55–59 (1972)

    Article  Google Scholar 

  20. Sanders, T., Platte, R. B.: Multiscale higher-order TV operators for L1 regularization. Adv. Struct. Chem. Imaging 4(1), 12 (2018)

    Article  Google Scholar 

  21. Sanders, T., Platte, R. B., Skeel, R. D.: Effective new methods for automated parameter selection in regularized inverse problems. Appl. Numer. Math. 152, 29–48 (2020)

    Article  MathSciNet  Google Scholar 

  22. Schulz, T., Abijaoude, M., Gray, D., Ostroot, M., Pineros, M., Holmes, T.: Advances in multiframe blind deconvolution for ground based telescopes. In: Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing 2018, vol. 10772, p. 107720F. International Society for Optics and Photonics (2018)

  23. Schulz, T. J.: Multiframe blind deconvolution of astronomical images. JOSA A 10(5), 1064–1073 (1993)

    Article  Google Scholar 

  24. Schulz, T. J., Stribling, B. E., Miller, J. J.: Multiframe blind deconvolution with real data: imagery of the hubble space telescope. Optics express 1 (11), 355–362 (1997)

    Article  Google Scholar 

  25. Su, W., Boyd, S., Candes, E.: A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights. In: Advances in neural information processing systems, pp. 2510–2518 (2014)

  26. Van Noort, M., Van Der Voort, L. R., Löfdahl, M.G.: Solar image restoration by use of multi-frame blind de-convolution with multiple objects and phase diversity. Sol. Phys. 228(1), 191–215 (2005)

    Article  Google Scholar 

  27. Wächter, A., Biegler, L. T.: Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM J. Optim. 16 (1), 1–31 (2005)

    Article  MathSciNet  Google Scholar 

  28. Wang, H., Miller, P. C.: Scaled heavy-ball acceleration of the Richardson-Lucy algorithm for 3D microscopy image restoration. IEEE Trans. Image Process. 23(2), 848–854 (2013)

    Article  MathSciNet  Google Scholar 

Download references

Funding

Funding for this work is provided by the Air Force Research Laboratory, contract number FA9451-19-C-0579. The views expressed in this document are those of the authors and do not reflect the official policy or position of the United States Air Force, the Department of Defense, or the United States Government. Approved for public release; distribution is unlimited. Public Affairs release approval #OPS-20-38985.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Toby Sanders.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Advanced Maui Optical and Space Surveillance Technologies (AMOS 2020)

Guest Editors: James M. Frith, Lauchie Scott, Islam Hussein

Appendix A: PSF Iterations

Appendix A: PSF Iterations

Recall from “Deconvolution Model” the blurry image data model given in Eq. 1. To solve the objective function (3) for the object u and the PSFs simultaneously, an alternating maximization is used: the set of estimated PSFs is fixed and a new estimate of u is iterated, and then u is fixed and the PSFs are iterated. For a fixed set of PSFs, the MLE iteration on the image was given in Eq. 4, and its corresponding accelerated version in Eq. 7. For a fixed image variable u, the iteration for the PSFs was given in [22], and we provide the main ideas again here.

The approach uses a constrained model for the PSFs based on the optics and aperture of the telescope. The aperture provides a model for the PSFs, with a free parameter to optimize for being the optically and turbulence-induced phase aberration. The PSF iterations are determined similarly by differentiating the objective function in Eq. 3 with respect to the phase aberration.

Let |A(ξ)| denote the magnitude of the aperture function and 𝜃k(ξ) denote the optically and turbulence-induced phase aberration of the k-th image frame. Then the corresponding coherent and incoherent PSFs are given by (see [4] for details)

$$ g(x;\theta_{k}) = \mathcal{F}^{-1} \left\{ A(\xi) e^{i\theta_{k}(\xi) } \right\} , $$
(15)

and

$$ h_{k}(x) = |g(x; \theta_{k})|^{2}. $$
(16)

Differentiating the ML objective function in Eq. 3 with respect to 𝜃k, the fixed point phase iteration step is determined to be

$$ \theta^{n+1} = \text{Phase} {\left [ \mathcal{F} \left\{ \left( \frac{d_{k}} { {i_{k}^{n}} } * \tilde u^{n} (x) \right) g(x;{\theta^{n}_{k}}) \right\} \right ] }. $$
(17)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanders, T., Hedges, R., Schulz, T. et al. Real Time Deconvolution of Adaptive Optics Ground Based Telescope Imagery. J Astronaut Sci 69, 175–191 (2022). https://doi.org/10.1007/s40295-021-00285-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40295-021-00285-w

Keywords

  • Telescope imaging - Blind deconvolution - Real time evaluation - Subpixel resolution - Contrast enhancement