Abstract
This article presents new developments in a multiframe blind deconvolution algorithm for imaging low earth orbiting objects during flyover. The foundational aspects of the algorithm rely on the constrained maximum likelihood (ML) formulation in the presence of Poisson noise, previously developed in Schulz et al. (2018). The new algorithm achieves real time evaluation capability at over two frames per second, which is allowed by two novel aspects. First, the prototype algorithms are transferred to highly parallelized computations on graphical processing units (GPUs) with CUDA implementation that reduce the computational time of a single iteration by a factor of up to 100. Second, new numerical optimization strategies are developed and demonstrated to accelerate the convergence of the algorithm by a factor of 5 to 10. Several other new capabilities are also demonstrated in this article. We derive and implement a modified variation of the algorithm that achieves subpixel resolution, which is shown effective on real and simulated data. Finally, a new post-processing visual enhancement technique is proposed with several examples, which in part helps deal with the dynamic range degradation due to glint.
This is a preview of subscription content, access via your institution.









Code Availability
The software used in this article is available for purchase through Lickenbrock Technologies, Inc.
Notes
In some instances, the optimal step length can be determined with an analytical expression.
The images shown in Fig. 2 are slightly cropped for for closer examination.
The examples were tested in prototype MATLAB code on standard CPU hardware, therefore they do not show the real time speed.
Observe here that dk(x) is listed over pixels in which it is not measured, but in these instances it is multiplied by m(x) = 0, so it can be considered to take any arbitrary values here.
References
Abijaoude, M., Schulz, T., Gray, D., Ostroot, M., Pineros, M., Holmes, T.: Practical aspects of multiframe blind deconvolution for ground based telescopes. In: Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing 2018, vol. 10772, p. 1077212. International Society for Optics and Photonics (2018)
Bertero, M., Boccacci, P., Desiderà, G., Vicidomini, G.: Image deblurring with Poisson data: from cells to galaxies. Inv. Probl. 25(12), 123006 (2009)
Biggs, D. S., Andrews, M.: Acceleration of iterative image restoration algorithms. Appl. Opt. 36(8), 1766–1775 (1997)
Castleman, K. R.: Digital Image Processing CUMINCAD (1993)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image restoration by sparse 3D transform-domain collaborative filtering. In: Image Processing: Algorithms and Systems VI, vol. 6812, p. 681207. International Society for Optics and Photonics (2008)
Fish, D., Brinicombe, A., Pike, E., Walker, J.: Blind deconvolution by means of the Richardson–Lucy algorithm. JOSA A 12(1), 58–65 (1995)
Gonzalez, R. C., Woods, R. E., Eddins, S. L.: Digital Image Processing Using MATLAB. Pearson Education India, Bengaluru (2004)
Hardy, J. W.: Adaptive Optics for Astronomical Telescopes, vol. 16. Oxford University Press on Demand, Oxford (1998)
Holmes, T. J.: Blind deconvolution of quantum-limited incoherent imagery: maximum-likelihood approach. JOSA A 9(7), 1052–1061 (1992)
Holmes, T. J., Liu, Y. H.: Acceleration of maximum-likelihood image restoration for fluorescence microscopy and other noncoherent imagery. JOSA A 8 (6), 893–907 (1991)
Jefferies, S. M., Christou, J. C.: Restoration of astronomical images by iterative blind deconvolution. Astrophys. J. 415, 862 (1993)
Lucy, L. B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745 (1974)
Matson, C. L., Borelli, K., Jefferies, S., Beckner, C.C., Jr, Hege, E. K., Lloyd-Hart, M.: Fast and optimal multiframe blind deconvolution algorithm for high-resolution ground-based imaging of space objects. Appl. Opt. 48 (1), A75–A92 (2009)
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer Science & Business Media, Berlin (2013)
Nesterov, Y.: A method of solving a convex programming problem with convergence rate O(1/k2). Sov. math., Dokl. 27, 372–376 (2013)
Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J. B., Zuiderveld, K.: Adaptive histogram equalization and its variations. Comput. Gr. Image Process. 39(3), 355–368 (1987)
Ramos, A. A., de la Cruz Rodríguez, J., Yabar, A.P.: Real-time, multiframe, blind deconvolution of solar images. Astron. Astrophys. 620, A73 (2018)
Richardson, W. H.: Bayesian-based iterative method of image restoration. JoSA 62(1), 55–59 (1972)
Sanders, T., Platte, R. B.: Multiscale higher-order TV operators for L1 regularization. Adv. Struct. Chem. Imaging 4(1), 12 (2018)
Sanders, T., Platte, R. B., Skeel, R. D.: Effective new methods for automated parameter selection in regularized inverse problems. Appl. Numer. Math. 152, 29–48 (2020)
Schulz, T., Abijaoude, M., Gray, D., Ostroot, M., Pineros, M., Holmes, T.: Advances in multiframe blind deconvolution for ground based telescopes. In: Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing 2018, vol. 10772, p. 107720F. International Society for Optics and Photonics (2018)
Schulz, T. J.: Multiframe blind deconvolution of astronomical images. JOSA A 10(5), 1064–1073 (1993)
Schulz, T. J., Stribling, B. E., Miller, J. J.: Multiframe blind deconvolution with real data: imagery of the hubble space telescope. Optics express 1 (11), 355–362 (1997)
Su, W., Boyd, S., Candes, E.: A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights. In: Advances in neural information processing systems, pp. 2510–2518 (2014)
Van Noort, M., Van Der Voort, L. R., Löfdahl, M.G.: Solar image restoration by use of multi-frame blind de-convolution with multiple objects and phase diversity. Sol. Phys. 228(1), 191–215 (2005)
Wächter, A., Biegler, L. T.: Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM J. Optim. 16 (1), 1–31 (2005)
Wang, H., Miller, P. C.: Scaled heavy-ball acceleration of the Richardson-Lucy algorithm for 3D microscopy image restoration. IEEE Trans. Image Process. 23(2), 848–854 (2013)
Funding
Funding for this work is provided by the Air Force Research Laboratory, contract number FA9451-19-C-0579. The views expressed in this document are those of the authors and do not reflect the official policy or position of the United States Air Force, the Department of Defense, or the United States Government. Approved for public release; distribution is unlimited. Public Affairs release approval #OPS-20-38985.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article belongs to the Topical Collection: Advanced Maui Optical and Space Surveillance Technologies (AMOS 2020)
Guest Editors: James M. Frith, Lauchie Scott, Islam Hussein
Appendix A: PSF Iterations
Appendix A: PSF Iterations
Recall from “Deconvolution Model” the blurry image data model given in Eq. 1. To solve the objective function (3) for the object u and the PSFs simultaneously, an alternating maximization is used: the set of estimated PSFs is fixed and a new estimate of u is iterated, and then u is fixed and the PSFs are iterated. For a fixed set of PSFs, the MLE iteration on the image was given in Eq. 4, and its corresponding accelerated version in Eq. 7. For a fixed image variable u, the iteration for the PSFs was given in [22], and we provide the main ideas again here.
The approach uses a constrained model for the PSFs based on the optics and aperture of the telescope. The aperture provides a model for the PSFs, with a free parameter to optimize for being the optically and turbulence-induced phase aberration. The PSF iterations are determined similarly by differentiating the objective function in Eq. 3 with respect to the phase aberration.
Let |A(ξ)| denote the magnitude of the aperture function and 𝜃k(ξ) denote the optically and turbulence-induced phase aberration of the k-th image frame. Then the corresponding coherent and incoherent PSFs are given by (see [4] for details)
and
Differentiating the ML objective function in Eq. 3 with respect to 𝜃k, the fixed point phase iteration step is determined to be
Rights and permissions
About this article
Cite this article
Sanders, T., Hedges, R., Schulz, T. et al. Real Time Deconvolution of Adaptive Optics Ground Based Telescope Imagery. J Astronaut Sci 69, 175–191 (2022). https://doi.org/10.1007/s40295-021-00285-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40295-021-00285-w
Keywords
- Telescope imaging - Blind deconvolution - Real time evaluation - Subpixel resolution - Contrast enhancement