Skip to main content
SpringerLink
Log in

Sequential Image Recovery from Noisy and Under-Sampled Fourier Data

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

A new algorithm is developed to jointly recover a temporal sequence of images from noisy and under-sampled Fourier data. Specifically we consider the case where each data set is missing vital information that prevents its (individual) accurate recovery. Our new method is designed to restore the missing information in each individual image by “borrowing” it from the other images in the sequence. As a result, all of the individual reconstructions yield improved accuracy. The use of high resolution Fourier edge detection methods is essential to our algorithm. In particular, edge information is obtained directly from the Fourier data which leads to an accurate coupling term between data sets. Moreover, data loss is largely avoided as coarse reconstructions are not required to process inter- and intra-image information. Numerical examples are provided to demonstrate the accuracy, efficiency and robustness of our new method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

All data sets are publicly available. MATLAB codes used for all numerical results are available from the authors upon request.

Notes

  1. We will use jump function and edge function interchangeably throughout our exposition.

  2. As in the one-dimensional case, it is straightforward to show that the final approximation of the edge mask given in (17) holds for multiple edge curves, see e.g. [2, 21, 22].

  3. To be clear, the methods cited here are primarily re-weighted \(\ell _1\) regularization methods (and often referred to as such), since the weights are iteratively adapted. By contrast the VBJS method does not iteratively adapt the weights, so it is a weighted \(\ell _1\) regularization scheme.

  4. To be clear, the typical MMV reconstruction process uses multiple observations at a single snapshot in time, that is, when there is no change in the underlying scene. This is in contrast to the problem of interest in this investigation, which considers a sequence of observations over time during which the underlying scene changes.

  5. Standard weighted \(\ell _1\) regularization schemes typically scale the weights to be inversely proportional to the magnitudes of the components in the sparse domain of the solution at each iteration. Numerical experiments in [2] demonstrate that the VBJS approach is more robust, especially in low SNR environments.

  6. Note that the change mask, as defined in (28), assigns nonzero values only when there is no change in the underlying image, which is consistent with the proposed optimization model (26).

  7. These are reasonable assumptions in many applications, such as the cars and tanks considered in our synthetic aperture radar (SAR) data example.

  8. Using the smallest or average radius would cause the reconstructed edge curve to sometimes lie inside the actual edge curve, resulting in an inaccurate change mask, \(\tilde{C}_j\) in (28).

  9. To the best of our knowledge, however, these procedures all use pixelated reconstructed images to detect the change, where as in our approach we use edge maps generated from the Fourier data to avoid data loss.

  10. Comparing two matrices by simply checking the relation for all elements (\(A = B \iff A(k,l) = B(k,l)\) for all k, l) is not appropriate since any single misidentified edge point, for example, due to noise, may cause the objects in two single object edge masks of the same object, say at corresponding times j and \(j+1\), to be determined as different.

  11. Although we incorporate information from \(J = 6\) data sets, for better visualization we display only four.

References

  1. Gelb, A., Tadmor, E.: Detection of edges in spectral data. Appl. Comput. Harmon. Anal. 7(1), 101–135 (1999)

    Article  MathSciNet  Google Scholar 

  2. Adcock, B., Gelb, A., Song, G., Sui, Y.: Joint sparse recovery based on variances. SIAM J. Sci. Comput. 41(1), 246–268 (2019). https://doi.org/10.1137/17M1155983

    Article  MathSciNet  MATH  Google Scholar 

  3. Gelb, A., Scarnati, T.: Reducing effects of bad data using variance based joint sparsity recovery. J. Sci. Comput. 78(1), 94–120 (2019)

    Article  MathSciNet  Google Scholar 

  4. Scarnati, T., Gelb, A.: Accurate and efficient image reconstruction from multiple measurements of fourier samples (2020)

  5. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MathSciNet  Google Scholar 

  6. Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 59(8), 1207–1223 (2006)

    Article  MathSciNet  Google Scholar 

  7. Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)

    Article  MathSciNet  Google Scholar 

  8. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  9. Candès, E.J., Wakin, M.B., Boyd, S.P.: Enhancing sparsity by reweighted \(\ell _1\) minimization. J. Fourier Anal. Appl. 14(5–6), 877–905 (2008)

    Article  MathSciNet  Google Scholar 

  10. Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE international conference on acoustics, speech and signal processing, pp. 3869–3872 (2008)

  11. Daubechies, I., DeVore, R., Fornasier, M., Gunturk, C.S.: Iteratively re-weighted least squares minimization for sparse recovery. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 63(1), 1–38 (2010)

    Article  Google Scholar 

  12. Liu, Y., Ma, J., Fan, Y., Liang, Z.: Adaptive-weighted total variation minimization for sparse data toward low-dose X-ray computed tomography image reconstruction. Phys. Med. Biol. 57(23), 7923–7956 (2012). https://doi.org/10.1088/0031-9155/57/23/7923

    Article  Google Scholar 

  13. Langer, A.: Automated parameter selection for total variation minimization in image restoration. J. Math. Imaging Vis. 57(2), 239–268 (2017)

    Article  MathSciNet  Google Scholar 

  14. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)

    Article  Google Scholar 

  15. Sobel, I., Feldman, G.: A \(3\times 3\) isotropic gradient operator for image processing. A Talk at the Stanford Artificial Project, 271–272 (1968)

  16. Jin-Yu, Z., Yan, C., Xian-Xiang, H.: Edge detection of images based on improved Sobel operator and genetic algorithms. In: IEEE international conference on image analysis and signal processing, pp. 31–35 (2009)

  17. Gao, W., Zhang, X., Yang, L., Liu, H.: An improved Sobel edge detection. In: IEEE 3rd international conference on computer science and information technology, vol. 5, pp. 67–71 (2010)

  18. Gelb, A., Tadmor, E.: Detection of edges in spectral data II. Nonlinear enhancement. SIAM J. Num. Anal. 38(4), 1389–1408 (2000)

    Article  MathSciNet  Google Scholar 

  19. Archibald, R., Gelb, A., Platte, R.: Image reconstruction from undersampled Fourier data using the polynomial annihilation transform. J. Sci. Comput. (2015). https://doi.org/10.1007/s10915-015-0088-2

    Article  MATH  Google Scholar 

  20. Martinez, A., Gelb, A., Gutierrez, A.: Edge detection from non-uniform Fourier data using the convolutional gridding algorithm. J. Sci. Comput. 61(3), 490–512 (2014)

    Article  MathSciNet  Google Scholar 

  21. Gelb, A., Song, G.: Detecting edges from non-uniform Fourier data using Fourier frames. J. Sci. Comput. 71(2), 737–758 (2017)

    Article  MathSciNet  Google Scholar 

  22. Viswanathan, A., Gelb, A., Cochran, D.: Iterative design of concentration factors for jump detection. J. Sci. Comput. 51(3), 631–649 (2012)

    Article  MathSciNet  Google Scholar 

  23. Xie, W., Deng, Y., Wang, K., Yang, X., Luo, Q.: Reweighted \(\ell _1\) regularization for restraining artifacts in FMT reconstruction images with limited measurements. Opt. Lett. 39(14), 4148–4151 (2014)

    Article  Google Scholar 

  24. Landi, G.: The Lagrange method for the regularization of discrete ill-posed problems. Comput. Optim. Appl. 39(3), 347–368 (2008)

    Article  MathSciNet  Google Scholar 

  25. Wen, Y.-W., Chan, R.H.: Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Trans. Image Process. 21(4), 1770–1781 (2011)

    Article  MathSciNet  Google Scholar 

  26. Yang, X., Hofmann, R., Dapp, R., Van de Kamp, T., dos Santos Rolo, T., Xiao, X., Moosmann, J., Kashef, J., Stotzka, R.: TV-based conjugate gradient method and discrete L-curve for few-view CT reconstruction of X-ray in vivo data. Opt. Express 23(5), 5368–5387 (2015)

    Article  Google Scholar 

  27. Gong, C., Zeng, L.: Adaptive iterative reconstruction based on relative total variation for low-intensity computed tomography. Signal Process. 165, 149–162 (2019)

    Article  Google Scholar 

  28. Vogel, C.R.: Regularization parameter selection methods. Soc. Ind. Appl. Math. (2002). https://doi.org/10.1137/1.9780898717570.ch7

    Article  Google Scholar 

  29. Sanders, T., Platte, R.B., Skeel, R.D.: Effective new methods for automated parameter selection in regularized inverse problems. Appl. Numer. Math. 152, 29–48 (2020)

    Article  MathSciNet  Google Scholar 

  30. Shchukina, A., Kasprzak, P., Dass, R., Nowakowski, M., Kazimierczuk, K.: Pitfalls in compressed sensing reconstruction and how to avoid them. J. Biomol. NMR 68(2), 79–98 (2017). https://doi.org/10.1007/s10858-016-0068-3

    Article  Google Scholar 

  31. Kang, M.-S., Kim, K.-T.: Compressive sensing based SAR imaging and autofocus using improved Tikhonov regularization. IEEE Sens. J. 19(14), 5529–5540 (2019). https://doi.org/10.1109/JSEN.2019.2904611

    Article  Google Scholar 

  32. Churchill, V., Archibald, R., Gelb, A.: Edge-adaptive \(\ell _2\) regularization image reconstruction from non-uniform Fourier data. Inverse Probl. Imaging. 13 (2019). https://doi.org/10.3934/ipi.2019042

  33. Archibald, R., Gelb, A., Yoon, J.: Polynomial fitting for edge detection in irregularly sampled signals and images. SIAM J. Numer. Anal. 43(1), 259–279 (2005)

    Article  MathSciNet  Google Scholar 

  34. Song, P., Mota, J.F.C., Deligiannis, N., Rodrigues, M.R.D.: Coupled dictionary learning for multimodal image super-resolution. In: IEEE global conference on signal and information processing (GlobalSIP), pp. 162–166 (2016). https://doi.org/10.1109/GlobalSIP.2016.7905824

  35. Song, P., Deng, X., Mota, J.F.C., Deligiannis, N., Dragotti, P.L., Rodrigues, M.R.D.: Multimodal image super-resolution via joint sparse representations induced by coupled dictionaries. IEEE Trans. Comput. Imaging 6, 57–72 (2020). https://doi.org/10.1109/TCI.2019.2916502

    Article  MathSciNet  Google Scholar 

  36. Rigie, D.S., Rivière, P.J.L.: Joint reconstruction of multi-channel, spectral CT data via constrained total nuclear variation minimization. Phys. Med. Biol. 60(5), 1741–1762 (2015). https://doi.org/10.1088/0031-9155/60/5/1741

    Article  Google Scholar 

  37. Kazantsev, D., Jørgensen, J.S., Andersen, M.S., Lionheart, W.R.B., Lee, P.D., Withers, P.J.: Joint image reconstruction method with correlative multi-channel prior for X-ray spectral computed tomography. Inverse Probl. 34(6), 064001 (2018). https://doi.org/10.1088/1361-6420/aaba86

    Article  MathSciNet  MATH  Google Scholar 

  38. Eslahi, N., Foi, A.: Joint sparse recovery of misaligned multimodal images via adaptive local and nonlocal cross-modal regularization. In: IEEE 8th international workshop on computational advances in multi-sensor adaptive processing (CAMSAP), pp. 111–115 (2019). https://doi.org/10.1109/CAMSAP45676.2019.9022478

  39. Kazantsev, D., Lionheart, W.R.B., Withers, P.J., Lee, P.D.: Multimodal image reconstruction using supplementary structural information in total variation regularization. Sens. Imaging 15(1), 97 (2014). https://doi.org/10.1007/s11220-014-0097-5

    Article  Google Scholar 

  40. Chen, Y., Fang, R., Ye, X.: Joint image edge reconstruction and its application in multi-contrast MRI (2017) arXiv:1712.02000 [math.NA]

  41. Chen, G., Hay, G.J., Carvalho, L.M.T., Wulder, M.A.: Object-based change detection. Int. J. Remote Sens. 33(14), 4434–4457 (2012). https://doi.org/10.1080/01431161.2011.648285

    Article  Google Scholar 

  42. Hussain, M., Chen, D., Cheng, A., Wei, H., Stanley, D.: Change detection from remotely sensed images: From pixel-based to object-based approaches. ISPRS J. Photogramm. Remote. Sens. 80, 91–106 (2013). https://doi.org/10.1016/j.isprsjprs.2013.03.006

    Article  Google Scholar 

  43. Tewkesbury, A.P., Comber, A.J., Tate, N.J., Lamb, A., Fisher, P.F.: A critical synthesis of remotely sensed optical image change detection techniques. Remote Sens. Environ. 160, 1–14 (2015). https://doi.org/10.1016/j.rse.2015.01.006

    Article  Google Scholar 

  44. Inglada, J., Mercier, G.: A new statistical similarity measure for change detection in multitemporal SAR images and its extension to multiscale change analysis. IEEE Trans. Geosci. Remote Sens. 45(5), 1432–1445 (2007)

    Article  Google Scholar 

  45. Thonfeld, F., Feilhauer, H., Braun, M., Menz, G.: Robust change vector analysis (RCVA) for multi-sensor very high resolution optical satellite data. Int. J. Appl. Earth Obs. Geoinf. 50, 131–140 (2016)

    Google Scholar 

  46. Ye, S., Chen, D., Yu, J.: A targeted change-detection procedure by combining change vector analysis and post-classification approach. ISPRS 114, 115–124 (2016). https://doi.org/10.1016/j.isprsjprs.2016.01.018

    Article  Google Scholar 

  47. McDermid, G.J., Linke, J., Pape, A.D., Laskin, D.N., McLane, A.J., Franklin, S.E.: Object-based approaches to change analysis and thematic map update: challenges and limitations. Can. J. Remote. Sens. 34(5), 462–466 (2008)

    Article  Google Scholar 

  48. Chen, G., Zhao, K., Powers, R.: Assessment of the image misregistration effects on object-based change detection. ISPRS J. Photogramm. Remote. Sens. 87, 19–27 (2014). https://doi.org/10.1016/j.isprsjprs.2013.10.007

    Article  Google Scholar 

  49. Hsiao, Y.-T., Chuang, C.-L., Jiang, J.-A., Chien, C.-C.: A contour based image segmentation algorithm using morphological edge detection. In: IEEE international conference on systems, man and cybernetics, vol. 3, pp. 2962–2967 (2005). https://doi.org/10.1109/ICSMC.2005.1571600

  50. Papari, G., Petkov, N.: Adaptive pseudo dilation for gestalt edge grouping and contour detection. IEEE Trans. Image Process. 17(10), 1950–1962 (2008)

    Article  MathSciNet  Google Scholar 

  51. Papari, G., Petkov, N.: Edge and line oriented contour detection: state of the art. Image Vis. Comput. 29(2–3), 79–103 (2011)

    Article  Google Scholar 

  52. Gao, F., Wang, M., Cai, Y., Lu, S.: Extracting closed object contour in the image: remove, connect and fit. Pattern Anal. Appl. 22(3), 1123–1136 (2019)

    Article  MathSciNet  Google Scholar 

  53. Archibald, R., Gelb, A.: A method to reduce the Gibbs ringing artifact in MRI scans while keeping tissue boundary integrity. IEEE Trans. Med. Imaging 21(4), 305–319 (2002)

    Article  Google Scholar 

  54. Archibald, R., Chen, K., Gelb, A., Renaut, R.: Improving tissue segmentation of human brain MRI through preprocessing by the Gegenbauer reconstruction method. Neuroimage 20(1), 489–502 (2003)

    Article  Google Scholar 

  55. Afaq, Y., Manocha, A.: Analysis on change detection techniques for remote sensing applications: a review. Ecol. Inf. 63, 101310 (2021)

    Article  Google Scholar 

  56. Ash, J.N.: A unifying perspective of coherent and non-coherent change detection. In: Zelnio, E., Garber, F.D. (eds.) Algorithms for Synthetic Aperture Radar Imagery XXI, vol. 9093, pp. 90–98. SPIE, (2014). https://doi.org/10.1117/12.2054338. International Society for Optics and Photonics

  57. Li, H., Gong, M., Wang, Q., Liu, J., Su, L.: A multiobjective fuzzy clustering method for change detection in SAR images. Appl. Soft Comput. 46, 767–777 (2016). https://doi.org/10.1016/j.asoc.2015.10.044

    Article  Google Scholar 

  58. Ji, S., Xue, Y., Carin, L.: Bayesian compressive sensing. IEEE Trans. Signal Process. 56(6), 2346–2356 (2008)

    Article  MathSciNet  Google Scholar 

  59. Tipping, M.E.: Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1, 211–244 (2001)

    MathSciNet  MATH  Google Scholar 

  60. Wipf, D.P., Rao, B.D.: Sparse Bayesian learning for basis selection. IEEE Trans. Signal Process. 52(8), 2153–2164 (2004)

    Article  MathSciNet  Google Scholar 

  61. Boyd, S., Parikh, N., Chu, E.: Distributed optimization and statistical learning via the alternating direction method of multipliers (2011). https://doi.org/10.1561/2200000016

  62. Ellsworth, M., Thomas, C.: A fast algorithm for image deblurring with total variation regularization. Unmanned Tech Solutions 4 (2014)

  63. Lalwani, G., Livingston Sundararaj, J., Schaefer, K., Button, T., Sitharaman, B.: Synthesis, characterization, in vitro phantom imaging, and cytotoxicity of a novel graphene-based multimodal magnetic resonance imaging - X-ray computed tomography contrast agent. J. Mater. Chem. B 2(22), 3519–3530 (2015). https://doi.org/10.1039/C4TB00326H

    Article  Google Scholar 

  64. Glaubitz, J., Gelb, A.: High order edge sensors with \(\ell ^1\) regularization for enhanced discontinuous Galerkin methods. SIAM J. Sci. Comput. 41(2), 1304–1330 (2019)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the NSF grants DMS #1521661 (GS), DMS #1912685 (AG), DMS #1939203 (GS), AFOSR grant #FA9550-18-1-0316 (AG), and ONR MURI grant #N00014-20-1-2595 (AG).

Author information

Authors and Affiliations

  1. Department of Mathematics, Dartmouth College, Hanover, NH, 03755, USA

    Yao Xiao, Jan Glaubitz & Anne Gelb

  2. Department of Mathematics and Statistics, Old Dominion University, Norfolk, VA, 23529, USA

    Guohui Song

Authors
  1. Yao Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Glaubitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anne Gelb
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guohui Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anne Gelb.

Ethics declarations

Conflict of interest

The authors have not disclosed any competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, Y., Glaubitz, J., Gelb, A. et al. Sequential Image Recovery from Noisy and Under-Sampled Fourier Data. J Sci Comput 91, 79 (2022). https://doi.org/10.1007/s10915-022-01850-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-022-01850-7

Keywords

Mathematics Subject Classification

Search

Navigation