Skip to main content
SpringerLink
Log in

Adaptive deep learning network for image reconstruction of compressed sensing

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, we study how to achieve sparse sampling and high-quality reconstruction of natural images, and propose an interpretable deep network based on proximal gradient descent (PGD), dubbed AICS-Net, while performing joint constraint optimization of adaptive sparse sampling and reconstruction of images. AICS-Net consists of a sampling sub-network, an initialization sub-network and a recovery sub-network. The sampling sub-network achieves adaptive sampling of images, and the initialization sub-network uses the transpose of the measurement matrix to achieve initialized reconstruction of images. Integrating the gradient estimation strategy into the gradient descent step of the PGD algorithm in the recovery sub-network, then an optimization-based staged network structure is constructed. Moreover, to increase the hardware reputability of AICS-Net, binary constraint and orthogonal constraint are added to the measurement matrix. To improve the quality and visual effect of reconstructed images, a loss function is created to account for the color and texture differences between images. Experimental results show that the proposed AICS-Net can achieve better image reconstruction while maintaining reconstruction speed compared to existing state-of-the-art network-based CS algorithms, especially at low CS ratios. When the CS ratio is 1%, for Set11, PNSR and SSIM can be improved by at least 6.63% and 7.57%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Availability of data and materials

Not applicable.

Code availability

Not applicable.

References

  1. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). https://doi.org/10.1109/TIT.2006.871582

    Article  MathSciNet  Google Scholar 

  2. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006). https://doi.org/10.1109/TIT.2005.862083

    Article  MathSciNet  Google Scholar 

  3. Liutkus, A., Martina, D., Popoff, S., Chardon, G., Katz, O., Lerosey, G., Gigan, S., Daudet, L., Carron, I.: Imaging with nature: compressive imaging using a multiply scattering medium. Sci. Rep. 4(4), 5552 (2014). https://doi.org/10.1038/srep05552

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  4. Sekar, K., Devi, K.S., Srinivasan, P.: Compressed tensor completion: a robust technique for fast and efficient data reconstruction in wireless sensor networks. IEEE Sens. J. 22(11), 10794–10807 (2022). https://doi.org/10.1109/JSEN.2022.3169226

    Article  ADS  Google Scholar 

  5. Roohi, S.F., Zonoobi, D., Kassim, A.A., Jaremko, J.L.: Multi-dimensional low rank plus sparse decomposition for reconstruction of under-sampled dynamic mri. Pattern Recognit. 63, 667–679 (2016). https://doi.org/10.1016/j.patcog.2016.09.040

    Article  ADS  Google Scholar 

  6. Rani, M., Dhok, S.B., Deshmukh, R.B.: A systematic review of compressive sensing: concepts implementations and applications. IEEE Access 6, 4875–4894 (2018). https://doi.org/10.1109/ACCESS.2018.2793851

    Article  Google Scholar 

  7. Wu, Z., Zhang, L., Liu, H., Kou, N.: Enhancing microwave metamaterial aperture radar imaging performance with rotation synthesis. IEEE Sens. J. 16(22), 8035–8043 (2016). https://doi.org/10.1109/JSEN.2016.2609200

    Article  ADS  Google Scholar 

  8. Jiang, W., Tong, F., Zheng, S., Cao, X.: Estimation of underwater acoustic channel with hybrid sparsity via static-dynamic discriminative compressed sensing. IEEE Sens. J. 20(23), 14548–14558 (2020). https://doi.org/10.1109/JSEN.2020.3008163

    Article  ADS  Google Scholar 

  9. Nath, S., Mala, C.: Thermal image processing-based intelligent technique for object detection. Signal Image Video Process. 16(6), 1631–1639 (2022). https://doi.org/10.1007/s11760-021-02118-7

    Article  Google Scholar 

  10. Lu, Z., Chen, Y.: Single image super-resolution based on a modified u-net with mixed gradient loss. Signal Image Video Process. 16(5), 1143–1151 (2022). https://doi.org/10.1007/s11760-021-02063-5

    Article  Google Scholar 

  11. Lee, D.-H., Chen, K.-L., Liou, K.-H., Liu, C.-L., Liu, J.-L.: Deep learning and control algorithms of direct perception for autonomous driving. Appl. Intell. 51, 237–247 (2021). https://doi.org/10.1007/s10489-020-01827-9

    Article  Google Scholar 

  12. Lyu, P., Wei, M., Wu, Y.: High-precision and real-time visual tracking algorithm based on the siamese network for autonomous driving. Signal Image Video Process. 17(4), 1235–1243 (2022). https://doi.org/10.1007/s11760-022-02331-y

    Article  Google Scholar 

  13. Shi, Y., Feng, D., Cheng, Y., Boswas, S.: A natural language-inspired multilabel video streaming source identification method based on deep neural networks. Signal Image Video Process. 15(6), 1161–1168 (2021). https://doi.org/10.1007/s11760-020-01844-8

    Article  Google Scholar 

  14. Lata, K., Singh, P., Dutta, K.: Mention detection in coreference resolution: survey. Appl. Intell. 52, 9816–9860 (2022). https://doi.org/10.1007/s10489-021-02878-2

    Article  Google Scholar 

  15. Lata, K., Singh, P., Dutta, K.: Mention detection in coreference resolution: survey. Appl. Intell. 52, 9816–9860 (2022). https://doi.org/10.1007/s10489-021-02878-2

    Article  Google Scholar 

  16. Zhang, J., Ghanem, B.: ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In: Paper presented at 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018 (2018)

  17. Yao, H., Dai, F., Zhang, D., Ma, Y., Zhang, S., Zhang, Y., Tian, Q.: DR2-Net: deep residual reconstruction network for image compressive sensing. Neurocomputing 359, 483–493 (2019). https://doi.org/10.1016/j.neucom.2019.05.006

    Article  Google Scholar 

  18. Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In: Paper presented at 2016 IEEE/CVF conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016 (2016)

  19. Canh, T.N., Jeon, B.: Multi-scale deep compressive imaging. IEEE Trans. Comput. Imaging 7, 86–97 (2021). https://doi.org/10.1109/TCI.2020.3034433

    Article  Google Scholar 

  20. Joseph, G., Kafle, S., Varshney, P.K.: One-bit compressed sensing using generative models. In: Paper presented at 2020 IEEE international conference on acoustics, speech and signal processing, Barcelona, Spain, 04–08 May 2020 (2020)

  21. Chen, Y., Tan, B., Wu, J., Zhang, Z., Ren, H.: A deep image coding scheme with generative network to learn from correlated images. IEEE Trans. Multimed. 23, 2235–2244 (2021). https://doi.org/10.1109/TMM.2021.3087011

    Article  Google Scholar 

  22. Zheng, B., Zhang, J., Sun, G., Ren, X.: EnGe-CSNet: A trainable image compressed sensing model based on variational encoder and generative networks. Electron. 10(9), 1089–2002 (2021). https://doi.org/10.3390/electronics10091089

    Article  Google Scholar 

  23. Bora, A., Jalal, A., Price, E., Dimakis, A.G.: Compressed sensing using generative models. arxiv:1703.03208v1 (2017)

  24. Zhang, J., Zhao, C., Gao, W.: Optimization-inspired compact deep compressive sensing. IEEE J. Sel. Top. Signal Process. 14(4), 765–774 (2020). https://doi.org/10.1109/JSTSP.2020.2977507

    Article  ADS  Google Scholar 

  25. Zhang, Z., Liu, Y., Liu, J., Wen, F., Zhu, C.: AMP-Net: denoising-based deep unfolding for compressive image sensing. IEEE Trans. Image Process. 30, 1487–1500 (2021). https://doi.org/10.1109/TIP.2020.3044472

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  26. Sun, Y., Chen, J., Liu, Q., Liu, B., Guo, G.: Dual-path attention network for compressed sensing image reconstruction. IEEE Trans. Image Process. 29, 9482–9495 (2020). https://doi.org/10.1109/TIP.2020.3023629

    Article  ADS  Google Scholar 

  27. Shi, W., Jiang, F., Liu, S., Zhao, D.: Image compressed sensing using convolutional neural network. IEEE Trans. Image Process. 29, 375–388 (2020). https://doi.org/10.1109/TIP.2019.2928136

    Article  ADS  MathSciNet  Google Scholar 

  28. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Paper presented at 2016 IEEE/CVF conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016 (2016)

  29. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2014). https://doi.org/10.1561/2400000003

    Article  Google Scholar 

  30. Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005). https://doi.org/10.1109/TIT.2005.858979

    Article  MathSciNet  Google Scholar 

  31. Candès, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Math. 346(9), 589–592 (2008). https://doi.org/10.1016/j.crma.2008.03.014

    Article  MathSciNet  Google Scholar 

  32. Mallat, S.G., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993). https://doi.org/10.1109/78.258082

    Article  ADS  Google Scholar 

  33. Needell, D., Tropp, J.A.: CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2009). https://doi.org/10.1016/j.acha.2008.07.002

    Article  MathSciNet  Google Scholar 

  34. Dai, W., Milenkovic, O.: Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans. Inf. Theory 55(5), 2230–2249 (2009). https://doi.org/10.1109/TIT.2009.2016006

    Article  MathSciNet  Google Scholar 

  35. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995). https://doi.org/10.1109/18.382009

    Article  MathSciNet  Google Scholar 

  36. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009). https://doi.org/10.1137/080716542

    Article  MathSciNet  Google Scholar 

  37. Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009). https://doi.org/10.1016/j.acha.2009.04.002

    Article  MathSciNet  Google Scholar 

  38. Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans. Image Process. 20(3), 681–695 (2011). https://doi.org/10.1109/TIP.2010.2076294

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  39. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001). https://doi.org/10.1137/S003614450037906X

    Article  ADS  MathSciNet  Google Scholar 

  40. Yang, Y., Sun, J., Li, H., Xu, Z.: ADMM-CSNet: a deep learning approach for image compressive sensing. IEEE Trans. Pattern Anal. Mach. Intell. 42(3), 521–538 (2020). https://doi.org/10.1109/TPAMI.2018.2883941

    Article  PubMed  Google Scholar 

  41. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020). https://doi.org/10.1109/TPAMI.2019.2913372

    Article  PubMed  Google Scholar 

  42. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Paper presented at IEEE international conference on computer vision, Vancouver, BC, Canada, 07–14 July 2001 (2001)

  43. Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Paper presented at 2015 IEEE conference on computer vision and pattern recognition, Boston, MA, USA, 07–12 June 2015 (2015)

  44. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011). https://doi.org/10.1109/TPAMI.2010.161

    Article  PubMed  Google Scholar 

  45. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

Download references

Acknowledgements

The authors would like to thank Tianjin Key Laboratory of Optoelectronic Sensor and Sensor Network Technology for providing working conditions.

Funding

This work was supported by Natural Science Foundation of Tianjin City (No.21JCZDJC00340) and National Natural Science Foundation of China (No.61901233).

Author information

Authors and Affiliations

  1. College of Electronic Information and Optical Engineering, Nankai University, Tongyan Road, Jinnan District, 300350, Tianjin, China

    Ruili Nan, Guiling Sun, Bowen Zheng & Lin Wang

Authors
  1. Ruili Nan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guiling Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bowen Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Material preparation, data collection and analysis were performed by RN, BZ and LW. The first draft of the manuscript was written by RN, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Guiling Sun.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nan, R., Sun, G., Zheng, B. et al. Adaptive deep learning network for image reconstruction of compressed sensing. SIViP 18, 1463–1475 (2024). https://doi.org/10.1007/s11760-023-02879-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02879-3

Keywords

Search

Navigation