Skip to main content
SpringerLink
Log in

Circuit Design and Analysis of Smoothed \({l}_0\) Norm Approximation for Sparse Signal Reconstruction

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

\({L}_0\) norm plays a crucial role in sparse optimization, but discontinuities and non-convexity make the minimization of the \({l}_0\) norm be an NP-hard problem. To alleviate this problem, we design a smoothing function based on the sigmoid function to approximate the \({l}_0\) norm. To illustrate the physical realizability of the smoothing function and the advanced quality of the approximation, the proposed smoothing function is compared experimentally with several existing smoothing functions. Additionally, we analyze the parameters in the functions to determine the quality of the approximation. We investigate the circuit implementation of the proposed function and five existing smoothing functions; the simulation results show the effectiveness of the designed circuit on the Multisim platform. Experiments on the reconstruction of simulated sparse signals and real image data show that the proposed smoothing function is able to reconstruct sparse signals and images with lower mean square error (MSE) and higher peak signal-to-noise ratio (PSNR), respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Data availability

The datasets generated during the current study are available from the corresponding author upon reasonable request.

Code availability

All codes generated during the study appear in https://github.com/123d-com/L0-norm.git.

References

  1. D. Adil, R. Peng, S. Sachdeva. Fast, provably convergent irls algorithm for p-norm linear regression. In: Advances in Neural Information Processing Systems, vol. 32 (2019). https://proceedings.neurips.cc/paper/2019/file/46c7cb50b373877fb2f8d5c4517bb969-Paper.pdf

  2. T. Blumensath, M.E. Davies, Gradient pursuits. IEEE Trans. Signal Process. 56(6), 2370–2382 (2008). https://doi.org/10.1109/TSP.2007.916124

    Article  MathSciNet  MATH  Google Scholar 

  3. H. Boche, R. Calderbank, G. Kutyniok, J. Vybíral (eds.): Compressed Sensing and Its Applications: MATHEON Workshop 2013, 1st ed. 2015 edn. Applied and Numerical Harmonic Analysis. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16042-9

  4. P.T. Boufounos. Greedy sparse signal reconstruction from sign measurements. in: 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers, pp. 1305–1309 (2009). https://doi.org/10.1109/ACSSC.2009.5469926

  5. H. Che, J. Wang, A. Cichocki. Bicriteria sparse nonnegative matrix factorization via two-timescale duplex neurodynamic optimization. IEEE Trans. Neural Netw. Learn. Syst., 1–11 (2021). https://doi.org/10.1109/TNNLS.2021.3125457

  6. H. Che, J. Wang, A. Cichocki, Sparse signal reconstruction via collaborative neurodynamic optimization. Neural Netw. 154, 255–269 (2022). https://doi.org/10.1016/j.neunet.2022.07.018

    Article  Google Scholar 

  7. H. Che, J. Wang, W. Zhang, A collaborative neurodynamic approach to sparse coding, in Advances in Neural Networks - ISNN 2019. ed. by H. Lu, H. Tang, Z. Wang (Springer, Cham, 2019), pp.454–462

    Chapter  Google Scholar 

  8. W. Chen, M. Huang, W. Ye, X. Lou, Cascaded form sparse fir filter design. IEEE Trans. Circuits Syst. I Regul. Pap. 67(5), 1692–1703 (2020). https://doi.org/10.1109/TCSI.2020.2964568

    Article  MathSciNet  MATH  Google Scholar 

  9. C. Dai, H. Che, M.-F. Leung, A neurodynamic optimization approach for l1 minimization with application to compressed image reconstruction. Int.J. Artif. Intell. Tools 30(01), 2140007 (2021). https://doi.org/10.1142/S0218213021400078

  10. I. Dassios, D. Baleanu, Optimal solutions for singular linear systems of caputo fractional differential equations. Math. Methods Appl. Sci. 44(10), 7884–7896 (2021). https://doi.org/10.1002/mma.5410

  11. I. Dassios, K. Fountoulakis, J. Gondzio, A preconditioner for a primal-dual newton conjugate gradient method for compressed sensing problems. SIAM J. Sci. Comput. 37(6), 2783–2812 (2015). https://doi.org/10.1137/141002062

    Article  MathSciNet  MATH  Google Scholar 

  12. I.K. Dassios, Analytic loss minimization: Theoretical framework of a second order optimization method. Symmetry (2019). https://doi.org/10.3390/sym11020136

    Article  MATH  Google Scholar 

  13. C. Guo, Q. Yang, A neurodynamic optimization method for recovery of compressive sensed signals with globally converged solution approximating to \(l_{0}\) minimization. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1363–1374 (2015). https://doi.org/10.1109/TNNLS.2014.2341654

    Article  MathSciNet  Google Scholar 

  14. Z. Guo, J. Wang. A neurodynamic optimization approach to constrained sparsity maximization based on alternative objective functions. in: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2010). https://doi.org/10.1109/IJCNN.2010.5596553

  15. M.M. Hyder, K. Mahata, An improved smoothed \(\ell ^0\) approximation algorithm for sparse representation. IEEE Trans. Signal Process. 58(4), 2194–2205 (2010). https://doi.org/10.1109/TSP.2009.2040018

    Article  MathSciNet  MATH  Google Scholar 

  16. Y. Ji, W.-P. Zhu, B. Champagne, Recurrent neural network-based dictionary learning for compressive speech sensing. Circ. Syst. Signal Process. 38(8), 3616–3643 (2019). https://doi.org/10.1007/s00034-019-01058-5

    Article  Google Scholar 

  17. A. Jiang, H.K. Kwan, Y. Zhu, X. Liu, N. Xu, Y. Tang, Design of sparse fir filters with joint optimization of sparsity and filter order. IEEE Trans. Circuits Syst. I Regul. Pap. 62(1), 195–204 (2015). https://doi.org/10.1109/TCSI.2014.2354771

    Article  Google Scholar 

  18. S.G. Lingala, M. Jacob, Blind compressive sensing dynamic mri. IEEE Trans. Med. Imaging 32(6), 1132–1145 (2013). https://doi.org/10.1109/TMI.2013.2255133

    Article  Google Scholar 

  19. X. Liu, J. Wang, An analog circuit design for k-winners-take-all operations, in Neural Inf. Process. ed. by L. Cheng, A.C.S. Leung, S. Ozawa (Springer, Cham, 2018), pp.666–675

    Google Scholar 

  20. M.R. Mohammadi, E. Fatemizadeh, M.H. Mahoor, Non-negative sparse decomposition based on constrained smoothed l0 norm. Signal Process. 100, 42–50 (2014). https://doi.org/10.1016/j.sigpro.2014.01.010

    Article  Google Scholar 

  21. D. Needell, J.A. Tropp, Cosamp: Iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2009). https://doi.org/10.1016/j.acha.2008.07.002

    Article  MathSciNet  MATH  Google Scholar 

  22. W.O. Popoola, Z. Ghassemlooy, C.G. Lee, A.C. Boucouvalas, Scintillation effect on intensity modulated laser communication systems-a laboratory demonstration. Opt. Laser Technol. 42(4), 682–692 (2010). https://doi.org/10.1016/j.optlastec.2009.11.011

    Article  Google Scholar 

  23. U. Saeed, A survey of automatic person recognition using eye movements. Int. J. Pattern Recognit. Artif. Intell. 28(08), 1456015 (2014). https://doi.org/10.1142/S0218001414560151

    Article  Google Scholar 

  24. M. Storace, M. Parodi, D. Pastorino, V. Tripodoro, A method for defining analog circuits for the minimization of discrete functionals: An image processing application. Circ. Syst. Signal Process. 18(5), 457–477 (1999). https://doi.org/10.1007/BF01387466

    Article  MATH  Google Scholar 

  25. M. Taher Abuelma’atti, Universal cmos current-mode analog function synthesizer. IEEE Trans. Circu. Syst. I Fundam. Theory Appl. 49(10), 1468–1474 (2002). https://doi.org/10.1109/TCSI.2002.803356

    Article  Google Scholar 

  26. J.A. Tropp, A.C. Gilbert, Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53(12), 4655–4666 (2007). https://doi.org/10.1109/TIT.2007.909108

    Article  MathSciNet  MATH  Google Scholar 

  27. S. Uenohara, K. Aihara, Time-domain digital-to-analog converter for spiking neural network hardware. Circuits Syst. Signal Process. 40(6), 2763–2781 (2021). https://doi.org/10.1007/s00034-020-01597-2

    Article  Google Scholar 

  28. J. Wang, X.T. Wang, Sparse approximate reconstruction decomposed by two optimization problems. Circuits Syst. Signal Process. 37(5), 2164–2178 (2018). https://doi.org/10.1007/s00034-017-0667-6

    Article  MathSciNet  MATH  Google Scholar 

  29. L. Wang, P. Ye, J. Xiang. A modified algorithm based on smoothed l0 norm in compressive sensing signal reconstruction. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 1812–1816 (2018). https://doi.org/10.1109/ICIP.2018.8451799

  30. J. Xiang, H. Yue, X. Yin, G. Ruan, A reweighted symmetric smoothed function approximating l0-norm regularized sparse reconstruction method. Symmetry (2018). https://doi.org/10.3390/sym10110583

    Article  MATH  Google Scholar 

  31. H. Xue, Y. Song, Non-convex approximation based l0-norm multiple indefinite kernel feature selection. Appl. Intell. 50(1), 192–202 (2020). https://doi.org/10.1007/s10489-018-01407-y

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Fundamental Research Funds for the Central Universities (Grant No. SWU020006), National Natural Science Foundation of China (Grant Nos. 6200 3281, 62103428), Natural Science Foundation of Chongqing, China (Grant No. cstc2021jcyj-msxmX1169), and Natural Science Fund of Hunan Province (Grant No. 2021JJ40702).

Author information

Authors and Affiliations

  1. College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China

    JianJun Li

  2. Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China

    Hangjun Che

  3. College of Systems Engineering, National University of Defense Technology, Changsha, China

    Xiaoyang Liu

Authors
  1. JianJun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hangjun Che
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hangjun Che.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Che, H. & Liu, X. Circuit Design and Analysis of Smoothed \({l}_0\) Norm Approximation for Sparse Signal Reconstruction. Circuits Syst Signal Process 42, 2321–2345 (2023). https://doi.org/10.1007/s00034-022-02216-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-022-02216-y

Keywords

Search

Navigation