Sparse Approximation by Matching Pursuit Using Shift-Invariant Dictionary

  • Karl SkrettingEmail author
  • Kjersti Engan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10269)


Sparse approximation of signals using often redundant and learned data dependent dictionaries has been successfully used in many applications in signal and image processing the last couple of decades. Finding the optimal sparse approximation is in general an NP complete problem and many suboptimal solutions have been proposed: greedy methods like Matching Pursuit (MP) and relaxation methods like Lasso. Algorithms developed for special dictionary structures can often greatly improve the speed, and sometimes the quality, of sparse approximation.

In this paper, we propose a new variant of MP using a Shift-Invariant Dictionary (SID) where the inherent dictionary structure is maximally exploited. The dictionary representation is simple, yet flexible, and equivalent to a general M channel synthesis FIR filter bank. Adapting the MP algorithm by using the SID structure gives a fast and compact sparse approximation algorithm with computational complexity of order \(\mathcal {O}(N \log N)\). In addition, a method to improve the sparse approximation using orthogonal matching pursuit, or any other block-based sparse approximation algorithm, is described. The SID-MP algorithm is tested by implementing it in a compact and fast C code (Matlab mex-file), and excellent performance of the algorithm is demonstrated.


Sparse approximation Matching pursuit Dictionary Shift-invariant 


  1. 1.
    Aharon, M., Elad, M.: Sparse and redundant modeling of image content using an image-signature-dictionary. SIAM J. Imaging Sci. 1(3), 228–247 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). CrossRefGoogle Scholar
  3. 3.
    Barzideh, F., Skretting, K., Engan, K.: The flexible signature dictionary. In: Proceedings of 23rd European Signal Processing Conference, EUSIPCO-2015, Nice, France, September 2015Google Scholar
  4. 4.
    Cotter, S.F., Adler, J., Rao, B.D., Kreutz-Delgado, K.: Forward sequential algorithms for best basis selection. IEE Proc. Vis. Image Signal Process 146(5), 235–244 (1999)CrossRefGoogle Scholar
  5. 5.
    Cvetković, Z., Vetterli, M.: Oversampled filter banks. IEEE Trans. Signal Process. 46(5), 1245–1255 (1998)CrossRefGoogle Scholar
  6. 6.
    Davis, G.: Adaptive nonlinear approximations. Ph.D. thesis, New York University, September 1994Google Scholar
  7. 7.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407–499 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Engan, K., Aase, S.O., Husøy, J.H.: Method of optimal directions for frame design. In: Proceedings of ICASSP 1999, Phoenix, USA, pp. 2443–2446, March 1999Google Scholar
  9. 9.
    Engan, K., Skretting, K., Husøy, J.H.: A family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation. Digit. Signal Proc. 17, 32–49 (2007)CrossRefGoogle Scholar
  10. 10.
    Krstulovic, S., Gribonval, R.: MPTK: matching pursuit made tractable. In: Proceedings ICASSP 2006, Toulouse, France, vol. 3, pp. 496–499, May 2006Google Scholar
  11. 11.
    Mailhé, B., Gribonval, R., Bimbot, F., Vandergheynst, P.: A low complexity orthogonal matching pursuit for sparse signal approximation with shift-invariant dictionaries. In: Proceedings ICASSP 2009, Taipei, Taiwan, pp. 3445–3448, April 2009Google Scholar
  12. 12.
    Mailhé, B., Lesage, S., Gribonval, R., Bimbot, F.: Shift-invariant dictionary learning for sparse representations: extending K-SVD. In: Proceedings of the 16th European Signal Processing Conference (EUSIPCO-2008), Lausanne, Switzerland, August 2008Google Scholar
  13. 13.
    Mairal, J., Bach, F., Ponce, J.: Sparse modeling for image and vision processing. Found. Trends Comput. Graph. Vis. 8(2–3), 85–283 (2014)CrossRefzbMATHGoogle Scholar
  14. 14.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: ICML 2009: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 689–696. ACM, New York, June 2009Google Scholar
  15. 15.
    Mallat, S.G., Zhang, Z.: Matching pursuit with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993)CrossRefzbMATHGoogle Scholar
  16. 16.
    Massachusetts Institute of Technology: The MIT-BIH Arrhythmia Database CD-ROM, 2 edn. MIT (1992)Google Scholar
  17. 17.
    Muramatsu, S.: Structured dictionary learning with 2-D non-separable oversampled lapped transform. In: Proceedings ICASSP 2014, Florence, Italy, pp. 2643–2647, May 2014Google Scholar
  18. 18.
    O’Hanlon, K., Plumbley, M.D.: Structure-aware dictionary learning with harmonic atoms. In: Proceedings of the 19th European Signal Processing Conference (EUSIPCO-2011), Barcelona, Spain, August 2011Google Scholar
  19. 19.
    Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of Asilomar Conference on Signals Systems and Computers, November 1993Google Scholar
  20. 20.
    Pope, G., Aubel, C., Studer, C.: Learning phase-invariant dictionaries. In: Proceedings ICASSP 2013, Vancouver, Canada, May 2013Google Scholar
  21. 21.
    Rubinstein, R., Zibulevsky, M., Elad, M.: Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit. Technical report, CS Technion, Haifa, Israel, April 2008Google Scholar
  22. 22.
    Rubinstein, R., Zibulevsky, M., Elad, M.: Double sparsity: learning sparse dictionaries for sparse signal approximation. IEEE Trans. Signal Process. 58(3), 1553–1564 (2010)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Skretting, K., Engan, K.: Recursive least squares dictionary learning algorithm. IEEE Trans. Signal Process. 58, 2121–2130 (2010). doi: 10.1109/TSP.2010.2040671 MathSciNetCrossRefGoogle Scholar
  24. 24.
    Skretting, K., Husøy, J.H.: Partial search vector selection for sparse signal representation. In: NORSIG-2003, Bergen, Norway (2003).

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of StavangerStavangerNorway

Personalised recommendations