Skip to main content
SpringerLink
Log in

Sparse Representations for Image Decompositions

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We are given an image I and a library of templates \({\mathcal{L}}\) , such that \({\mathcal{L}}\) is an overcomplete basis for I. The templates can represent objects, faces, features, analytical functions, or be single pixel templates (canonical templates). There are infinitely many ways to decompose I as a linear combination of the library templates. Each decomposition defines a representation for the image I, given \({\mathcal{L}}\) .

What is an optimal representation for I given \({\mathcal{L}}\) and how to select it? We are motivated to select a sparse/compact representation for I, and to account for occlusions and noise in the image. We present a concave cost function criterion on the linear decomposition coefficients that satisfies our requirements. More specifically, we study a “weighted L norm” with 0 < p < 1. We prove a result that allows us to generate all local minima for the L norm, and the global minimum is obtained by searching through the local ones. Due to the computational complexity, i.e., the large number of local minima, we also study a greedy and iterative “weighted L Matching Pursuit” strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barlow, H.B. 1961. The Coding of Sensory Messages. Current Problems in Animal Behavior. Cambridge University Press; Cambridge.

    Google Scholar 

  • Barrodale, I. and Roberts, F. 1972. Solution of an overdetermined system of equations in the L 1 norm. Math. Dept. Report No. 69, University of Victoria.

  • Bell, A.J. and Sejnowski, T.J. 1995. An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7:1129–1159.

    Google Scholar 

  • Ben-Arie, J. and Rao, K.R. 1993. On the recognition of occluded shapes and generic faces using multiple-template expansion matching. In Proceedings IEEE International Conference on Pattern Recognition, New York City.

  • Bergeaud, F. and Mallat, S. 1995. Matching Pursuit of Images. SPIE: Orlando.

    Google Scholar 

  • Chen, S. and Donoho, D. 1994. Basis Pursuit. TR, Stanford University.

  • Chen, S. and Donoho, D. 1995. Atomic decomposition by basis pursuit. Technical Report, Stanford University.

  • Coifman, R. and Wickerhauser, V. 1992. Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38:2.

    Google Scholar 

  • Cormen, T.H., Leiserson, C.E., and Rivest, R.L. 1990. Introduction to Algorithms. McGraw-Hill.

  • Dahlquist, G. and Björck, Å. 1974. Numerical Methods. Prentice-Hall: Englewood Cliffs, N.J.

    Google Scholar 

  • Donahue, M.J. and Geiger, D. 1993. Template matching and function decomposition using non-minimal spanning Sets. Technical Report, Siemens.

  • Donahue, M.J., Geiger, D., Hummel R., and Liu, T. 1996. Representations for image decomposition with occlusions. In Proc. Computer Vision and Pattern Recognition, San Francisco, pp. 7–12.

  • Ekblom, H. 1973. L p -methods for robust regression. BIT 14, pp. 22–32.

    Google Scholar 

  • Field, D. 1994. What is the goal of sensory coding. Neural Comp. 6, pp. 559–601.

    Google Scholar 

  • Friedman, J.H. and Stuetzle, W. 1981. Projection pursuit regression. Journal of the American Statistical Association, 76, 817–823.

    Google Scholar 

  • Grenander, U. 1989. Advances in pattern theory: the 1985 rietz lecture. Annals Statists., 17(1):1–30.

    Google Scholar 

  • Huber, P.J. 1985. Projection pursuit. The Ann. of Stat., 13(2):435–475.

    Google Scholar 

  • Huber, P.J. 1981. Robust Statistics. John Wiley & Sons: New York.

    Google Scholar 

  • Huttenlocher, D., Klanderman, G., and Rucklidge, W. 1993. Comparing images using the hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850–863.

    Google Scholar 

  • Jones, L.K. 1987. On a conjecture of huber concerning the convergence of projection pursuit regression. The Ann. of Stat., 15(2):880–882.

    Google Scholar 

  • Kirby, M. and Sirovich, L. 1990. Application of the Karhunen-Loève Procedure for the characterization of human face. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-12(1):103–108.

    Google Scholar 

  • Krumm, J. 1996. Eigenfeatures for planar pose measurement of partially occluded objects. In Proc. Computer Vision and Pattern Recognition, San Francisco, pp. 55–60.

  • Leonardis, A. and Bischof, H. 1996. Dealing with occlusions in the eigenspace approach. In Proc. Computer Vision and Pattern Recognition, San Francisco, pp. 453–458.

  • Liu, T., Donahue, M., Geiger, D., and Hummel, R. 1996. Image recognition with occlusions. In Proc. 4th European Conf. on Computer Vision, London.

  • Mallat, S. and Zhang, Z. 1993. Matching pursuit with Time-Frequency Dictionaries. IEEE Trans. on Signal Processing.

  • Olshausen, B.A. and Field, D.J. 1996. Natural image statistics and efficient coding. Network: Computation in Neural Systems, 7:333–339.

    Google Scholar 

  • Poggio, T. and Girosi, F. 1990. Regularization algorithms for learning that are equivalent to multilayer network. Science, 247:978–982.

    Google Scholar 

  • Poggio, T. and Girosi, F. 1998. A sparse representation for function approximation. Neural Computation, in press.

  • Rissanen, J. 1978. Modeling by shortest data description. Automatica, 14:465–471.

    Google Scholar 

  • Rousseeuw, P.J. and Leroy, A. 1987. Robust Regression and Outlier Detection. John Wiley: New York.

    Google Scholar 

  • Ruderman, D.L. and Bialek, W. 1994. Statistics of natural images: scaling in the woods. Phys. Rev. Letters, 73.

  • Spyropoulos, K., Kountouzis, E., and Young, A. 1973. Discrete approximation in the L 1 norm. The Computer Journal, 16(2):180–186.

    Google Scholar 

  • Turk, M. and Pentland, A. 1991. Eigenfaces for recognition. J. of Cognitive Neuroscience, (3):71–86.

    Google Scholar 

  • Uhrin, B. 1990. An Elementary Constructive Approach to Discrete Linear l p -approximation, 0 < p ≤ + ∞. Colloquia Mathematica Societatis János Bolyai, 58. Approximation Theory, Kecskemét.

  • Wang, J. and Adelson, E.H. 1994. Representing moving images with layers. IEEE Transactions on Image Processing, 3(5):625–638.

    Google Scholar 

  • Zhu, S.C., Wu, Y.N., and Mumford, D. 1996. FRAME (filters, random fields, and minimax entropy): Towards a unified theory for texture modeling. In Proc. Comp. Vision and Patt. Recog., San Francisco.

Download references

Author information

Authors and Affiliations

  1. Courant Institute, New York University, New York NY, 10012, USA

    Davi Geiger

  2. Institute of Information Science, Academia Sinica, 115, Taipei, Taiwan

    Tyng-Luh Liu

  3. IMA, University of Minnesota, Minneapolis, MN, 55455, USA

    Michael J. Donahue

Authors
  1. Davi Geiger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tyng-Luh Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael J. Donahue
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Geiger, D., Liu, TL. & Donahue, M.J. Sparse Representations for Image Decompositions. International Journal of Computer Vision 33, 139–156 (1999). https://doi.org/10.1023/A:1008146126392

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008146126392

Search

Navigation