Uniqueness and Uncertainty

  • Michael EladEmail author


We return to the basic problem (P 0), which is at the core of our discussion,
$${\left(P_o\right):\quad \min\limits_X \parallel \mathbf{X}\parallel_0 \,{\rm subject\,\, to} \mathbf \quad \mathbf{b}=\mathbf{A\mathbf{x}}}.$$
While we shall refer hereafter to this problem as our main goal, we stress that we are quite aware of its two major shortcomings in leading to any practical tool. 1. The equality requirement b = A X is too strict, as there are small chances for any vector b to be represented by a few columns from A. A better requirement would be one that allows for small deviation. 2. The sparsity measure is too sensitive to very small entries in X, and a better measure would adopt a more forgiving approach towards such small entries.


Uncertainty Principle Diagonal Entry Generalize Uncertainty Principle Sparse Solution Small Entry 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. 1.
    A.R. Calderbank and P.W. Shor, Good quantum error-correcting codes exist, Phys. Rev. A, 54(2):1098–1105, August 1996.CrossRefGoogle Scholar
  2. 2.
    D.L. Donoho and M. Elad, Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization, Proc. of the National Academy of Sciences, 100(5):2197–2202, 2003.zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    D.L. Donoho and X. Huo, Uncertainty principles and ideal atomic decomposition, IEEE Trans. On Information Theory, 47(7):2845–2862, 1999.CrossRefMathSciNetGoogle Scholar
  4. 4.
    D.L. Donoho and P.B. Starck, Uncertainty principles and signal recovery, SIAM Journal on Applied Mathematics, 49(3):906–931, June, 1989.zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    M. Elad and A.M. Bruckstein, A generalized uncertainty principle and sparse representation in pairs of bases, IEEE Trans. On Information Theory, 48:2558–2567, 2002.zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    I.F. Gorodnitsky and B.D. Rao, Sparse signal reconstruction from limited data using FOCUSS: A re-weighted norm minimization algorithm, IEEE Trans. On Signal Processing, 45(3):600–616, 1997.CrossRefGoogle Scholar
  7. 7.
    S. Gurevich, R. Hadani, and N. Sochen, The finite harmonic oscillator and its associated sequences Proc. Natl. Acad. Sci. USA, 105(29):9869–9873, July, 2008.CrossRefMathSciNetGoogle Scholar
  8. 8.
    S. Gurevich, R. Hadani, and N. Sochen, On some deterministic dictionaries supporting sparsity, Journal of Fourier Analysis and Applications, 14(5–6):859–876, December, 2008.zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    R. Gribonval and M. Nielsen, Sparse decompositions in unions of bases, IEEE Trans. on Information Theory, 49(12):3320–3325, 2003.CrossRefMathSciNetGoogle Scholar
  10. 10.
    W. Heisenberg, The physical principles of the quantum theory, (C. Eckart and F.C. Hoyt, trans.), University of Chicago Press, Chicago, IL, 1930.Google Scholar
  11. 11.
    R.A. Horn C.R. Johnson, Matrix Analysis, New York: Cambridge University Press, 1985.zbMATHGoogle Scholar
  12. 12.
    X. Huo, Sparse Image representation Via Combined Transforms, PhD thesis, Stanford, 1999.Google Scholar
  13. 13.
    J.B. Kruskal, Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics, Linear Algebra and its Applications, 18(2):95–138, 1977.zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    P.W.H. Lemmens and J.J. Seidel, Equiangular lines, Journal of Algebra, 24(3):494–512, 1973.zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    X. Liu and N.D. Sidiropoulos, Cramer-Rao lower bounds for low-rank decomposition of multidimensional arrays, IEEE Trans. on Signal Processing, 49(9):2074–2086, 2001.CrossRefMathSciNetGoogle Scholar
  16. 16.
    B.K. Natarajan, Sparse approximate solutions to linear systems, SIAM Journal on Computing, 24:227–234, 1995.zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    W.W. Peterson and E.J. Weldon, Jr., Error-Correcting Codes, 2nd edition, MIT Press: Cambridge, Mass., 1972.zbMATHGoogle Scholar
  18. 18.
    A. Pinkus, N-Width in Approximation Theory, Springer, Berlin, 1985.Google Scholar
  19. 19.
    T. Strohmer and R.W. Heath, Grassmannian frames with applications to coding and communication, Applied and Computational Harmonic Analysis, 14:257–275, 2004.CrossRefMathSciNetGoogle Scholar
  20. 20.
    J.A. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. On Information Theory, 50(10):2231–2242, October 2004.CrossRefMathSciNetGoogle Scholar
  21. 21.
    J.A. Tropp, I.S. Dhillon, R.W. Heath Jr., and T. Strohmer, Designing structured tight frames via alternating projection, IEEE Trans. Info. Theory, 51(1):188–209, January 2005.CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Computer Science DepartmentThe Technion – Israel Institute of TechnologyHaifaIsrael

Personalised recommendations