Skip to main content
SpringerLink
Log in

Nonlinear Least Squares in ℝN

  • Published:
Acta Applicandae Mathematicae Aims and scope Submit manuscript

Abstract

Recent research and new paradigms in mathematics, engineering, and science assume nonlinear signal models of the form ℳ=∪i∈I V i consisting of a union of subspaces V i instead of a single subspace ℳ=V. These models have been used in sampling and reconstruction of signals with finite rate of innovation, the Generalized Principle Component Analysis and the subspace segmentation problem in computer vision, and problems related to sparsity, compressed sensing, and dictionary design.

In this paper, we develop an algorithm that searches for the best nonlinear model of the form ℳ=∪ li=1 V i ⊂ℝN that is optimally compatible with a set of observations ℱ={f 1,…,f m }⊂ℝN. When l=1 this becomes the classical least squares optimization. Thus, this problem is a nonlinear version of the least squares problem. We test our algorithm on synthetic data as well as images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aharon, M., Elad, M., Bruckstein, A.M.: The k-svd: An algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  2. Aharon, M., Elad, M., Bruckstein, A.M.: On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them. Linear Algebra Appl. 416(1), 48–67 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  3. Aldroubi, A., Gröchenig, K.-H.: Non-uniform sampling in shift-invariant space. SIAM Rev. 43(4), 585–620 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  4. Aldroubi, A., Cabrelli, C.A., Molter, U.M.: Optimal nonlinear models for sparsity and sampling. J. Funct. Anal. Appl. 14(5–6), 793–812 (2008). Special issue on compressed sampling

    Article  MathSciNet  MATH  Google Scholar 

  5. Candès, E., Romberg, J.: Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6, 227–254 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  6. Candès, E., Tao, T.: Near optimal signal recovery from random projections: Universal encoding strategies. IEEE Trans. Inf. Theory 52, 5406–5425 (2006)

    Article  Google Scholar 

  7. Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)

    Article  Google Scholar 

  8. DeVore, R.A.: Deterministic constructions of compressed sensing matrices. J. Complex. 23(4–6), 918–925 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  9. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  10. Dragotti, P.L., Vetterli, M., Blu, T.: Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets strang-fix. IEEE Trans. Signal Process. 55, 1741–1757 (2007)

    Article  MathSciNet  Google Scholar 

  11. Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrica 1, 211–218 (1936)

    Article  Google Scholar 

  12. Gribonval, R., Nielsen, M.: Sparse decompositions in unions of bases. IEEE Trans. Inf. Theory 49, 3320–3325 (2003)

    Article  MathSciNet  Google Scholar 

  13. Ho, J., Yang, M.-H., Lim, J., Lee, K.-C., Kriegman, D.: Clustering appearances of objects under varying illumination conditions. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 11–18 (2003)

  14. Kanatani, K.: Motion segmentation by subspace separation and model selection. In: Proc. IEEE Intl Conf. Computer Vision, vol. 2, pp. 586–591 (2001)

  15. Kanatani, K., Sugaya, Y.: Multi-stage optimization for multi-body motion segmentation. In: Proc. Australia-Japan Advanced Workshop Computer Vision, pp. 335–349 (2003)

  16. Lu, Y., Do, M.N.: A theory for sampling signals from a union of subspaces. IEEE Trans. Signal Process. 56(6), 2334–2345 (2008)

    Article  MathSciNet  Google Scholar 

  17. Ma, Y., Derksen, H., Hong, W., Wright, J.: Segmentation of multivariate mixed data via lossy coding and compression. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 29(9), 1546–1562 (2007)

    Article  Google Scholar 

  18. Ma, Y., Yang, A., Derksen, H., Fossum, R.: Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Rev. 50(3), 413–458 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  19. Maravic, I., Vetterli, M.: Sampling and reconstruction of signals with finite rate of innovation in the presence of noise. IEEE Trans. Signal Process. 53, 2788–2805 (2005)

    Article  MathSciNet  Google Scholar 

  20. Rabbani, T., van den Heuvel, F.: Efficient Hough transform for automatic detection of cylinders in point clouds. In: Workshop Laser Scanning, Enschede, The Netherlands, September 12–14, 2005

  21. Rauhut, H., Schass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theory 54(5), 2210–2219 (2008)

    Article  Google Scholar 

  22. Tessera, R., Zarringhalam, K.: Subspace segmentation with unknown cardinality and dimensions. Preprint (2008)

  23. Torii, A., Imiya, A.: The randomized-Hough-transform-based method for great-circle detection on sphere. Pattern Recognit. Lett. 28, 1186–1192 (2007)

    Article  Google Scholar 

  24. Tropp, J.A.: Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50, 2231–2242 (2004)

    Article  MathSciNet  Google Scholar 

  25. Vidal, R., Ma, Y.: A unified algebraic approach to 2-D and 3-D motion segmentation. J. Math. Imaging Vis. 25, 403–421 (2006)

    Article  MathSciNet  Google Scholar 

  26. Vidal, R., Ma, Y., Sastry, S.: Generalized principal component analysis (gpca). IEEE Trans. Pattern Anal. Mach. Intell. 27, 1–15 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Vanderbilt University, 1326 Stevenson Center, Nashville, TN, 37240, USA

    Akram Aldroubi & Kourosh Zaringhalam

Authors
  1. Akram Aldroubi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kourosh Zaringhalam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akram Aldroubi.

Additional information

The research of Akram Aldroubi is supported in part by NSF Grant DMS 0807464.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aldroubi, A., Zaringhalam, K. Nonlinear Least Squares in ℝN . Acta Appl Math 107, 325–337 (2009). https://doi.org/10.1007/s10440-008-9398-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10440-008-9398-9

Keywords

Search

Navigation