Skip to main content
SpringerLink
Log in

Nonlinearly Structured Low-Rank Approximation

  • Chapter
  • First Online:
Low-Rank and Sparse Modeling for Visual Analysis

Abstract

Polynomially structured low-rank approximation problems occur in

  • algebraic curve fitting, e.g., conic section fitting,

  • subspace clustering (generalized principal component analysis), and

  • nonlinear and parameter-varying system identification.

The maximum likelihood estimation principle applied to these nonlinear models leads to nonconvex optimization problems and yields inconsistent estimators in the errors-in-variables (measurement errors) setting. We propose a computationally cheap and statistically consistent estimator based on a bias correction procedure, called Adjusted Least-Squares Estimation. The method is successfully used for conic section fitting and was recently generalized to algebraic curve fitting. The contribution of this book’s chapter is the application of the polynomially structured low-rank approximation problem and, in particular, the adjusted least-squares method to subspace clustering, nonlinear and parameter-varying system identification. The classical in system identification input-output notion of a dynamical model is replaced by the behavioral definition of a model as a set, represented by implicit nonlinear difference equations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We use the notation \(d\) for data in problems involving static models and \(w\) for data in problems involving dynamical models.

  2. 2.

    The choice of the monomials is related to the model class selection in system identification.

References

  1. C. Bishop, Pattern Recognition and Machine Learning (Springer, 2006)

    Google Scholar 

  2. F.L. Bookstein, Fitting conic sections to scattered data. Comput. Graphics Image Proc. 9, 59–71 (1979)

    Article  Google Scholar 

  3. C. Cheng, H. Schneeweiss, Polynomial regression with errors in the variables. J. R. Stat. Soc. B 60(1), 189–199 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  4. W. Chojnacki, M. Brooks, A.V.D. Hengel, D. Gawley, From FNS to HEIV: a link between two vision parameter estimation methods. IEEE Trans. Pattern Anal. Mach. Intell. 26, 264–268 (2004)

    Article  Google Scholar 

  5. D. Cox, J. Little, D. O’Shea, IDeals, Varieties, and Algorithms (Springer, 2004)

    Google Scholar 

  6. G. Eckart, G. Young, The approximation of one matrix by another of lower rank. Psychometrika 1, 211–218 (1936)

    Article  MATH  Google Scholar 

  7. A. Fitzgibbon, M. Pilu, R. Fisher, Direct least-squares fitting of ellipses. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 476–480 (1999)

    Article  Google Scholar 

  8. W. Gander, G. Golub, R. Strebel, Fitting of circles and ellipses: Least squares solution. BIT 34, 558–578 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  9. G. Golub, V. Pereyra, Separable nonlinear least squares: the variable projection method and its applications. Inst. Phys. Inverse Prob. 19, 1–26 (2003)

    Article  MathSciNet  Google Scholar 

  10. T. Hastie, W. Stuetzle, Principal curves. J. Am. Stat. Assoc. 84, 502–516 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  11. J. Jackson, A User’s Guide to Principal Components (Wiley, 2003)

    Google Scholar 

  12. I. Jolliffe, Principal Component Analysis (Springer, 2002)

    Google Scholar 

  13. K. Kanatani, Statistical bias of conic fitting and renormalization. IEEE Trans. Pattern Anal. Mach. Intell. 16(3), 320–326 (1994)

    Article  MATH  Google Scholar 

  14. A. Kukush, I. Markovsky, S. Van Huffel, Consistent estimation in an implicit quadratic measurement error model. Comput. Statist. Data Anal. 47(1), 123–147 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  15. A. Kukush, S. Zwanzig, On inconsistency of the least squares estimator in nonlinear functional error-in-variables models. Preprint N96–12 (1996), Institut für Mathematische Stochastik, Universität Hamburg

    Google Scholar 

  16. Y. Ma, Y. Fu, Manifold Learning Theory and Applications, (CRC Press, 2011)

    Google Scholar 

  17. I. Markovsky, Structured low-rank approximation and its applications. Automatica 44(4), 891–909 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  18. I. Markovsky, Low Rank Approximation: Algorithms, Implementation (Springer, Applications. Communications and Control Engineering, 2012)

    Book  MATH  Google Scholar 

  19. I. Markovsky, Recent progress on variable projection methods for structured low-rank approximation. Signal Process. 96PB, 406–419 (2014)

    Google Scholar 

  20. I. Markovsky, A. Kukush, S. Van Huffel, Consistent least squares fitting of ellipsoids. Numer. Math. 98(1), 177–194 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  21. B. Matei, P. Meer, Estimation of nonlinear errors-in-variables models for computer vision applications. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1537–1552 (2006)

    Article  Google Scholar 

  22. J. Polderman, J.C. Willems, Introduction to mathematical systems theory (Springer-Verlag, New York, 1998)

    Book  Google Scholar 

  23. B. Schölkopf, A. Smola, K. Müller, in Kernel Principal Component Analysis. (Cambridge, MA, 1999 MIT Press), pp. 327–352

    Google Scholar 

  24. S. Shklyar, A. Kukush, I. Markovsky, S. Van Huffel, On the conic section fitting problem. J. Multivar. Anal. 98, 588–624 (2007)

    Article  MATH  Google Scholar 

  25. E.D. Sontag, Mathematical Control Theory: Deterministic Finite Dimensional Systems (Springer, 1990)

    Google Scholar 

  26. I. Vajk, J. Hetthéssy, Identification of nonlinear errors-in-variables models. Automatica 39(12), 2099–2107 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  27. G. Vandersteen, Identification of linear and nonlinear systems in an errors-in-variables least square. Ph.D. thesis (Vrije Universiteit Brussel 1997), http://wwwtw.vub.ac.be/elec/Papersonweb/Papers/GerdVandersteen/Phd.pdf

  28. R. Vidal, Y. Ma, S. Sastry, Generalized principal component analysis (GPCA). IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1945–1959 (2005)

    Article  Google Scholar 

  29. J.C. Willems, From time series to linear system–Part I. Finite dimensional linear time invariant systems, Part II. Exact modelling, Part III. Approximate modelling. Automatica 22, 23, 561–580, 675–694, 87–115 (1986, 1987)

    Google Scholar 

  30. Z. Zhang, Parameter estimation techniques: A tutorial with application to conic fitting. Image Vision Comp. J. 15(1), 59–76 (1997)

    Article  Google Scholar 

  31. Z. Zhang, H. Zha, Principal manifolds and nonlinear dimension reduction via local tangent space alignment. SIAM J. Sci. Comput. 26, 313–338 (2005)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

Funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007–2013)/ERC Grant agreement number 258581 “Structured low-rank approximation: Theory, algorithms, and applications” is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department ELEC, Vrije Universiteit Brussel, Pleinlaan 2, Building K, B-1050, Brussels, Belgium

    Ivan Markovsky & Konstantin Usevich

Authors
  1. Ivan Markovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantin Usevich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ivan Markovsky .

Editor information

Editors and Affiliations

  1. Northeastern University, Boston, Massachusetts, USA

    Yun Fu

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Markovsky, I., Usevich, K. (2014). Nonlinearly Structured Low-Rank Approximation. In: Fu, Y. (eds) Low-Rank and Sparse Modeling for Visual Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-12000-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12000-3_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-11999-1

  • Online ISBN: 978-3-319-12000-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation