Skip to main content
SpringerLink
Log in

Learning Theory Approach to a System Identification Problem Involving Atomic Norm

  • Published:
Journal of Fourier Analysis and Applications Aims and scope Submit manuscript

Abstract

This paper aims at proposing a learning theory approach to the topic of estimating transfer functions in system identification. A frequency domain identification problem is formulated as an atomic norm regularization scheme in a random design framework of learning theory. Such a formulation makes it possible to obtain sparsity and provide finite sample estimates for learning the transfer function in a learning theory framework. Error analysis is done for the learning algorithm by applying a local polynomial reproduction formula, concentration inequalities and iteration techniques. The convergence rate obtained here is the best in the literature. It is hoped that the learning theory approach to the frequency domain identification problem would bring new ideas and lead to more interactions among the areas of system identification, learning theory and frequency analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen, T., Ohlsson, H., Ljung, L.: On the estimation of transfer functions, regularizations and Gaussian processes - Revisited. Automatica 48, 1525–1535 (2012)

    Article  MathSciNet  Google Scholar 

  2. Cucker, F., Zhou, D.X.: Learning Theory: An Approximation Theory Viewpoint. Cambridge University Press, Cambridge (2007)

    Book  Google Scholar 

  3. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407–499 (2004)

    Article  MathSciNet  Google Scholar 

  4. Gröchenig, K.: Foundations of Time-Frequency Analysis. Birkhäuser, Boston (2001)

    Book  Google Scholar 

  5. Heckel, R., Bölcskei, H.: Identification of sparse linear operators. IEEE Trans. Inform. Theory 59, 7985–8000 (2013)

    Article  MathSciNet  Google Scholar 

  6. Hu, T., Fan, J., Wu, Q., Zhou, D.X.: Regularization schemes for minimum error entropy principle. Anal. Appl. (2014). doi:10.1142/S0219530514500110

  7. Jetter, K., Stöckler, J., Ward, J.D.: Error estimates for scattered data interpolation on spheres. Math. Comput. 68, 733–747 (1999)

    Article  Google Scholar 

  8. Kailath, T.: Measurements on time-variant communication channels. IEEE Trans. Inform. Theory 8, 229–236 (1962)

    Article  Google Scholar 

  9. Ljung, L.: Perspectives on system identification. Ann. Rev. Control 34, 1–12 (2010)

    Article  Google Scholar 

  10. Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice-Hall, Englewood Cliffs (1999)

    Google Scholar 

  11. Pfander, G.E., Walnut, D.F.: Measurement of time-variant linear channels. IEEE Trans. Inform. Theory 52, 4808–4820 (2006)

    Article  MathSciNet  Google Scholar 

  12. Pillonetto, G., Minh, H.Q., Chiuso, A.: A new kernel-based approach for nonlinear system identification. IEEE Trans. Autom. Control 56, 2825–2840 (2011)

    Article  Google Scholar 

  13. Pillonetto, G., Nicolao, G.D.: A new kernel-based approach for linear system identification. Automatica 46, 81–93 (2010)

    Article  Google Scholar 

  14. Shah, P., Bhaskar, B.N., Tang, G., Recht, B.: Linear system identification via atomic norm regularization. In: Proceedings of the 51st IEEE Conference on Decision and Control, Maui, Hawaii, USA, pp. 6265–6270 (2012)

  15. Shi, L., Feng, Y.L., Zhou, D.X.: Concentration estimates for learning with \(\ell ^1\)-regularizer and data dependent hypothesis spaces. Appl. Comput. Harmon. Anal. 31, 286–302 (2011)

    Article  MathSciNet  Google Scholar 

  16. Smale, S., Zhou, D.X.: Shannon sampling and function reconstruction from point values. Bull. Am. Math. Soc. 41, 279–305 (2004)

    Article  MathSciNet  Google Scholar 

  17. Smale, S., Zhou, D.X.: Learning theory estimates via integral operators and their approximations. Constr. Approx. 26, 153–172 (2007)

    Article  MathSciNet  Google Scholar 

  18. Steinwart, I., Scovel, C.: Fast rates for support vector machines using Gaussian kernels. Ann. Stat. 35, 575–607 (2007)

    Article  MathSciNet  Google Scholar 

  19. Sun, H.W., Wu, Q.: Indefinite kernel networks with dependent sampling. Anal. Appl. 11, 1350020 (2013). 15 pages

    Article  MathSciNet  Google Scholar 

  20. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267–288 (1996)

    MathSciNet  Google Scholar 

  21. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    Google Scholar 

  22. Vidyasagar, M., Karandikar, R.L.: A learning theory approach to system identification and stochastic adaptive control. J. Process Control 18, 421–430 (2008)

    Article  Google Scholar 

  23. Wang, H.Y., Xiao, Q.W., Zhou, D.X.: An approximation theory approach to learning with \(\ell ^1\) regularization. J. Approx. Theory 167, 240–258 (2013)

    Article  MathSciNet  Google Scholar 

  24. Wendland, H.: Local polynomial reproduction and moving least squares approximation. IMA J. Numer. Anal. 21, 285–300 (2001)

    Article  MathSciNet  Google Scholar 

  25. Wendland, H.: Scattered Data Approximation. Cambridge Monographs on Applied and Computational Mathematics, vol. 17. Cambridge University Press, Cambridge (2005)

    Google Scholar 

  26. Wu, Q., Ying, Y., Zhou, D.X.: Learning rates of least-square regularized regression. Found. Comput. Math. 6, 171–192 (2006)

    Article  MathSciNet  Google Scholar 

  27. Wu, Q., Zhou, D.X.: Learning with sample dependent hypothesis spaces. Comput. Math. Appl. 56, 2896–2907 (2008)

    Article  MathSciNet  Google Scholar 

  28. Xiao, Q.W., Zhou, D.X.: Learning by nonsymmetric kernels with data dependent spaces and \(\ell ^1\)-regularizer. Taiwan. J. Math. 14, 1821–1836 (2010)

    MathSciNet  Google Scholar 

  29. Zhao, P., Yu, B.: On model selection consistency of Lasso. J. Mach. Learn. Res. 7, 2541–2563 (2006)

    MathSciNet  Google Scholar 

  30. Zhou, D.X.: Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory 49, 1743–1752 (2003)

    Article  MathSciNet  Google Scholar 

  31. Zhou, D.X.: Derivative reproducing properties for kernel methods in learning theory. J. Comput. Appl. Math. 220, 456–463 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The work described in this paper is supported partially by the Research Grants Council of Hong Kong [Project No. CityU 105011] and by National Natural Science Foundation of China under Grants 11371007 and 11461161006.

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Statistics, Hubei University, Wuhan, 430062, China

    Luoqing Li

  2. Department of Mathematics, City University of Hong Kong, Hong Kong, China

    Ding-Xuan Zhou

Authors
  1. Luoqing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ding-Xuan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ding-Xuan Zhou.

Additional information

Communicated by Gitta Kutyniok.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Zhou, DX. Learning Theory Approach to a System Identification Problem Involving Atomic Norm. J Fourier Anal Appl 21, 734–753 (2015). https://doi.org/10.1007/s00041-015-9389-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00041-015-9389-y

Keywords

Mathematics Subject Classification

Search

Navigation