Skip to main content
SpringerLink
Log in

Variable selection in identification of a high dimensional nonlinear non-parametric system

  • Published:
Control Theory and Technology Aims and scope Submit manuscript

Abstract

The problem of variable selection in system identification of a high dimensional nonlinear non-parametric system is described. The inherent difficulty, the curse of dimensionality, is introduced. Then its connections to various topics and research areas are briefly discussed, including order determination, pattern recognition, data mining, machine learning, statistical regression and manifold embedding. Finally, some results of variable selection in system identification in the recent literature are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. K. Li, J. Peng. Neuro input selection — a fast model based approach. Neurocomputing, 2007, 70(4/6): 762–769.

    Article  Google Scholar 

  2. K. Li, J. Peng, E. W. Bai. A two-stage algorithm for identification of nonlinear dynamic systems. Automatica, 2006, 42(7): 1189–1197.

    Article  MATH  MathSciNet  Google Scholar 

  3. J. Peng, S. Ferguson, K. Rafferty, et al. An efficient feature selection method for mobile devices with application to activity recognition. Neurocomputing, 2011, 74(17): 3543–3552.

    Article  Google Scholar 

  4. I. N. Lobato, J. Nankervis, N. Savin. Testing for zero autocorrelation in the presence of statistical dependence. Econometric Theory, 2002, 18(3): 730–743.

    Article  MATH  MathSciNet  Google Scholar 

  5. T. Söderström, P. Stoica. System Identification. New York: Prentice Hall, 1989.

    MATH  Google Scholar 

  6. C. Velasco, I. Lobato. A simple and general test for white noise. Proceedings of Econometric Society 2004 Latin American Meetings, Santiago, Chile, 2004: 112–113.

    Google Scholar 

  7. S. Su, F. Yang. On the dynamical modeling with neural fuzzy networks. IEEE Transactions on Neural Network, 2002, 13(6): 1548–1553.

    Article  MathSciNet  Google Scholar 

  8. J. D. Bomerger, D. E. Seborg. Determination of model order for NARX models directly from input-output data. Journal of Process Control, 1998, 8(5/6): 459–568.

    Article  Google Scholar 

  9. W. Zhao, H.-F. Chen, E. W. Bai, et al. Kernel-based local order estimation of nonlinear non-parametric systems. Automatica, 2015, 51(1): 243–254.

    Article  MathSciNet  Google Scholar 

  10. G. Pillonetto, M. Quang, A. Chiuso. A new kernel-based approach for nonlinear system identification. IEEE Transactions on Automatic Control, 2011, 56(12): 2825–2840.

    Article  MathSciNet  Google Scholar 

  11. X. Hong, R. T. Mitchell, S. Chen, et al. Model selection approaches for nonlinear system identification: a review. International Journal of Systems Science, 2008, 39(10): 925–949.

    Article  MATH  MathSciNet  Google Scholar 

  12. X. He, H. Asada. A new method for identifying orders of input-output models for nonlinear dynamic systems. Proceedings of the American Control Conference, San Francisco, CA: IEEE, 2003: 2520 - 2523.

  13. M. B. Kennel, M. R. Brown, H. Abarbanel. Determining embedding dimension for phase-space reconstruction using geometrical construction. Physical Review A, 1992, 45(6): 3403–3411.

    Article  Google Scholar 

  14. L. Cao. Practical method for determining the minimum embedding dimension input-output models for nonlinear dynamic systems. Physica D: Nonlinear Phenomena, 1997, 110(1/2): 43–50.

    Article  MATH  Google Scholar 

  15. S. Roweis, L. Saul. Nonlinear dimensionality reduction by local linear embedding. Science, 2000, 290(5500): 2323–2326.

    Article  Google Scholar 

  16. D. Achlioptas. Database friendly random projections: Johnson-Lindenstrass with binary coins. Journal of Computer and System Sciences, 2003, 66(4): 671–687.

    Article  MATH  MathSciNet  Google Scholar 

  17. H. Ohlsson, J. Roll, T. Glad, et al. Using manifold learning for nonlinear system identification. Proceedings of the 7th IFAC Symposium on Nonlinear Control Systems, Pretoria, South Africa: Elsevier, 2007: 170–175.

    Google Scholar 

  18. J. Roll, A. Nazin, L. Ljung. Nonlinear system identification via direct weight optimization. Automatica, 2005, 41(3): 475–490.

    Article  MATH  MathSciNet  Google Scholar 

  19. J. Kocijan, A. Girard, B. Banko, et al. Dynamic systems identification with Gaussian process. Mathematical and Computer Modelling of Dynamical Systems, 2003, 11(4): 411–424.

    Article  MathSciNet  Google Scholar 

  20. K. Knight, W. Fu. Asymptotics for Lasso-type estimators. The Annals of Statistics, 2000, 28(5): 1356–1378.

    Article  MATH  MathSciNet  Google Scholar 

  21. H. Zou. The adaptive Lasso and its oracle properties. Journal of American Statistical Association, 2006, 101(476): 1418–1429.

    Article  MATH  Google Scholar 

  22. A. Arribas-Gil, K. Bertin, C. Meza, et al. LASSO-type estimators for semiparametric nonlinear mixed-effects model estimation. Statistics and Computing, 2013, 24(3): 443–460.

    Article  MathSciNet  Google Scholar 

  23. E. W. Bai, K. Li, W. Zhao, et al. Kernel based approaches to local nonlinear non-parametric variable selection. Automatica, 2014, 50(1): 100–113.

    Article  MATH  MathSciNet  Google Scholar 

  24. E. W. Bai. Non-parametric nonlinear system identification: an asymptotic minimum mean squared error estimator. IEEE Transactions on Automatic Control, 2010, 55(7): 1615–1626.

    Article  Google Scholar 

  25. K. Mao, S. A. Billings. Variable selection in nonlinear system modeling. Mechanical Systems and Signal Processing, 2006, 13(2): 351–366.

    Article  Google Scholar 

  26. R. Mosci, L. Rosasco, M. Santoro, et al. Is there sparsity beyond additive models? Proceedings of the 16th IFAC Symposium on Systems Identification, Brussels, Belgium: Elsevier, 2012: 971–976.

    Google Scholar 

  27. E. W. Bai, Y. Liu. Recursive direct weight optimization in nonlinear system identification: a minimal probability approach. IEEE Transactions on Automatic Control, 2007, 52(7): 1218–1231.

    Article  MathSciNet  Google Scholar 

  28. W. Zhao, H.-F. Chen, E. W. Bai, et al. Variable selection for NARX systems in first approximation. 2015: http://user.engineering.uiowa.edu/erwei/VariableSelection.pdf.

    Google Scholar 

  29. W. Zhao, W. Zheng, E. W. Bai. A recursive local linear estimator for identification of nonlinear ARX systems: asymptotical convergence and applications. IEEE Transactions on Automatic Control, 2013, 58(12): 3054–3069.

    Article  MathSciNet  Google Scholar 

  30. P. Peduzzi. A stepwise variable selection procedure for nonlinear regression methods. Biometrics, 1980, 36(3): 510–516.

    Article  Google Scholar 

  31. G. E. P. Box, D. Pierce. Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. Journal of American Statistical Association, 1970, 65(332): 1509–1526.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa, 52242, USA

    Er-Wei Bai

  2. School of Electronics, Electrical Engineering and Computer Science, Queen’s University, Belfast, UK

    Er-Wei Bai

  3. Key Laboratory of Systems and Control, Academy of Mathematics and Systems Science, National Center for Mathematics and Interdisciplinary Sciences, Chinese Academy of Sciences, Beijing, 100190, China

    Wenxiao Zhao

  4. School of Computing, Engineering and Mathematics, University of Western Sydney, Penrith, NSW, 2751, Australia

    Weixing Zheng

Authors
  1. Er-Wei Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenxiao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weixing Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Er-Wei Bai.

Additional information

This work was partially supported by the National Science Foundation (No. CNS-1239509), the National Key Basic Research Program of China (973 program) (No. 2014CB845301), the National Natural Science Foundation of China (Nos. 61104052, 61273193, 61227902, 61134013), and the Australian Research Council (No. DP120104986).

Er-Wei BAI was educated in Fudan University, Shanghai Jiaotong University, both in Shanghai, China, and the University of California at Berkeley. Dr. Bai is Professor and Chair of Electrical and Computer Engineering Department, and Professor of Radiology at the University of Iowa where he teaches and conducts research in identification, control, signal processing and their applications in engineering and life science. He holds the rank of World Class Research Chair Professor, Queen’s University, Belfast, U.K. Dr. Bai is an IEEE Fellow and a recipient of the President’s Award for Teaching Excellence and the Board of Regents Award for Faculty Excellence.

Wenxiao ZHAO earned his B.Sc. degree from the Department of Mathematics, Shandong University, China in 2003 and a Ph.D. degree from the Institute of Systems Science, AMSS, the Chinese Academy of Sciences (CAS) in 2008. After this he was a postdoctoral student at the Department of Automation, Tsinghua University. During this period he visited the University of Western Sydney, Australia. Dr. Zhao then joined the Institute of Systems of Sciences, CAS in 2010. He now is with the Key Laboratory of Systems and Control, CAS as an Associate Professor. His research interests are in system identification, adaptive control, and system biology. He serves as the General Secretary of the IEEE Control Systems Beijing Chapter and an Associate Editor of the Journal of Systems Science and Mathematical Sciences.

Weixing ZHENG received the B.Sc. degree in 1982, the M.Sc. degree in 1984, and the Ph.D. degree in 1989, all from Southeast University, Nanjing, China. He is currently a Professor at University of Western Sydney, Australia. Over the years he has also held various faculty/research/visiting positions at Southeast University, China; Imperial College of Science, Technology and Medicine, U.K.; University of Western Australia; Curtin University of Technology, Australia; Munich University of Technology, Germany; University of Virginia, U.S.A.; and University of California-Davis, U.S.A. Dr. Zheng is a Fellow of IEEE.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bai, EW., Zhao, W. & Zheng, W. Variable selection in identification of a high dimensional nonlinear non-parametric system. Control Theory Technol. 13, 1–16 (2015). https://doi.org/10.1007/s11768-015-5010-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11768-015-5010-9

Keywords

Search

Navigation