On Convexification of System Identification Criteria
- 47 Downloads
System Identification is about estimating models of dynamical systems from measured input-output data. Its traditional foundation is basic statistical techniques, such as maximum likelihood estimation and asymptotic analysis of bias and variance and the like. Maximum likelihood estimation relies on minimization of criterion functions that typically are non-convex, and may cause numerical search problems and estimates trapped in local minima. Recent interest in identification algorithms has focused on techniques that are centered around convex formulations. This is partly the result of developments in semidefinite programming, machine learning and statistical learning theory. The development concerns issues of regular-ization for sparsity and for better tuned bias/variance trade-offs. It also involves the use of subspace methods as well as nuclear norms as proxies to rank constraints. A special approach is to look for difference-of-convex programming (DCP) formulations, in case a pure convex criterion is not found. Other techniques are based on Lagrangian relaxation and contraction theory. A quite different route to convexity is to use algebraic techniques to manipulate the model parameterizations. This article will illustrate all this recent development.
Keywordsmaximimum likelihood regularization convexification
Unable to display preview. Download preview PDF.
- 9.Manchester, I.R., Tobenkin, M.M., and Megretski, A., Stable Nonliner System Identifiction: Convexity, Model Class and Consistency, Proc. 16 IFAC Sympos. Syst. Identificat., Brussels, Belgium, July 2012, pp. 328–333.Google Scholar
- 18.Carli, F.P., Chiuso, A., and Pillonetto, C., Efficient Algorithms for Large Scale Linear System Identification Using Stable Spline Estimators, Proc. 16 IFAC Sympos. Syst. Identificat (SYSID 2012), 2012, pp. 119–124.Google Scholar
- 19.Ljung, L., The System Identification Toolbox: The Manual, Natick: The MathWorks Inc., USA, 1986, 1st ed., 2012, 8th ed.Google Scholar
- 20.Chen, T., Ljung, L., Andersen, M., Chiuso, A., Carli, P.F., and Pillonetto, C., Sparse Multiple Kernels for Impulse Response Estimation with Majorization Minimization Algorithms, IEEE Conf. Decision and Control, Hawaii, 2012, pp. 1500–1505.Google Scholar
- 27.Parrilo, P. and Ljung, L., Initialization of Physical Parameter Estimates, in Proc. 13 IFAC Sympos. Syst. Identificat., van der Hof, P., Wahlberg, B., and Weiland, S., Eds., Rotterdam, The Netherlands, 2003, pp. 1524–1529.Google Scholar
- 28.Xie, L.L. and Ljung, L., Estimate Physical Parameters by Black Box Modeling, Proc. 21 Chinese Control Conf, Hangzhou, China, 2002, pp. 673–677.Google Scholar
- 30.Yu, C., Ljung, L., and Verhaegen, M., Gray Box Identification Using Difference of Convex Programming, Proc. IFAC World Congress, Toulouse, France, 2017.Google Scholar
- 31.Wills, A., Yu, C., Ljung, L., and Verhaegen, M., Affinely Parametrized State-Space Models: Ways to Maximize the Likelihood Function, Proc. IFAC Sympos. Syst. Identificat. SYSID18, Stockholm, July 2018.Google Scholar
- 33.Glad, S.T., Implementing Ritt's Algorithm of Differential Algebra, IFAC Sympos. Control Systems Design, NOLCOS'92, Bordeaux, France, 1992, pp. 398–401.Google Scholar
- 34.Ljung, L. and Chen, T., Convexity Issues in System Identification, Proc. 10 IEEE Int. Conf. Control Automat. (ICCA), Hangzhou, China, 2013.Google Scholar
- 35.Ljung, L., Pespectives on System Identification, IFAC Annual Reviews, Spring Issue, 2010.Google Scholar