Nonlinear Modeling Problems

Abstract

Applied to nonlinear modeling problem, the maximum-likelihood estimation principle leads to nonconvex optimization problems and yields inconsistent estimators in the errors-in-variables setting. This chapter presents a computationally cheap and statistically consistent estimation method based on a bias correction procedure, called adjusted least squares estimation. The adjusted least squares method is applied to curve fitting (static modeling) and system identification. Section 7.1 presents a general nonlinear data modeling framework. The model class consists of affine varieties with bounded complexity (dimension and degree) and the fitting criteria are algebraic and geometric. Section 7.2 shows that the underlying computational problem is polynomially structured low-rank approximation. In the algebraic fitting method, the approximating matrix is unstructured and the corresponding problem can be solved globally and efficiently. The geometric fitting method aims to solve the polynomially structured low-rank approximation problem, which is nonconvex and has no analytic solution. The equivalence of nonlinear data modeling and low-rank approximation unifies existing curve fitting methods, showing that algebraic fitting is a relaxation of geometric fitting, obtained by removing the structural constraint. Motivated by the fact that the algebraic fitting method is efficient but statistically inconsistent, Sect. 7.3.3 proposes a bias correction procedure. The resulting adjusted least squares method yields a consistent estimator. Simulation results show that it is effective also for small sample sizes. Section 7.4 considers the class, called polynomial time-invariant, of discrete-time, single-input, single-output, nonlinear dynamical systems described by a polynomial difference equation. The identification problem is: (1) find the monomials appearing in the difference equation representation of the system (structure selection), and (2) estimate the coefficients of the equation (parameter estimation). Since the model representation is linear in the parameters, the parameter estimation by minimization of the 2-norm of the equation error leads to unstructured low-rank approximation. However, knowledge of the model structure is required and even with the correct model structure, the method is statistically inconsistent. For the structure selection, we propose to use 1-norm regularization and for the bias correction, we use the adjusted least squares method.

With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.

J. von Neumann