Exact Methods for Hybrid System Identification

Abstract

This chapter investigates the possibility of exactly solving three core optimization problems encountered in hybrid system identification: switching linear regression, piecewise affine regression, and bounded-error linear estimation. It formally analyzes their computational complexity, and the main conclusion is that they are $\mathcal {NP}$-hard, meaning that, in general, we cannot hope to compute an exact solution in reasonable time. Then, the chapter focuses on more restrictive settings and shows algorithms based on combinatorial enumerations or branch-and-bound methods that can be guaranteed to yield exact (or sufficiently close to exact) solutions in reasonable time for low-dimensional problems, i.e., when the number of regressors is small. Finally, a few numerical results highlight the limitations of such approaches and the need for the heuristics/approximations developed in the next chapters.

Notes

This chapter only provided a brief introduction to computational complexity, mainly sufficient to understand the results dedicated to hybrid system identification. For a full understanding of the technical issues involved, many textbooks are available, among which we recommend [2] for a focus on $\mathcal {NP}$-completeness. Alternatively, an introduction to these concepts with an automatic control perspective is given in [3]. Two major papers set the foundations of this field: [4] identified the first $\mathcal {NP}$-complete problem and [5] established a list of 21 $\mathcal {NP}$-complete problems (including the Partition Problem 5.4), from which reductions are still used to prove the $\mathcal {NP}$-hardness of many problems. Note that models of computation over the reals have also been considered and we refer to [6] for more details on this topic.

The complexity results ($\mathcal {NP}$-hardness and polynomial-time algorithm) for PWA regression were derived in [7], which also formalized the equivalence between linear classifiers and hyperplanes passing through subsets of points. Similar results for switching linear regression and the proof of Theorem 5.1 for the noiseless case are found in [8], while the proof of Theorem 5.2 for the noisy case has not been published elsewhere. The MIN PFS formulation of the general bounded-error problem is due to [9], in which a proof of $\mathcal {NP}$-hardness for MIN PFS can be found. For the greedy approach in which the models are estimated one by one, the $\mathcal {NP}$-hardness of the bounded-error estimation subproblem, i.e., Problem 5.2, is due to [10], while the polynomial-time algorithm is due to [11].

The global optimization methods based on branch-and-bound for switching regression and bounded-error estimation were derived in [1]. Hinging hyperplane models were introduced in [12] and their global optimization set as a MIQP problem as presented in this chapter was proposed in [13]. Other works on the global optimization of switching linear regression models with a mixed-integer programming approach also appeared in the operation research community; see, e.g., [14, 15].