Keywords

1 Introduction

Cyber Physical Systems (CPS) are prone to various faults caused by failing actuators, sensors, or structural components. Modern systems are becoming increasingly larger and complex, making manual fault handling costly and time-consuming. To enable systems to adapt autonomously to faults, fault-tolerant control (FTC) is necessary. For severe faults, such as the loss of a complete actuator, a new control law is required. This process is called reconfiguration and has the goal of identifying a new valid control law that can restore operation [1].

Due to the immense cost reduction and miniaturization of embedded controllers and sensors in the last decade, data and computational resources are widely available. This allows for computationally expensive algorithms to be incorporated in CPS of arbitrary size and led to the development and implementation of advanced model-based fault-tolerant control architectures, such as explicit model following (EMF), Linear Virtual Actuators (LVA) or Fault-tolerant Model Predictive Control (MPC) approaches [2].

However, all these model-based FTC strategies rely on accurate dynamic models to reconfigure the controller that is handling the faulty system. Generally speaking, the impact of various faults on the system dynamics are not known a-priori. Thus, when developing a comprehensive fault handling strategy, a model of the dynamics of the faulty system needs to be identified online. This requires the algorithm for model identification to be executed online and to rapidly convergence (low data requirement) to allow for a quick reconfiguration. Additionally, a high robustness against noise is desirable to be applicable in practice. In the last years, the well-established research field of system identification has received renewed attention with the rise of data-driven modeling, advanced machine learning algorithms and new findings in the area of compressed sensing [3]. In this paper we strive to present a model-based fault handling strategy for a classical control system, the Inverted Pendulum on a Cart (IPoC) with threefold actuator redundancy. The required dynamic model of the faulty system is identified online through the sparse regression algorithm SINDYc [4]. The identified model of the faulty system is then used for a model-matching reconfiguration algorithm, the Pseudo-Inverse Method [5, p. 391] to reconfigure a full-state feedback (FSF) controller. In this work, we will limit our online identification to linear models, although SINDYc readily extends to nonlinear models. Further, we assume full observability of the system. The continuous plant model, the discrete controller and reconfiguration are implemented in MATLAB/Simulink. In the following we aim to address the following research questions:

  1. 1.

    Are sparse regression algorithms, such as SINDYc, suited to identify models of closed-loop CPS online?

  2. 2.

    What perturbation strategy ensures continuous and adequate identification of the faulty system dynamics?

The paper is structured as follows: After a section on the related work 2, we will first describe the IPoC system followed by a brief overview of the SINDYc algorithm. Section 5 describes the control reconfiguration methodology and architecture. The results will be discussed in Sect. 6 and followed by a discussion and conclusion.

2 Related Work

2.1 Model-Based Fault Tolerant Control

To achieve fault tolerance, technical systems are typically augmented with a Fault Detection, Identification and Recovery (FDIR) scheme. If the fault is severe (e.g. actuator breakdown) and requires a redesign of the control-loop, control reconfiguration is necessary. Otherwise, a control accommodation approach is sufficient by means of robust or adaptive control. Blanke et al. [5] have described multiple approaches to model-based control reconfiguration [5]. Generally, the model-matching design is prevailing in the literature, where the controller of the faulty plant is reconfigured, so that the closed-loop characteristics of the system are consistent to the non-faulty behavior. Beside the Pseudo-Inverse approach presented in this work, the Markov Parameter approach or Linear Virtual Actuators (LVA) approach are alternative model-based reconfiguration methods [5]. In the aerospace sector, a historically strong field of applied FTC, many control architectures and approaches have been developed, as exemplified in [6]. Lombaerts et al., for example, applied a the model reference adaptive control approach to a flight control [7].

Beside the classical FTC approaches originating from control theory, another research field has evolved, mainly dealing with hybrid systems and employing Formal Methods to find a valid configuration [8]. For example, the AutoConf algorithm by Balzereit et al. transfers the reconfiguration problem into a logical formula, which then can be solved using Satifiability Theory (SAT) [9]. Despite the many different approaches to model-based control reconfiguration, the need of an up-to-date system model is ubiquitous, since a system subject to faults is inherently time-variant and faults are not known beforehand.

2.2 Online, Closed-Loop System Identification

Closed-loop online system identification has been of interested in many areas. Ljung renders a comprehensive view on system identification and specifically covers the problem of identifiability of closed-loop systems [10, Sect. 13.4]. Leveraging advances in sparse sensing and machine learning, Brunton, Proctor and Kutz have developed the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm [11]. The algorithm employs the thresholded sequencial least-squares regression algorithm, resulting in models that are sparse and thus balance model complexity with descriptive ability. The algorithm then has been further developed to include forced nonlinear systems by extending the function base to include control inputs [4]. The resulting SINDYc formulation has been applied to numerous problems, including a Model Predictive Control (MPC) in the low-data limit by Kaiser, Kutz and Brunton [12]. However, to the best of our knowledge, data-driven sparse regression system identification techniques, such as SINDYc, have not yet been employed explicitly in the context of online closed-loop Control Reconfiguration specifically.

3 System Description and Modeling

Figure 1 shows the redundantly actuated IPoC system, which will constitute the use-case in this study.

Fig. 1
A diagram of a redundantly actuated I P O C system. The pendulum angle is denoted as theta, the length is l, and the mass is lowercase m. The mass of the cart is uppercase M. The states of the wheels are denoted as u 1, u 2, and u 3.

Redundantly actuated Inverted Pendulum on a Cart (IPoC)

Full size image

Mathematically, the IPoC is described by the following differential equations [3]:

$$\begin{aligned} \begin{aligned} \dot{x} = \; & v \\ \dot{v} = \; & \frac{-m^2 \, l^2 \, g \, cos(\theta ) \, sin(\theta ) + m \, l^2 \, (m \, l \, \omega ^2 \, sin(\theta ) - \delta \, v) + m \, l^2 \, (1/D) \, F_t}{m \, l^2 \, (M+m \, (1-cos(\theta )^2))} \\ \dot{\theta } = \; & \omega \\ \dot{\omega } = \; & \frac{(m+M) \, m \, g \, l \, sin(\theta ) - m \, l \, cos(\theta ) \, (m \, l \, \omega ^2 \, sin(\theta ) - \delta \, v) - m \, l \, cos(\theta ) \, F_t}{ m \, l^2 \, (M+m \, (1-cos(\theta )^2))} \end{aligned} \end{aligned}$$
(1)

where x is the cart position, v is the velocity, \(\theta \) is the pendulum angle, \(\omega \) is the angular velocity. The parameters describe pendulum mass m, the cart mass M, pendulum arm length l, gravitational acceleration g, and the velocity-proportional (Stoke’s law) friction damping \(\delta \) on the cart. The system control input \(F_t = \sum _{i=1} F_i\) is compound force applied to the cart and is a linear combination of the three redundant actuators. The actuators are modeled as ideal linear force actuators, not introducing any additional dynamics into the plant. The system is implemented as a nonlinear system in MATLAB/Simulink.

4 Closed-Loop System Identification with SINDYc

Sparse Identification of Nonlinear Dynamics with control (SINDYc) extends the original SINDy algorithm to include external inputs and feedback control [4].

4.1 Sparse Identification—SINDYc

In this section, a brief overview of the sparse regression technique used in SINDYc is presented. Given a dynamical nonlinear system of the form

$$\begin{aligned} \frac{d}{dt} \textbf{x}(t) = \textbf{f} (\textbf{x}(t), \textbf{u}(t)) \end{aligned}$$
(2)

where \(\textbf{x}(t)\) denotes the state of a system at time t, and the function \(\textbf{f} (\textbf{x}, \textbf{u})\) describes the dynamic constraints. The time-dependency is omitted for better readability.

Then, to identify the system from data, we first collect m measurements of the state history \(\textbf{X} \in \mathbb {R}^{m \times n}\), corresponding to n state-vector measurements \( \textbf{x}_i \in \mathbb {R}^m\) and the input history \(\boldsymbol{\Upsilon } \in \mathbb {R}^{l \times m}\).

(3)

These measurement matrices can be expanded with nonlinear candidate functions f into a library matrix \(\boldsymbol{\Theta }(\textbf{X}, \boldsymbol{\Upsilon })\),

(4)

which in turn constitutes the function basis to regress upon, so that the regression problem is defined as

$$\begin{aligned} \dot{\textbf{X}} = \mathbf {\Xi } \boldsymbol{\Theta }^T ( \textbf{X}) \end{aligned}$$
(5)

where \(\Xi \) denotes the sparse coefficient matrix to be determined, and \(\dot{\textbf{X}}\) describes derivative of the state history matrix with respect to time, which can be measured or computed. The resulting over-determined system can be solved efficiently with the thresholded sequential least-squares algorithm, a relaxation of the Sparse Regularized Regression Problem, as described by Zheng et al. [13].

4.2 Identifiability in Closed-Loop Systems

As shown by Ljung, for a well-defined closed-loop system that exhibits integrative behavior (contains a delay) and is stable, the convergence theorem generally holds under the additional assumption that the data is informative and the model set contains the true system [10, p. 430].

Therefore it is possible to identify a system under closed-loop feedback control from data. However, it is more challenging to obtain informative data, since an important purpose of feedback is to make the closed-loop system less sensitive to disturbances.

As indicated by Brunton et al. SINDYc can be applied to closed-loop systems, but requires an additional input perturbation [4]. The key issue is that for closed-loop systems, where \(\textbf{u} = \textbf{K}(\textbf{x})\), with \(\textbf{K}\) being the feedback gain matrix, the regression problem becomes ill-conditioned and the coefficients matrix \(\Xi \) thus rank-deficient. This is because it is impossible to disambiguate the influence of the feedback control and the internal dynamics. The single requirement for obtaining informative data from closed-loop systems is that the input must be be persistently exciting of a certain order, i.e. that it contains sufficiently many distinct frequencies [10, p. 415]. In the case of online system identification, the perturbation strategy is a tradeoff between optimal identifiability (high input power) vs. optimal reference tracking (smallest possible input power).

5 Control Reconfiguration

The control reconfiguration architecture is detailed in Fig. 2. Generally, a dynamic system can experience faults in three locations: actuator, plant, and sensor faults. Only actuator faults are considered in this work. To reconfigure the controller in the case of a fault, the online identification layer first estimates a new system model of the faulty system \(\dot{\textbf{x}} = \textbf{A}_f \textbf{x} + \textbf{B}_f \textbf{u}\) based on recent system behavior, described by the input \(\textbf{u}(t)\) and states \(\textbf{x}(t,\textbf{u}, f)\) time series. To disambiguate between the full state feedback controller dynamics and the plant dynamics the perturbation injector excites the system directly and continuously. The new dynamic model is then served to the control reconfiguration layer. Depending on the severity of the fault, the reconfiguration algorithm reallocates resources by altering the full state feedback controller matrix \(K_f\).

Fig. 2
A block diagram depicts the architecture of online control reconfiguration. It indicates the flow of process going through actuators, plant, sensors, online system identification, control reconfiguration, feedback controller, and perturbation injector.

Online Control Reconfiguration architecture for a linear full state feedback controller

Full size image

For simplicity, a model-matching reconfiguration strategy is chosen, based on the Pseudo-Inverse method detailed in [5, pp. 391–394]. Given a linear state-space model of a nonlinear dynamic system unsubject and subject (index f) to a fault

$$\begin{aligned} \dot{\textbf{x}}(t) = \textbf{A} \; \textbf{x}(t) + \textbf{B}\; \textbf{u}(t), \; \; \dot{\textbf{x}}(t) = \textbf{A}_f \; \textbf{x}(t) + \textbf{B}_f \; \textbf{u}(t) \end{aligned}$$
(6)

a state feedback controller for the faulty plant can be readily expressed. The optimal gains are determined by solving the Riccati Equation for a Linear Quadratic Regulator (LQR) [3]

$$\begin{aligned} \textbf{u} (t)= - \textbf{K}_f \; \textbf{x}(t), \end{aligned}$$
(7)

where \(\textbf{K}_f\) denotes the control matrix for the faulty plant. To achieve similar closed-loop dynamics, the difference between the two system models needs to be minimized. The solution to the unknown control matrix \(\textbf{K}_f\) is then given by:

$$\begin{aligned} \textbf{K}_f = \textbf{B}^+_f (\textbf{A}_f - \textbf{A} + \mathbf {B \;K})f \end{aligned}$$
(8)

with the Pseudo-Inverse \(\textbf{B}^+_f\) of the input matrix. The resulting control matrix minimizes the differences of dynamical properties between the nominal loop and the faulty loop and gives the best possible controller design for the linear system subject to a fault.

6 Results

This section presents first results of a parameter study for the identification of the IPoC system with SINDYc, yet without reconfiguration. The second section then presents the results of the complete reconfiguration of IPoC system.

6.1 Closed-Loop Identification Parameter Study

There are a variety of parameters that are relevant to the identification process of closed-loop systems. In this section we will study effect of the perturbation signal power and the signal-to-noise ratio (SNR) for the measured sensor signals used for identification. There exists a number of different signal types like White Gaussian Noise (WGN), Pseudo-Random Binary Sequence (PRBS) or multisine signals.

However, we found that, for our purpose, the perturbation signal type, after being band-pass filtered, has only marginal influence on the identification results. To reduce the parameter space we will therefore only consider the bandpass-filtered WGN signal. To ensure comparability for different frequency bands, the power of the perturbation signal \(P(u_p)\) is normalized. Figure 3 shows the unfiltered and filtered WGN perturbation signal and a frequency band of \(f_b = 2 .. 15 \, \text {Hz}\).

Fig. 3
Four line graphs represent the plots of the White Gaussian Node, power spectral density of W G N, W G N signal with band speed between 2 and 15 hertz, and P S D of band speed W G N. The graphs represent the trends of power with respect to time and frequency.

Filtered White Gaussian Noise (WGN) as perturbation signal

Full size image

For the parameter study the perturbation signal power was varied from \(P(u_p) = 0.01 .. 0.63 \; \text {W} = 10 .. 28 \; \text {dBm}\) and a signal-to-noise ratio from \(SNR_{id} = 100\) to 10 was studied.

For all experiments, a sinusoidal reference signal with frequency \(f_r = 0.1 \text {Hz}\) is imposed on the system.

Figure  4 shows the simulation and the resulting identification plots for a signal power of \(P(u_p) = 14.5 \; \text {dBm}\) and a SNR of 100. The states, derivatives and input plots on the top show the closed-loop system response to the reference signal. On the bottom left, the identified column sum \(b_i\) of the input matrix (pertaining to one of the three actuators each) is plotted for progressive identification timeframes \(n_{id}\) corresponding to \(N_tf = 300\) samples at a sampling frequency of \(f_s = 500\,\text {Hz}\) each. The identified open loop eigenvalues \(\lambda _i\) are plotted in the bottom center plot, and the closed-loop eigenvalues \(\lambda _{i,cl}\) on the right.

The input matrix corresponding to the actuator effectiveness is identified correctly throughout the experiment and exhibits low variance. The system matrix is not identified correctly and exhibits high variance for the first few identification timeframes. This is probably due to settling effects, i.e. that the systems initial conditions are equal to the “pendulum-up” fixpoint, but the reference signal demands a constant velocity initially, resulting in large gradients. For later timeframes, however, the system is identified correctly.

Fig. 4
Six line graphs are labeled states, state derivatives, and perturbed input in the top row, and input matrix column sums, output loo Eigenvalues, and closed-loop Eigenvalues in the bottom row. The x-axis in the top row denotes time, and in the bottom row, it denotes n i d.

Parameter study case for IPoC system identification

Full size image

The semilog-plot in Fig. 5 shows the statistical evaluation of the identification results for varying perturbation signal power and Signal-to-Noise ratios for a perturbation frequency band of \(f_b = 2 .. 15\, \text {Hz}\). The different signal power levels are shown by the different colors. The shaded areas correspond to the variance and the solid line to the mean of all \(n_{id} = 32\) identification timeframes of one parameter set.

The left plot shows the identification results of the only the first actuator to reduce clutter. For low perturbation signal powers, a significant increase in variance can be seen for a SNR of 30 and below. For higher signal power the identification remains at low variance even for very low SNR. It is important to note, that the algorithm overestimates the effectiveness consistently. A similar behavior is observed for the open-loop and closed-loop identification of the system matrix. For low perturbation signal powers, the eigenvalues are poorly identified for a SNR of 40 and below. For higher perturbation signal powers, the eigenvalues are identified with less variance. Here, a clear tendency towards overestimating the system dynamics is observed both for lower SNR as well as lower perturbation signal powers.

Fig. 5
Three area graphs represent the trends of 10, 14.50, 19.00, 23.50, and 28.00 d b m. 1. It represents the plots of b 1 versus S N R i d. 2. It depicts R e gamma i versus S N R i d. 3. It denotes the relationship between R e gamma i d and S N R i d.

Statistical evaluation of identification quality for varying perturbation signal power and Signal-to-Noise ratio (SNR) for a perturbation frequency band of \(f_b = 2 .. 15\, \textrm{Hz}\)

Full size image

Further experiments have shown that there is an optimal frequency band of the perturbation signal. Both very low and very high frequencies lead to identification results with significantly higher variance.

6.2 Closed-Loop Identification and Control Reconfiguration

In this section we will simulate the complete the identification and reconfiguration strategy for the closed-loop IPoC system subject to both abrupt and incipient actuator faults.

Figure 6 shows the simulated system response of the the closed loop system. The top three graphs show the states, the derivatives and the forces on the cart, which are produced by the actuators subject to faults. At time \(t = 5 \, \text {s}\) an abrupt fault is introduced to actuator 3, and at time \(t = 10 \, \text {s}\) an incipient fault is introduced to actuator 2, linearly loosing effectiveness over \(\Delta t = 10 \, \text {s}\).

The bottom three graphs show the online identification and reconfiguration results. The input matrix is identified, as expected, with low variance. Both the abrupt and incipient actuator faults are caputured well. The system matrix (bottom center graph) is identified with high variance, and the corresponding eigenvalues only settle after the faults have transpired. The bottom left graph shows the row sums of the reconfigured full state feedback controller matrix \(\textbf{K}_f\). The reconfiguration of the relevant entries is clearly seen, as the controller is adjusted to changing plant behavior. Despite the full failure of two actuators, the closed-loop system remains stable and retains it’s pre-fault characteristics.

Fig. 6
Six line graphs are labeled states, state derivatives, and actuator force in the top row, and column sums of input matrix B, open loop Eigenvalues, and row sums feedback matrix K in the bottom row. The x-axis in the top row denotes time, and in the bottom row, it denotes identification timeframe.

Perturbed IPoC simulation data with fault injection and reconfiguration, \(P(u_p) = 20 \, \text {dBm}\), \(SNR = 80\)

Full size image

7 Limitations and Outlook

The presented approach for fault-handling by control reconfiguration via online identification with SINDYc has shown general applicability to closed-loop cyber-physical systems, such as the Inverted Pendulum on a Cart. In order to successfully identify a closed-loop system, the perturbation signal needs to be chosen carefully. It has been shown that the higher the perturbation power and signal-to-noise ratio, the better the performance of SINDYc. What remains unclear why both the input matrix coefficients and the stability of the system (more negative eigenvalues eigenvalues) are overestimated consistently. Also, what has not been shown sufficiently is the dependency upon the perturbation frequency-band. Failures in plant dynamics (e.g. changing pendulum mass) have also not been considered. However, the presented fault handling strategy readily integrates these types of faults as well.

The presented approach is limited in several ways: First of all, the requirement of full observability strongly limits the applicability for many real-world systems. On this end, a combination with a estimator-filters (e.g. Kalman filter) would relax this requirement. This would also allow for treatment of sensor faults. Second, the requirement of consistent excitation compromises tracking capabilities of the system and poses increased strain on the actuators. Future implementations could make use of external anomaly detection algorithms to trigger a system “self-test” consisting of a perturbation, identification and possible reconfiguration. Thirdly, the limited identifiability of the system matrix \(\textbf{A}\) restricts the application for weakly actuated systems.

Future research could focus on the implementation of this approach on a real-world system with real, noisy data. The influence of actuator dynamics or explicit time-delays on the identifiabiliy might be worthwile to study. Further, the identification of the optimal perturbation frequency-band, or alternative perturbation strategies, like a pulsed and synchronized perturbation and identification approach, would present an interesting research opportunity. Finally, a combination of the presented approach with input linearization techniques, such as Incremental Nonlinear Dynamic Inversion (INDI), where only the input matrix is required, offers a very promising nonlinear fault-tolerant control strategy.