# On an Optimization Problem for a Class of Impulsive Hybrid Systems

- First Online:

- Received:
- Accepted:

DOI: 10.1007/s10626-009-0068-5

- Cite this article as:
- Attia, S.A., Azhmyakov, V. & Raisch, J. Discrete Event Dyn Syst (2010) 20: 215. doi:10.1007/s10626-009-0068-5

- 15 Citations
- 601 Downloads

## Abstract

This contribution addresses the problem of optimal control for a class of hybrid systems, where discrete transitions are accompanied by instantaneous changes in the continuous state variables, and where these changes can be considered as control variables. Based on a variational approach, necessary conditions of optimality are first established. The problem is then cast as a parametric optimization problem for which gradient information is derived. Finally, we discuss assumptions that guarantee convergence of a conceptual algorithm to a stationary solution. A brief discussion on the main implementation issues is also included.

### Keywords

Hybrid systems Optimization Gradient methods State jump Switching Parameter optimization## 1 Introduction

For a long time, discrete-continuous dynamical interactions have been recognized as a major challenge in the process control area. The emergence of a hybrid systems modelling framework has provided a new perspective on some important problems. The ability to operate hybrid systems in an optimal way remains a challenging task. Indeed, for the general setting of hybrid systems, one has to deal not only with the infinite dimensional optimization problems related to the continuous dynamics, but also with a potential combinatorial explosion related to the discrete part.

In this context and with focus on particular classes, many schemes have been proposed to tackle the problem. Some are based on a newly elaborated condition of optimality (see e.g., Shaikh and Caines 2003, 2007; Rantzer 2006), others are more related to semi-classical approaches, (see e.g., Attia et al. 2005; Alamir and Attia 2004; Hedlund and Rantzer 2002; Xu and Antsaklis 2003a). In the last few years, there has been a revived interest in gradient based methods, (see e.g., Lu et al. 1993; Cassandras et al. 1998; Azhmyakov and Raisch 2006; Egerstedt et al. 2006; Axelsson et al. 2006). This fact is due to their intuitive interpretation, reliability and the existence of well established convergence results. It is the aim of this contribution to extend this approach to a particular class of hybrid systems with autonomous switching. This class is characterized by the fact that the discrete transitions are accompanied by instantaneous changes (jumps) in the continuous states and that these state jumps can be considered as control variables. Discrete transitions, or switches, are called autonomous if they cannot be triggered by a discrete control command but depend on the evolution of the continuous states. Hybrid systems with jumps in the continuous states are after referred to as *impulsive hybrid systems*. The problem of interest here is formulated as a sequential problem i.e., for a particular execution the time axis is partitioned into subintervals, in each interval, the dynamics are characterized by a set of ODE’s, with transitions being triggered internally (autonomous switches). This is the approach that has been considered since the initial formulation of the corresponding optimal control problem (Clarke and Vinter 1989) and can be seen as a natural way to tackle the problem. The class of impulsive hybrid systems discussed in this paper arises most frequently in the area of process control and may be a result of adding and processing material, (see e.g., Alamir 2006, for a typical benchmark problem). In particular, we have been motivated by a specific chemical process control application, namely a preferential crystallization process used to separate enantiomers (see Elsner et al. 2005 for the physical aspects and Raisch et al. 2005 for more details on the control problems). This process is often operated in cyclic batch mode, where after each cycle, material to be separated is added. This represents an instantaneous change of the system state. The amount of added material is a degree of freedom and can be considered as an additional control input.

Hybrid systems with autonomous switching and non-controlled state jumps have been considered in Xu and Antsaklis (2003b). The objective is to find the sequence of jump instants such that a cost functional is minimized. For that purpose, the authors develop a second order scheme that is further specialized to linear systems in Xu and Antsaklis (2003c). Recently, Verriest and coworkers (Verriest et al. 2004) have considered another class of hybrid systems. This consists of systems with controlled switching where some delay is present on the states and where both the jump magnitudes and instants of switching are considered as control variables. Based on variational arguments, necessary conditions of optimality are derived and used in a first order scheme, see Verriest et al. (2005) for an epidemic control application. Another contribution by the same authors is the class of hybrid systems introduced in Verriest (2006). The class includes systems with variable state space dimensions where the transitions are time triggered and can thus be viewed as a general class of switched systems. The variation of the the state space dimension is captured by introducing fictitious reset maps. The tools developed can be as well applied to the case where the reset maps are real as exemplified in Mehta et al. (2007) where an optimal control problem is formulated and solved using a variational approach. In this contribution, we consider a related problem where no delay is present on the state and where the switching is autonomous.

The paper is organized as follows: in Section 2 the problem is stated formally. Section 3 is devoted to the statement of the necessary conditions of optimality. In Section 4, the gradient formulas are derived and a conceptual algorithm together with some convergence properties is stated. Finally, some conclusions and suggestions for future work are given in Section 5.

## 2 Problem formulation

We consider the following class of impulsive hybrid systems with autonomous switching (see e.g., Simic et al. 2009, for the general modelling framework).

**Definition 1**

\({\mathcal Q} = \{q_0, q_1 \ldots,q_Q\}\) is a finite set of locations

\({\mathcal E} \subseteq {\mathcal Q}\times {\mathcal Q}\) is a set of edges

\({\mathcal X} = \{{\mathcal X}_q \}_{q \in {\mathcal Q}}\) is a collection of state spaces where, for all \(q \in {\mathcal Q}\), \({\mathcal X}_q\) is an open subset of ℝ

^{n}.\({\mathcal F}=\{f_q\}_{q \in {\mathcal Q}}\) is a collection of vector fields. For all \(q \in {\mathcal Q}\), \(f_q : {\mathcal X}_q \rightarrow \mathbb{R}^n\)

\({\mathcal G} = \{{\mathcal G}_e\}_{e \in {\mathcal E}}\) is a collection of guards. For all possible transitions \(e =(q_i,\ q_j)\in {\mathcal E}\), \({\mathcal G}_e \subset {\mathcal X}_{q_i}\)

\({\mathcal R}=\{{\mathcal R}_e\}_{e \in {\mathcal E}}\) is a collection of reset maps. For all \(e = (q_i,\ q_j) \in {\mathcal E}\), \({\mathcal R}_e : {\mathcal G}_e \rightarrow 2^{{\mathcal X}_{q_j}}\) where \(2^{{\mathcal X}_{q_j}}\) denotes the power set of \({\mathcal X}_{q_j}\)

*f*

_{q}are smooth enough (see assumptions below), and that the sets \({\mathcal G}_e\) are nonempty for all \(e\in {\mathcal E}\). An execution is as follows: starting from an initial condition \((x_0, q_{i_0})\) the continuous state evolves according to the autonomous differential equations

*q*(·) will switch from \(q_{i_0}\) to \(q_{i_1}\),

^{1}at the same time the continuous state gets reset to some value according to the map \({\mathcal R}_{(q_{i_0}, q_{i_1})}\) and the whole process is repeated. Next, we suppose that the guards can be described by smooth (

*n*− 1) dimensional surfaces in ℝ

^{n}

*θ*

_{k}belonging to Θ a compact subset of ℝ

^{n}. In our setup, the variables

*θ*

_{k}are degrees of freedom that can be selected by the controller.

The dynamic optimization problem of interest can now be formulated as follows:

**Problem 1**

*x*(

*t*

_{0}) =

*x*

_{0},

*K*∈ ℤ

^{ + }is the total number of switches, the vector

*θ*denotes the

*n*×(

*K*+ 1) dimensional vector \(\left(\theta_0' \ldots \theta_K'\right)'\),

*L*: ℝ

^{n}→ℝ

^{ + }is a supplied cost function,

*t*

_{0}and

*t*

_{K + 1}=

*T*are both finite and given.

### Remark 1

*not*defined à priori. This follows from the degree of freedom allowed on the state jumps, i.e., variation of these quantities induces variations of the switching instants, see Fig. 1 for an example of an execution within the framework of Problem 1.

- A1
For all

*q*_{i}in \({\mathcal Q}\), \({\mathcal X}_{q_i} = \mathbb{R}^n\) - A2
For all

*q*_{i}in \({\mathcal Q}\), the functions \(f_{q_i}\) are continuously differentiable - A3
*L*is a twice continuously differentiable function - A4
There exists a constant

*M*such that \(\|f_{q_i}(x)\| < M\) for all \(x \in {\mathcal X}_{q_i}\) and \(q_i \in {\mathcal Q}\)

### Remark 2

Since the number of switchings and the sequence of locations are fixed, the problem is well posed. Indeed, sliding and Zeno behaviors are ruled out.

Problem 1 could be seen as a collection of initial value optimization subproblems. Using classical approaches, the subproblems might be solved separately but nothing guarantees that the trajectory obtained by concatenation of the different solutions is the optimal one. The link between the different subproblems is provided by a set of necessary conditions stated in the next section.

## 3 Necessary conditions of optimality

*λ*denotes the adjoint variables. We then have the following result

**Theorem 1**

*If θ*

^{*}

*is an interior optimal solution to Problem*1

*under Assumptions*A1–A4

*and x*

^{*}(

*t*)

*is its corresponding state trajectory for t*∈ [

*t*

_{0},

*T*]

*, then there exists a nontrivial adjoint λ*

^{*}(

*t*)

*and multipliers*\(\pi_k^*\)

*such that the following equations hold*

*At the switching instant t*

_{k + 1},

*the following jump conditions are satisfied*

*for k*= 0, ...

*K*− 1

*, with*

*and*

### Proof

*x*can be written as

*h*is a continuously differentiable function of time. Development of the these terms lead to the following

*dt*

_{k}a small time increment (the existence of which follows from the smoothness assumptions). After integration by part of the second term under the integral sign and rearrangement, Eq. 18 can be written as

*dx*(

*t*

_{k + 1}) is the exact state variation at the instant

*t*

_{k + 1}. Following simple geometrical arguments, it can be approximated to the first order by the following expression

*x*

^{*},

*θ*

^{*}) the following is satisfied

### Remark 3

Instead of affine reset maps, it is straightforward to derive necessary conditions of optimality for the case of a nonlinear parametrized map of the form \(x(t_k^+)=\psi(x(t_k), \theta)\), provided that some smoothness requirements are satisfied. In this case, the gradient of *ψ* would have appeared in the above conditions. On the other hand, the gradient formulas obtained next will be much harder to derive for this case, which will therefore not be pursued further in this contribution.

### Remark 4

The proof techniques used here are of the variational type meaning that the derived necessary conditions of optimality are valid for smooth hybrid impulsive systems under the aforementioned assumptions. The characterized minimum is of the weak type in contrast to a strong minimum that has been recently derived for other classes of hybrid systems (see e.g., Sussmann 1999; Shaikh and Caines 2007).

### Remark 5

## 4 A gradient based approach

The basic idea is to cast Problem 1 into a parameter optimization framework and then to compute, as required by the necessary conditions of optimality, the gradient of the cost functional. Gradient descent techniques can then be used to compute the optimal parameter values. Before the gradient formula is stated in a theorem, a lemma concerning the sensitivity of the states is given.

**Lemma 1**

*The sensitivity*\(\Delta_{q_{i_k}}^{\theta_k} x(\cdot)\)

*of the state trajectory x*(·)

*, corresponding to the dynamics*(5)–(9)

*under Assumptions*A1–A4

*, w.r.t. the vectors θ*

_{k}

*can be computed as a solution to the following variational equation*

*under the following initial conditions*

*with*

### Proof

See Appendix 1.□

Equations 24–27 allow for the computation of the sensitivity trajectories. The first three Eq. 24–26 reflect the sensitivity of the states with respect to the parameters *θ* and are obtained by introducing smooth parametric variations in the initial conditions corresponding to each location. The perturbations are then carefully propagated through the dynamics of the system. Equation 27 is the sensitivity of the switching instants with respect to the parameters *θ*. It is obtained using a first order Taylor series approximation. If the switching instants were fixed or freely controllable, the above conditions would simplify. Specifically, Eq. 27 would be zero.

The result in Lemma 1 is used to establish the following theorem.

**Theorem 2**

*The gradient of the cost functional J corresponding to Problem*1,

*under Assumptions*A1–A4,

*can be computed as follows for k*= 0, ...

*K*− 1

### Proof

See Appendix 2.□

Equation 28 is the gradient of the cost functional with respect to the parameter vector *θ*_{0}. The same type of equation can be obtained if one considers a classical variational problem (in a non-hybrid setting) where only the initial conditions are allowed to vary. Equation 28 can readily be used to study the impact that the initial condition can have on a classical variational problem and can also be used on its own in a gradient algorithm to derive optimal initial conditions. The theory developed here recovers this important result. Equation 29 is the gradient of the cost with respect to parameter *θ*_{k + 1} and is one of the main contributions of the paper. It depends on the different sensitivity components already derived in Lemma 1. The idea behind the proof is again to introduce variations in the parameters and study their impact on the cost. Using the necessary conditions of optimality obtained in Theorem 1, Eqs. 28 and 29 follow. As will be described in more detail in the forthcoming paragraphs, the computational complexity in this type of problems is considerable. Indeed, the difficulty consists in solving a boundary value problem of a special type. We will now state a conceptual algorithm and show that, under some additional assumptions, it converges to the infinimum of the optimization Problem 1 under Assumptions A1–A4.

**Conceptual Algorithm 1**

- 1.
*Choose an admissible parameter vector*\(\theta^{(0)}=\left(\theta_0^{(0)}, \theta_1^{(0)}, \ldots, \theta_K^{(0)}\right)\)*and set l*= 0 - 2.
*Compute the trajectory x*^{(l)}(·)*and the corresponding adjoint λ*^{(l)}(·)*such that the conditions*(12)–(14)*are satisfied,* - 3.
*Update the parameter vector θ*^{(l)}*using the gradient information Eq.*28*and*29*in a gradient projection algorithm, set l*: =*l*+ 1*and go to step 2*

We can now state the following result

**Proposition 1**

*If θ*^{*}* is an accumulation point of the sequence*\(\{\theta^{(l)}\}_l\)* generated by the Conceptual Algorithm *1,* then it is a stationary point i.e.,*\(\nabla J[\theta^*] = 0\).

### Proof

The proof of Proposition 1 follows using descent properties (see e.g., Bertsekas 1995; Pshenichny and Danilin 1982).□

In the following paragraph, some important implementation issues are discussed.

Denote by *P*_{Θ} the projection operator on the set Θ. An implementable version of the preceding Conceptual Algorithm can be stated as follows

**Algorithm 1**

**Step 0***Choose parameters β*,*μ as positive real numbers from the set*(0, 1),*a small positive real number ε and an admissible parameter vector θ*^{(0)}.*Set l*= 0**Step 1***Compute the trajectory x*^{(l)}(·)*by forward integrating the state Eqs.*5–9*under the specified initial condition. Let t*^{(l)}*be the resulting sequence of switching instants*{*t*_{k}}_{k}**Step 2***Compute the sensitivity trajectories using Eq.*24*and the corresponding initial conditions*(25)*and*(26)*and Eq.*27**Step 3***Backward integrate the adjoint λ*^{(l)}(·)*using Eq.*11*with the terminal condition*(14).*At the switching instants t*^{(l)}*compute the multipliers*\(\pi_k^{(l)}\)*in Eq.*12*such that Eq.*13*is enforced. Update the adjoint at the switching instants t*^{(l)}*using Eq.*12*and the so far computed multipliers π*^{(l)}.**Step 4***Compute the gradient using Eqs.*28*and*29.*Update the parameter vector θ*^{(l + 1)}$$ \theta^{(l+1)} = P_\mathit{\Theta}\left(\theta^{(l)} - \gamma^{(l)} \nabla_{\theta} J\right) $$(30)*with*\(\gamma^{(l)}=\mu^{j_l}\)*and j*_{l}*as the smallest nonnegative integer j satisfying the following inequality-***Armijo step size rule**$$J\left[\theta^{(l)}-\mu^{j}\nabla_{\theta} J\right]- J\left[\theta^{(l)}\right] \leq -\beta \mu^{j} \|\nabla_{\theta} J\|^2 $$(31)**Step 5***If J*[*θ*^{(l)}] −*J*[*θ*^{(l + 1)}] ≤*ε Then STOP Else set l*: =*l*+ 1*and go to Step 1*

*β*,

*μ*and

*ε*depends on the problem at hand. Numerical experience has shown the universality of some values (see e.g., Polak 1971) for indications. The computation in Step 1 involves the solution of (

*K*+ 1) Initial Value Problems (IVP). Particular attention should be paid to the detection of the switching instants. This can be done using the event location capabilities of Matlab IVP solvers (Shampine and Thompson 2000). Step 2 involves the solution of a linear time varying system that should pose no major difficulties. In Step 3, the multipliers are computed such that the Hamiltonian continuity condition is enforced. Recall here that a closed form solution of the multipliers can be found by combining Eqs. 12 and 13. Step 4 is the costliest part of the algorithm. Indeed, evaluation of the cost functional in the right-hand side of inequality (31) makes internal calls to Step 1–Step 3. However, the number of such calls is finite. In Figs. 3 and 4, some iterations of Algorithm 1 applied to a first-order system are shown schematically. The rate of decay of the cost is provided by Eq. 31. The rate is large whenever the gradient is large and this happens usually during the first iterations of the algorithm (Figs. 5 and 6).

## 5 Conclusions

This paper addresses optimization problems for a class of hybrid systems arising frequently in the process industries. Using variational principles, necessary conditions of optimality and a gradient formula are derived. A conceptual algorithm is then presented together with a convergence analysis and a discussion of implementation issues. A topic of current research is the study of the cyclic operation of the system i.e., the switching sequence of locations is periodic. We expect that a cycle-to-cycle improvement is possible and could be used to further reduce the complexity of the developed approach.

Note that this is different from the standard hybrid automaton framework, where switches are enforced by the continuous state violating invariants.

## Acknowledgements

The authors thank the anonymous referees for valuable remarks and suggestions from which the final version of the paper greatly benefited.

**Open Access**

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.