Decomposition of Convex High Dimensional Aggregative Stochastic Control Problems

Abstract

We consider the framework of convex high dimensional stochastic control problems, in which the controls are aggregated in the cost function. As first contribution, we introduce a modified problem, whose optimal control is under some reasonable assumptions an $\varepsilon$-optimal solution of the original problem. As second contribution, we present a decentralized algorithm whose convergence to the solution of the modified problem is established. Finally, we study the application of the developed tools in an engineering context, studying a coordination problem for large populations of domestic thermostatically controlled loads.

Appendix

Lemma 1

Let H be a Hilbert space and $f:H\rightarrow {\mathbb {R}}$ be l.s.c. and convex. The function f has at most quadratic growth if and only if its subgradient has linear growth.

Proof

Let the subgradient have linear growth, that is, $\Vert q\Vert _H \le c_1(1+\Vert x \Vert _H)$ whenever $x \in H$ and $q\in \partial f(x)$. Then $f(x) \le f(0) + \langle q, x \rangle _H \le f(0) + \Vert q\Vert _H \Vert x\Vert _H \le c_2 (1+\Vert x \Vert _H^2)$, so that f has at most quadratic growth.

Conversely, let f have at most quadratic growth. Since f is convex, one has for all $x\in H$ and $q_0\in \partial f(0)$:

$$\begin{aligned} f(x)\ge f(0) + \langle q_0,x \rangle _H \ge -c_3(1+ \Vert x \Vert _H^2), \end{aligned}$$

where $c_3>0$ depends only on f(0) and $q_0$. Then, using the growth assumption on f and the inequality above, one gets for all $x,y\in H$ and $q\in \partial f(x)$:

$$\begin{aligned} c_4(1+ \Vert y \Vert _H ^2) \ge f(y) \ge f(x) + \langle q, y-x \rangle _H \ge -c_3(1+\Vert x \Vert _H^2) + \langle q, y-x \rangle _H. \end{aligned}$$

Take $y = x + \alpha q$, with $\alpha \in (0,1)$, we get

$$\begin{aligned} 2c_4(1 + \Vert x \Vert _H^2 + \alpha ^2 \Vert q \Vert _H^2)\ge -c_3(1+\Vert x \Vert _H^2) + \alpha \Vert q\Vert _H^2 \end{aligned}$$

so that $( \alpha - 2c_4\alpha ^2) \Vert q\Vert _H^2 \le (2c_4+c_3)(1+\Vert x \Vert _H^2)$. Take $\alpha = 1/(4c_4)$, then $\alpha - 2c_4 \alpha ^2 =1 /(8c_4) >0$ and then

$$\begin{aligned} \Vert q \Vert _H^2\le 8c_4(2c_4+c_3)(1+ \alpha \Vert x\Vert _H^2) \end{aligned}$$

and the conclusion follows. $\square$