Abstract
We are interested in the optimal control problem associated with certain quadratic cost functionals depending on the solution \(X=X^\alpha \) of the stochastic mean-field type evolution equation in \({\mathbb {R}}^d\)
under assumptions that enclose a system of FitzHugh–Nagumo neuron networks, and where for practical purposes the control \(\alpha _t\) is deterministic. To do so, we assume that we are given a drift coefficient that satisfies a one-sided Lipschitz condition, and that the dynamics (2) satisfies an almost sure boundedness property of the form \(\pi (X_t)\le 0\). The mathematical treatment we propose follows the lines of the recent monograph of Carmona and Delarue for similar control problems with Lipschitz coefficients. After addressing the existence of minimizers via a martingale approach, we show a maximum principle for (2), and numerically investigate a gradient algorithm for the approximation of the optimal control.
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
1 Introduction
1.1 Motivations
Based on a modification of a model by van der Pol, FitzHugh [17] proposed in 1961 the following system of equations in order to describe the dynamics of a single neuron subject to an external current I:
for some constants \(a,b,c>0\), where the unknowns v, w correspond respectively to the so-called voltage and recovery variables (see also Nagumo [19]). In presence of interactions, one has to enlarge the previous pair by an additional unknown y that counts a fraction of open channels (synaptic channels), and which is sometimes referred to as gating variable.
When it comes to an interacting network of neurons, it is customary to assume that the corresponding graph is fully connected, which is arguably a good approximation at small scales [24]. This implies that all the neurons in the given network add a contribution to the interaction terms in the equation. Precisely, for a population of size \(N\in {\mathbb {N}},\) the state at time t of the i-th neuron is described by the three-dimensional vector
and one is led to study the system of 3N stochastic differential equations:
In the above, \(W^i\), \(B^{i}\), \({{\tilde{B}}}^i\) are i.i.d. Brownian motions modelling independent sources of noise with respective intensities \(\sigma ^J,\sigma _{ext},\sigma ^{y^i}(v^i)>0\). The last of these intensities depends on the solution, through the formula
with given constants \({\overline{a}},{\overline{b}}>0\) and some smooth cut-off function \(\chi :{\mathbb {R}}\rightarrow {\mathbb {R}}\) supported in (0, 1).
In this model, the voltage variable \(v^i\) describes the membrane potential of the \( i \)-th neuron in the network, while the recovery variable \(w^i\) models the dynamics of corresponding ion channels, which can influence the membrane potential of neuron i by opening and closing, depending on \(v^i\).
Without interaction term (i.e. when \(J=\sigma ^J=0\)), equation (3) reduces to a network of N independent neurons, where the dynamics of each neuron is modelled by the FitzHugh–Nagumo equation (2).
When interaction is present (\(J\not =0\)), the model describes the situation where each neuron of the network affects its adjacent neurons by releasing chemical transmitters, causing particular ion channels of adjacent neurons to open. This induces a current to the adjacent neuron, affecting its membrane potential. In this extended model, the gating variable \(y^i\) models a fraction of open ion channels in the adjacent neurons of neuron i, and thus ought to be a number between 0 and 1 (hence the cut-off \(\chi (y^i)\) in (4)). Loosely speaking, \(y^i\) should be thought as the output contribution of the neuron i.
Depending on the fraction \(y^i\) of open channels, the induced current for an adjacent neuron with membrane potential v is given by \(-J(v-V_{rev})y^i\), where the constant \(V_{rev}\) describes the membrane potential, at which there is no net current flow. The coupling strength J originally refers to the (mean of) the maximum conductance, which is typically affected by noise coming from the environment. This explains the diffusion part of the interaction term in Eq. (3) for \(\sigma ^J\not =0\). For further details we refer to [2].
The dynamics of the gating variable \(y^i\) in Eq. (3) depends on some physical constants, which we will now briefly introduce:
-
\(S(v^i)\) refers to the concentration of chemical transmitters, released by neuron i; explicitly for \(v\in {\mathbb {R}}\)
$$\begin{aligned} S(v)=\dfrac{T_{max}}{1+e^{-\lambda (v-V_T)}} \end{aligned}$$(5)where \(T_{max}\) is a given maximal concentration and \(\lambda ^{-1}>0,V_T>0,\) are constants setting the steepness, resp. the value, at which S(v) is half-activated (for typical values, see for instance [13]);
-
\({\overline{a}},{\overline{b}}>0\) correspond to some rise and decay rates, respectively.
For a better understanding of the interaction, we included a small illustration in Fig. 1.
Synaptic dynamics
In this representation, a rapid increase of the membrane potential of the neuron i will cause it to release chemical messengers into the (synaptic) cleft between the neuron i and the adjacent neuron j, which in turn will bind to receptors of the neuron j. The receptors will cause ion channels of neuron j to open, thus neuron i induces the opening of a fraction \(y^i\) of ion channels at the dendrites of the neuron j. As already mentioned, the resulting current from i to j affecting the neuron j is then given by \(-J(v^j-V_{rev})y^i\). For a thorough presentation of (3) and its applications in the field of neurosciences, we refer for instance to the monograph of Ermentrout and Terman [16].
When it comes to monitoring neural activity in the brain, one does typically not measure single neuron activity, but consider more macroscopic measurements like, e.g., local field potentials (LFPs). The LFP refers to the electric potential in the extracellular space around neurons and it is influenced by all ionic processes around the electrode. It is assumed that action potentials have a limited impact to the LFP and that it is strongly influenced by synaptic currents. Although we are aware of the fact that the correct interpretation of measurements like the intracerebral local field potential is highly non trivial, for our type of application it is reasonable to assume that measurements only depend on the distribution of our network, rather than on activities of single neurons from the network. Furthermore, external stimulation acts homogeneously on every neuron of the network, that is every neuron of the network receives the same external input I. We therefore model the dynamics of a typical neuron as a controlled mean-field type equation and admissible controls need to be independent of the state of the individual neuron, hence we will consider deterministic controls. In the following we will formulate our mean-field model and the set of admissible controls will be defined in Sect. 2.2.
1.2 Propagation of Chaos
The system (3) has the generic form
for \(i=1,\dots ,N\), where \(u_0\) is a probability measure on \({\mathbb {R}}^d\), \((\alpha _t)\) is a control and \({{\bar{\mu }}}_{X_t^N}\) denotes the empirical measure
For \(N\rightarrow \infty \), one is naturally pushed to investigate the convergence in law of the solutions of (6) towards the probability measure \(\mu ={{\,\mathrm{\mathcal {L}}\,}}(X|{\mathbb {P}})\), where X solves
and where \(b,\sigma \) are the coefficients obtained by substituting expectations in (6) in place of empirical means. In the context of (3), a first mathematical investigation of such convergence is due to Baladron, Fasoli, Faugeras and Touboul [2] (see also the clarification notes [6]). In this direction, the authors show that the sequence of symmetric probability measures
is \(\mu \)-chaotic. Namely, for each integer \(k\le N\) and every finite sequence of bounded and continuous functions \( \phi _i:C([0,T];{\mathbb {R}}^{d})\rightarrow {\mathbb {R}},\) for \( i=1,...,k, \) it holds
where \( \langle \mu ,\phi \rangle :=\int \phi d\mu \) and the symbol “\(\otimes \)” denotes the usual tensor product, i.e. \( \phi _i\otimes \phi _j(f,g):=\phi _i(f)\phi _j(g) \), for any \( f,g\in C([0,T];{\mathbb {R}}^d) .\) This situation is usually referred to as “propagation of chaos”. Although we could not find any literature concerning propagation of chaos results in the general non Lipschitz case, propagation of chaos for the system (3) was investigated in [6].
1.3 Mean-Field Limit and Control
In this regard, taking \(N\gg 1\) guarantees that a “good enough” approximation of (3) is given by the mean-field limit (7), where the corresponding coefficients \((b,\sigma ):[0,T]\times {\mathbb {R}}^3\times \mathcal {P}({\mathbb {R}}^3)\times {\mathbb {R}}\rightarrow {\mathbb {R}}^3\times {\mathbb {R}}^{3\times 3}\), are given by
for \(x=(v,w,y),\) and
In this paper, we concentrate our attention on the optimal control problem associated with a cost functional of the form
for suitable functions f and g, and where \(X^\alpha \) is subject to the dynamical constraint (7). The functional cost ought to be minimized over some convex, admissible set of controls \({{\mathbb {A}}}.\)
Because of potential applications in neuroscience, the control of the stochastic FHN model has gained a lot of attention during the last years (see, e.g., [3, 11]). The need to introduce random perturbations in the original model is widely justified from a physics perspective (see for instance [12] and the references therein). In [11] the authors investigate a FitzHugh–Nagumo SPDE which results from the continuum limit of a network of coupled FitzHugh–Nagumo equations. We have a similar structure in mind regarding the dependence of the coefficients on the control (namely, the dynamics of the membrane potential depends linearly on the control). Our approach here is however completely different, in that we hinge on the McKean-Vlasov type SDE (7) that originates from the propagation of chaos.
McKean-Vlasov control problems of this type were investigated in the past decade by Bensoussan, Frehse and Yam [4], but also by Carmona and co-authors (see for instance [9]). These developments culminated with the monograph of Carmona and Delarue [8], where a systematic treatment is made (under reasonable assumptions). Other related works include [1, 5, 7, 14]. These results fail however to encompass (7)–(9), due for instance to the lack of Lipschitz property for the drift coefficient.
From the analytic point of view, the FitzHugh–Nagumo model also suffers the fact that the diffusion matrix is degenerate, making difficult to obtain energy estimates for the Kolmogorov equation (see Remark 3.2).
Our objective in this work is twofold. At first, our purpose is to extract some of the qualitative features of FitzHugh–Nagumo system and its mean field limit, in a broader treatment that encloses (3) and (7)–(9). In this sense, our intention is not to deal with the previous models “as such” but instead, we aim to take a step further by dealing with a certain class of equations that has the following peculiarities
-
(Monotonicity) – though the drift coefficient in (7) displays a cubic non-linearity, it satisfies the monotonicity condition \(\langle x-x',b(t,x,\mu ,\alpha )-b(t,x',\mu ,\alpha )\rangle \lesssim |x-x'|^2\).
-
(Structural assumption on dynamics and level set boundedness) – the dynamics of the coupling variable ensures that the boundedness property \(y_t\in [0,1]\) holds for all times.
-
(Interaction with linear growth w.r.t. the unknown) – the drift nonlinearity displays the behaviour \(|b(t,x,\mu ,\alpha )-b(t,x,\nu ,\alpha )|\lesssim (1+|x|)W_2(\mu ,\nu )\), where \(W_2\) denotes the usual 2-Wasserstein distance defined in Sect. 2.1.
Under the above setting, we aim to develop and implement direct variational methods, in the spirit of the stochastic approach of Yong and Zhou [25] for classical control problems (note that some work in this direction has been already done by Pfeiffer [21, 22], in a slightly different setting). Second, we aim to derive a Pontryagin maximum principle for mean-field type control problems of the previous form, with a view towards efficient numerical approximations of optimal controls (e.g. gradient descent).
1.4 Organization of the Paper
In Sect. 2 we introduce our assumptions on the coefficients and give the main results. Section 3 is devoted to the well-posedness of the main optimal control problem (Theorem 2.1). In Sect. 4, we show the corresponding maximum principle (Theorem 2.2). Finally, Sect. 5 will be devoted to numerical examples.
2 Preliminaries
2.1 Notation and Settings
In the whole manuscript, we consider an arbitrary but finite time horizon \(T>0\). We fix a dimension \(d\ge 1\), and denote the scalar product in \({\mathbb {R}}^d\) by \(\langle \cdot ,\cdot \rangle .\) If A, B are matrices of the same size, we shall also write \(\langle A, B\rangle \) for their scalar product, namely
where \(A^\dagger \) is the transposed matrix, and \(\mathrm {tr}\) the trace operator. For a continuously differentiable function \(f:{\mathbb {R}}^d\rightarrow {\mathbb {R}}\), we adopt the suggestive notation \(f_x\) to denote its Jacobian (seen for each \(x\in {\mathbb {R}}^d\) as an element of the dual of \({\mathbb {R}}^d\)). Given \(h\in {\mathbb {R}}^d\), we let
be the evaluation of \(f_x(x)\) at h. A similar convention will be used for vector-valued functions.
Throughout the paper, we fix a complete filtered probability space \((\Omega ,\mathcal {F}, (\mathcal {F}_t)_{t\in [0,T]},{\mathbb {P}})\) carrying an m-dimensional Wiener process \((W_t)_{t\in [0,T]}\). Given \(p\in [1,\infty )\) and a p-integrable random variable X, we denote its usual \(L^p\)-norm by \(\Vert X\Vert _p:={\mathbb {E}}(|X|^p)^{1/p}\). We further introduce the spaces
For \(m\in {\mathbb {N}}\), the notations \(\mathcal {S}^{2,d\times m}\), \(\mathcal {H}^{2,d\times m}\) will also be used to denote the corresponding sets of \(d\times m\) matrix-valued processes. Whenever clear from the context, we will omit to indicate dimensions and write \(\mathcal {S}^2\) or \(\mathcal {H}^2\) instead.
We will denote by \(\mathcal {P}({\mathbb {R}}^d)\) the set of all probability measures on \(({\mathbb {R}}^d,\mathcal {B}({\mathbb {R}}^d))\). For \(p\in [1,\infty )\), \(\mu \in \mathcal {P}({\mathbb {R}}^d)\) we define the moment of order p:
and we let \(\mathcal {P}_p({\mathbb {R}}^d):=\left\{ \mu \in \mathcal {P}({\mathbb {R}}^d)\,\big |\,{\mathcal {M}}_p(\mu )<\infty \right\} .\) By \(W_p,\) \(p\in [1,\infty ),\) we denote the usual p-Wasserstein distance on \({\mathcal {P}}_p\), that is for \(\mu ,\nu \in {\mathcal {P}}_p({\mathbb {R}}^d)\)
where \(\Pi (\mu ,\nu )\) denotes the set of probability measures on \({\mathbb {R}}^d\times {\mathbb {R}}^d\) with \(\mu \) and \(\nu \) as respective first and second marginals (we refer to [8, Chap. 5] for a thorough introduction to the subject). Moreover, we recall the following elementary but useful consequence of the previous definition. Let \(\mu ,\nu \) be in \({\mathcal {P}}_p,\) and assume that there are random variables X, Y on \((\Omega ,{\mathcal {F}},{\mathbb {P}})\) such that \(X\sim \mu \) and \(Y\sim \nu .\) Then, it holds
Finally, we will shortly recall the notion of L-differentiability. A function \(f:\mathcal {P}_2({\mathbb {R}}^d)\rightarrow {\mathbb {R}}\) is called L-differentiable at \(\mu _0\in \mathcal {P}_2({\mathbb {R}}^d)\) if there exists a random variable \(X_0\) with law \(\mu _0\), such that the lifted function
is Fréchet differentiable at \(X_0\). The following property for a L-differentiable function f is well-known (see [8, Chap. 5]): for any \(\mu _0\in \mathcal {P}_2({\mathbb {R}}^d)\), there exists a \(\mu _0\)-almost everywhere uniquely defined measurable function \(\xi :{\mathbb {R}}^d\rightarrow {\mathbb {R}}^d\), such that for all \(X_0\in L^2(\Omega ,\mathcal {F},{\mathbb {P}};{\mathbb {R}}^d)\) with \(\mathcal {L}(X_0)=\mu _0\), it holds \(D{\tilde{f}}(X_0)=\xi (X_0)\). We write \(f_\mu (\mu _0)\) to denote the equivalence class of \(\xi \) in \(L^2({\mathbb {R}}^d,\mu _0;{\mathbb {R}}^d)\). In keeping with the notation (11) on differentials, we will let \(f_\mu (\nu )(x)\cdot h\) be its evaluation (as an element of the dual of \({\mathbb {R}}^d\)) at \(h\in {\mathbb {R}}^d\). If \(f_\mu \) is continuous, we call f continuously L-differentiable.
2.2 Controls and Cost Functional
Our controlled dynamics will be given by a McKean-Vlasov type SDE (state equation) of the form (7), where \(X_0\in L^r(\Omega ,\mathcal {F}_0,{\mathbb {P}};{\mathbb {R}}^d)\) for some fixed \(r\ge 6\) and \(\alpha \) is an admissible control, i.e. for some bounded convex set \(A\subset {\mathbb {R}}^k\) and throughout the paper,
In the whole manuscript, we assume that we are given continuous running and terminal cost functions
which have quadratic growth in the following sense: there exists \(C>0\) such that for all \(t\in [0,T]\), \(x\in {\mathbb {R}}^d\), \(\alpha \in A\) and \(\mu \in \mathcal {P}_2({\mathbb {R}}^d)\)
We will then consider the cost functional
where \(X^\alpha \) is subject to the dynamical constraint (7).
2.3 Level Set Boundedness
A formal application of Itô Formula reveals that the solutions of the state equation associated with a network of FitzHugh–Nagumo neurons take values in the set
This is of course coherent with the intuition that y is a fraction of open channels. In other words, we have \(\pi (X)\le 0\) where \(\pi :{\mathbb {R}}^3\rightarrow {\mathbb {R}},\) is the map \(x\mapsto y(y-1).\) Motivated by this example, we will assume in the sequel that we are given a convex function \(\pi \in C^2({\mathbb {R}}^d,{\mathbb {R}})\) such that any solution X is supported in \({\mathcal {C}}\subset {\mathbb {R}}^d\) for all times, where \({\mathcal {C}}\) is the set
We suppose moreover that \({\mathcal {C}}\) contains at least one element, which for convenience is assumed to be 0. To ensure that the solutions are indeed \( {\mathcal {C}}\)-valued, we need to assume that \(\pi (X_0)\le 0\), \({\mathbb {P}}\text {-almost surely}\). Furthermore we need to make the following compatibility assumptions on \(\pi :{\mathbb {R}}^d\rightarrow {\mathbb {R}}.\)
Assumption 2.1
(structural assumption on the dynamics) For all \(\mu \in \mathcal {P}({\mathbb {R}}^d),\alpha \in A,\) \(t\in [0,T]\) and \(x\in {\mathbb {R}}^d\setminus {\mathcal {C}}\), we have
while
Example 2.1
(Gating variable boundedness for FitzHugh–Nagumo) Assumption 2.1 is fulfilled for (7)–(9) and with \(\pi (v,w,y)=y(y-1)\), as can be seen as follows. We have the identities (recall the notation (4))
Clearly,
But using \({\mathrm {Supp}}\chi \subset (0,1)\), we find indeed that \((2y-1)\sigma ^y(v)=0\) outside \({\mathcal {C}}\). The same argument implies
and the latter vanishes if \(x\notin {\mathcal {C}}\), hence (18).
Towards (17), one observes letting \(q={\bar{a}} S(v)\) that
The polynomial P(y) has discriminant \((q-b)^2\), hence the roots
which both lie in the interval (0, 1). It follows that P(y) is negative outside \({\mathcal {C}}\), implying (17).
2.4 Regularity Assumptions and Main Results
Besides Assumption 2.1, one needs to make suitable hypotheses on the regularity of the drift and diffusion coefficients. In the sequel, we denote by \({\mathcal {P}}_2^{{\mathcal {C}}}({\mathbb {R}}^d)\) the subset of all probability measures in \({\mathcal {P}}_2({\mathbb {R}}^d)\) which are supported in \({\mathcal {C}}=\pi ^{-1}((-\infty ,0]).\)
Assumption 2.2
(MKV Regularity) We assume that the coefficients
are locally Lipschitz. Moreover, there are constants \(L_1,L_2,L_3>0\) such that the following properties hold.
-
(L1)
– (regularity of the diffusion coefficient) – The diffusion coefficient \(\sigma \) satisfies the property \(\sup _{0\le t\le T}|\sigma (t,0,\delta _0,0)|^2<\infty .\) Moreover, for all \(t\in [0,T]\), \(x\in \mathcal {C}\), \(\alpha \in A\) and \(\mu \in \mathcal {P}^{{\mathcal {C}}}_2({\mathbb {R}}^d)\) we have
$$\begin{aligned} |\sigma (t,x,\mu ,\alpha )|^2\le L_1(1+|\alpha |^2+|x|^2) . \end{aligned}$$(19)For all \(x,x'\in \mathcal {C} \), \(\alpha '\in A\), it holds
$$\begin{aligned} |\sigma (t,x,\mu ,\alpha )-\sigma (t,x',\mu ,\alpha ')|^2\le L_1(|x-x'|^2+|\alpha -\alpha '|^2). \end{aligned}$$(20)Finally, if \(x\in \mathcal {C}\) and \(\mu '\in \mathcal {P}_2({\mathbb {R}}^d)\), then
$$\begin{aligned} |\sigma (t,x,\mu ,\alpha )-\sigma (t,x,\mu ',\alpha )|^2\le L_1(1+|x|^2)W_2(\mu ,\mu ')^2. \end{aligned}$$(21) -
(L2)
– (regularity of the drift coefficient) – There exists \(q\in {\mathbb {N}}\) with \(4q\le r\), such that for all \(t\in [0,T]\), \(x,x'\in \mathcal {C}\), \(\alpha ,\alpha '\in A\) and \(\mu \in \mathcal {P}_2({\mathbb {R}}^d)\)
$$\begin{aligned} \begin{aligned}&|b(t,x,\mu ,\alpha )-b(t,x',\mu ,\alpha ')| \\&\le \sqrt{L_2}(1+|x|^{q-1}+|x'|^{q-1}+|\alpha |^{q-1}+|\alpha '|^{q-1}+\mathcal {M}_2(\mu )^2)(|x-x'|+|\alpha -\alpha '|). \end{aligned} \end{aligned}$$(22)In addition, b satisfies the following Lipschitz property with respect to the Wasserstein distance: for all \(t\in [0,T]\), \(x\in \mathcal {C}\), \(\alpha \in A\) and \(\mu ,\mu '\in \mathcal {P}_2({\mathbb {R}}^d)\)
$$\begin{aligned} |b(t,x,\mu ,\alpha )-b(t,x,\mu ',\alpha )|^2\le L_2(1+|x|^2)W_2(\mu ,\mu ')^2. \end{aligned}$$(23) -
(L3)
– (monotonicity of the drift) – The drift coefficient b is such that \(\sup _{0\le t\le T}|b(t,0,\delta _0,0)|<\infty .\) Moreover, for all \(t\in [0,T]\), \(x\in {\mathcal {C}}\), \(\alpha \in A\) and \(\mu \in \mathcal {P}^{{\mathcal {C}}}_2({\mathbb {R}}^d)\) it holds
$$\begin{aligned} \langle x,b(t,x,\mu ,\alpha )\rangle \le L_3(1+|\alpha |^2+|x|^2) \end{aligned}$$(24)and if \(x'\in {\mathcal {C}}\), \(\alpha '\in {{\mathbb {A}}}\), then
$$\begin{aligned} \left\langle x-x',b(t,x,\mu ,\alpha )-b(t,x',\mu ,\alpha ')\right\rangle \le L_3(|x-x'|^2+|\alpha -\alpha '|^2). \end{aligned}$$(25)
Example 2.2
(Analysis of the FitzHugh–Nagumo model) Let us go back to the settings of (7)–(9) for a coupled system of FitzHugh–Nagumo neurons. Trivially, one has \(\sup _{0\le t\le T}|\sigma (t,0,\delta _0,0)|=|\sigma _{ext}|<\infty .\) The map \(v\mapsto S(v)\) being positive and bounded, we further see that the (3, 3)-th entry of \(\sigma \) is Lipschitz, as deduced immediately from the fact that \(\chi \) is supported in (0, 1). For the remaining non-trivial component, we have
where to ease notation we introduce the barycenter \(\beta (\mu )\), defined as the quantity
The condition \({\mathrm {Supp}}\mu \subset {\mathcal {C}},\) implies trivially that \(|\beta (\mu )|\le 1\) and thus we obtain (19) for \(L_1=(V_{rev}J)\vee 1.\) The Lipschitz-type property (20) is shown in a similar fashion.
The Wasserstein-type regularity (21) is hardly more problematic: using the Kantorovitch duality Theorem [8, Prop. 5.3 & Cor. 5.4] and the fact that the projector \(z=(z_1,z_2,z_3)\mapsto z_3\) is Lipschitz, one finds that
hence
As is classical, the 1-Wasserstein distance \(W_1(\mu ,\mu ')\) can be estimated by \(W_2(\mu ,\mu '),\) which in turn implies (21), and thus 2.2.
As for the drift coefficient, since \(b(t,0,\delta _0,0)\) is also independent of t, the supremum condition in 2.2 is clear. Moreover, it has polynomial dependency on the variables v, w, y, which implies the local Lipschitz property (22) with \(q=3\). We also have
and we conclude by (27) that 2.2 holds.
To show (24) and (25), it is enough to prove the corresponding bounds when \(c=0={{\overline{b}}} ,\) since the related contributions are affine linear in the variables. Similarly, by linearity we can let \(w=\alpha =0\). But in that case, it holds
Observe that, since \(\mu \) is supported inside \({\mathcal {C}}\), one has in particular \(\beta (\mu )\ge 0\). Consequently, the fourth term in the right hand side can be ignored, showing (24) with \(L_3=L_3({{\overline{a}}} ,|S|_{\infty },J,V_{rev})>0.\)
Similarly, if \(x'=(v',0,y')\in {\mathbb {R}}^3\)
It follows that (24) holds with \(L_3=L_3({{\overline{a}}},{{\overline{b}}},c,|S|_{C^1})>0.\)
Assumption 2.3
(Weak continuity) For any \(t\in [0,T]\), \(x\in {\mathbb {R}}^d\) and \(\mu \in \mathcal {P}_2({\mathbb {R}}^d)\), the function \(A\rightarrow {\mathbb {R}}\), \(\alpha \mapsto f(t,x,\mu ,\alpha )\) is convex and the functions \(A\rightarrow {\mathbb {R}}^d\times {\mathbb {R}}^{d\times m}, \alpha \mapsto (b,\sigma )(t,x,\mu ,\alpha )\) are affine. Furthermore, for all \(x\in C([0,T];{\mathbb {R}}^d)\) and \(\mu \in C([0,T];\mathcal {P}_2^{\mathcal {C}}({\mathbb {R}}^d))\) the functions
are weakly sequentially continuous, that is for all \((\alpha ^n)_{n\in {\mathbb {N}}}\subset {\mathbb {A}}\) with \(\alpha ^n\rightharpoonup \alpha \), for some \(\alpha \in {\mathbb {A}}\), it holds \((b,\sigma )(\cdot ,x_\cdot ,\mu ,\alpha ^n_\cdot )\rightharpoonup (b,\sigma )(\cdot ,x_\cdot ,\mu ,\alpha _\cdot )\).
Remark 2.1
The continuity and convexity of \(f(t,x,\mu ,\cdot )\) leads to weak lower semicontinuity of the map
for all \(x\in C([0,T];{\mathbb {R}}^d)\) and \(\mu \in C([0,T];\mathcal {P}_2({\mathbb {R}}^d))\).
We can now present our main results. At first, we investigate the existence of an optimal control for the following problem
subject to
Theorem 2.1
Under assumptions 2.1–2.3, the problem (SM) is finite and has an optimal control. Namely, \(\inf _{\alpha \in {{\mathbb {A}}}}J(\alpha )<\infty \) and there is \({\overline{\alpha }}\in {{\mathbb {A}}}\), such that
for all \(\alpha \in {{\mathbb {A}}}\).
In order to address the corresponding maximum principle, we now introduce further assumptions on our coefficients.
Assumption 2.4
(Pontryagin Principle) The coefficients \(b,\sigma ,f\) and g are continuously differentiable with respect to \((x,\alpha )\) and continuously \(\mathrm {L}\)-differentiable with respect to \(\mu \in {\mathcal {P}}_2({\mathbb {R}}^d).\) Furthermore there exist \(A_1,A_2,A_3>0\) such that:
-
(A1)
For every \((s,x,\mu ,\alpha )\in [0,T]\times {\mathcal {C}}\times {\mathcal {P}}_2^{\mathcal {C}}({\mathbb {R}}^d)\times A,\) and each \(y,z\in {\mathbb {R}}^d\):
$$\begin{aligned} \langle b_x(t,x,\mu ,\alpha )\cdot z,z\rangle&\le A_1|z|^2,\\ |b_x(t,x,\mu ,\alpha )|&\le A_1(1+|x|^{q-1}),\\ |b_\alpha (t,x,\mu ,\alpha )|&\le A_1,\\ |b_\mu (t,x,\mu ,\alpha )(y)|&\le A_1(1+|x|), \end{aligned}$$where q is the same constant as in 2.2.
-
(A2)
For every \((s,x,\mu ,\alpha )\in [0,T]\times {\mathcal {C}}\times {\mathcal {P}}_2^{\mathcal {C}}({\mathbb {R}}^d)\times A,\) and \(y\in {\mathbb {R}}^d\):
$$\begin{aligned} |\sigma _x(t,x,\mu ,\alpha )|&\le A_2,\\ |\sigma _\alpha (t,x,\mu ,\alpha )|&\le A_2,\\ |\sigma _\mu (t,x,\mu ,\alpha )(y)|&\le A_2(1+|x|). \end{aligned}$$ -
(A3)
For all \(R>0\) and every \((s,x,\mu ,\alpha )\in [0,T]\times {\mathbb {R}}^d\times {\mathcal {P}}_2 ({\mathbb {R}}^d)\times A,\) such that \(|x|\vee \mathcal {M}_2(\mu )\vee |\alpha |\le R\) the quantities
$$\begin{aligned}&f_x(t,x,\mu ,\alpha ),\, f_\alpha (t,x,\mu ,\alpha ),\, g_x(x,\mu ),\, \int _{{\mathbb {R}}^d}|f_\mu (t,x,\mu ,\alpha )(y)|^2\mu (dy),\\&\quad \int _{{\mathbb {R}}^d}^{}|g_\mu (x,\mu )(y)|^2\mu (dy),\, \end{aligned}$$are all bounded in norm by \(A_3(1+R)\).
Example 2.3
Again, we investigate the above properties for the setting of a FitzHugh–Nagumo neural network. The property 2.4 depends on the choice of f and g, hence we do not discuss it here (it is however clear for the ansatz (47) below). Concerning assumption 2.4 and 2.4 we have
where we recall the notation (26). Using that \({\mathrm {Supp}}(\mu )\subset {\mathcal {C}}\), together with the boundedness of \(S'(v)\), this leads to
hence the first estimate. Letting as before \(\beta (\mu ):=\int _{{\mathbb {R}}^3}z_3\mu (dz),\) it is easily seen by definition of the L-derivative that
In a matrix representation, this gives the following constant value for the L-derivative of the drift coefficient at a given point \(x\equiv (v,w,y)\in {\mathbb {R}}^3\)
Thus we have \(|b_\mu (t,x,\mu ,\alpha )({{\tilde{x}}})|\le J\vee (JV_{rev})(1+|x|),\) showing the desired property.
Next, we introduce the corresponding adjoint equation, which will be essential for the maximum principle. For a solution \(X\in \mathcal {S}^{2,d}\) of (28) consider the following backward SDE
where the tilde variables \({{\tilde{X}}},{{\tilde{P}}}\) are independent copies of the corresponding random variables (carried on some arbitrary probability space \(({{\tilde{\Omega }}},\mathcal {{{\tilde{F}}}},\mathbb {{{\tilde{P}}}})\)), and \(\mathbb {{{\tilde{E}}}}\) denotes integration in \({{\tilde{\Omega }}}\) (this convention will be adopted throughout the paper). Herein, we recall that \(\langle \sigma (t,x,\mu ,\alpha ),q\rangle \) is a synonym for \(\text {tr}(\sigma (t,x,\mu ,\alpha )^\dagger q)\).
A pair of processes \((P,Q)\in \mathcal {H}^{2,d}\times \mathcal {H}^{2,d\times m}\) will be called a solution to the adjoint equation corresponding to X if it satisfies (29) for all \(t\in [0,T]\), \({\mathbb {P}}\)-almost surely.
We are now in position to formulate the maximum principle. For that purpose, we introduce the Hamiltonian, which for each \(x,p\in {\mathbb {R}}^d,\) \(q\in {\mathbb {R}}^{d\times m}\) \(\mu \in {\mathcal {P}}_2\) and \(\alpha \in A,\) is the quantity
Theorem 2.2
Let assumptions 2.1–2.4 hold. Let \({\overline{\alpha }}\in {{\mathbb {A}}}\) be an optimal control for the problem (SM). If \((P,Q)\in \mathcal {H}^{2,d}\times \mathcal {H}^{2,d\times m}\) is the solution to the corresponding adjoint equation, then we have for Lebesgue-almost every \(t\in [0,T]\)
for all \(\alpha \in A\).
It should be noticed that in contrast to the maximum principle stated in [8, Thm. 6.14 p. 548], the maximum principle here is formulated in terms of the expectation for almost every \(t\in [0,T]\) instead of \(dt\otimes {\mathbb {P}}-\) almost everywhere, since we only consider deterministic controls and thus we only alter the control in deterministic directions.
3 Well-Posedness of the Optimal Control Problem
The main purpose of this section is to prove the existence of an optimal control for the stated control problem. For that purpose, we will need to show (among other results) that the state equation (7) is well-posed, and that the solution satisfies uniform moment bounds up to a certain level. Hereafter, we suppose that assumptions 2.1, 2.2 and 2.3 are fulfilled.
3.1 Well-Posedness of the State Equation
Our first task is to show that the level-set constraint which was alluded to in Sect. 2.3 is preserved along the flow of solutions. This statement is contained in the next result. The proof is partially adapted from that of [6, Prop. 3.3].
Lemma 3.1
For every \(\alpha \in {{\mathbb {A}}}\) and \(\mu \in C([0,T];\mathcal {P}^{{\mathcal {C}}}_2({\mathbb {R}}^d))\) we have that
where \(X_t^{\alpha ,\mu }\) is the unique solution to
Proof
First, observe that given \(\mu \in C([0,T];\mathcal {P}_2({\mathbb {R}}^d)),\) equation (31) has a unique strong solution \(X^\mu \) in \(\mathcal {S}^2.\) Indeed, if we let
then from Assumption 2.2 we see that \(\sigma ^\mu \) is Lipschitz, while 2.2 and 2.2 imply the local Lipschitz continuity and the monotonicity of the drift coefficient \(b^\mu \). Hence, by standard results on monotone SDEs (see for instance [20, Thm. 3.26 p. 178]) (31) has a unique strong solution, this solution being progressivey measurable and square integrable. This proves our assertion.
In order to show (30), consider a family \((\Psi _\epsilon )_{\epsilon >0}\) of non-negative and non-decreasing functions in \( C^2({\mathbb {R}})\) which for all \(\epsilon >0\) satisfy:
and such that \(\Psi _\epsilon \) converges pointwise to \({\mathbf {1}}_{(0,\infty )}\) as \(\epsilon \rightarrow 0\). Let \(\tau _n:=\inf \{t\ge 0\text { s.t.\ }|X_t|\ge n\}\). By Itô Formula, we have for each \(n\ge 0\) and \(\epsilon >0\)
where we let \(M_t^{\epsilon }:=\sum \nolimits _{k=1}^m\int _{0}^{\tau _n\wedge t} \pi _x(X_s)\cdot \sigma ^{\cdot ,k}(s,X_s,\mu _s,\alpha _s) \Psi _\epsilon '(\pi (X_s))dW^k_s\). Since \(\Psi _\epsilon \) is supported on the real positive axis, only the values of X which satisfy \(\pi (X)>0\) contribute to the above expression. Hence, making use of Assumption 2.1, we see that the first term in the previous right hand side is bounded above by 0, while the two last terms simply vanish. We arrive at the relation
Letting first \(n\rightarrow \infty \), and then \(\epsilon \rightarrow 0,\) we observe by Fatou Lemma that
and our claim follows. \(\square \)
We are now able to prove the existence of a unique solution to equation (7).
Theorem 3.1
There exists a unique strong solution to equation (7) in \(\mathcal {S}^2\), which is supported in \({\mathcal {C}}\) for all times. Furthermore, for each \(p\in [2,r]\) and every \(\alpha \in {{\mathbb {A}}},\) the solution satisfies the moment estimate
where the constant C depends only upon the indicated quantities.
Proof
Recall that \(\mathcal {P}_2^{\mathcal {C}}\) denotes the set of probability measures in \(\mathcal {P}_2({\mathbb {R}}^d)\) which are supported in \({\mathcal {C}}:=\pi ^{-1}((-\infty ,0]).\) Equipped with the standard Wasserstein distance, it is a closed subset of \(\mathcal {P}_2({\mathbb {R}}^d)\). Indeed, it is standard (see for instance [15]) that given probability measures \(\{\mu _n,n\in {\mathbb {N}}\}\) and \(\mu \) such that \(\mu _n\Rightarrow \mu \), then
so that our claim follows. Thus, for fixed \(\alpha \in {{\mathbb {A}}}\), we can rightfully consider the operator
where \(X^{\mu }=X^{\alpha ,\mu }\) is the unique solution to equation (31). Using similar arguments as in [8], the existence of a unique solution to (28) follows if one can show that \(\Theta \) has a unique fixed point. In fact, we are going to show that it is a contraction (for a well-chosen metric). The moment estimate (32) will follow from the fixed point argument, provided one can show that
where the displayed constant depends on the indicated quantities but not on the particular element \(\mu \) in \(C([0,T];{\mathcal {P}}^{{\mathcal {C}}}_2)\). We now divide the proof into two steps.
Step 1: moment bounds. We need a localization argument. For each \( n>0 \), introduce \( \tau _n:=\inf \{t\in (0,T], |X^{\mu }_t|>n\} \), and denote by \( X^{\mu ,n}_t:= X^{\mu }_{t\wedge \tau _n}\). Itô Formula gives
where \(N_t^{n,\mu }:=\int _0^{t\wedge \tau _n}\left| X^\mu _s|^{p-2}\langle X^\mu _s, \sigma (s,X^\mu _s,\mu _s,\alpha _s)dW_s\right\rangle \) is the corresponding martingale term. Denoting by \(\kappa >0\) the constant in the Burkholder-Davis-Gundy Inequality, the latter is estimated for each \( t\in [0,T] \) thanks to (19) and Cauchy-Schwarz Inequality as
But from Young’s inequality, the previous right hand side is also bounded by
Define \(\Psi ^n_t:={\mathbb {E}}\left( \sup _{0\le s\le t}|X_s^{n,\mu }|^p\right) .\) Taking the expectation in (34), we infer from (24), (19), Young’s inequality \(ab\le \frac{2}{p}a^{\frac{p}{2}}+\frac{p-2}{p}b^{\frac{p}{p-2}}\) and the previous discussion that
for some universal constant \(C_p>0.\) Applying Gronwall Inequality, we obtain the corresponding moment estimate for \( X^{\mu ,n} \), for each \( n>0 \). Since none of the constants used in the above computations depend on \( n \), the localization is removed by letting \( n\nearrow \infty \) and using the monotone convergence theorem.
Step 2: the fixed point argument. From Lemma 3.1, it is clear that for all \(t\in [0,T],\) the probability measure \({\mathbb {P}}\circ (X_t^\mu )^{-1}\) is supported in \({\mathcal {C}}\). For simplicity, let \(L:=L_1\vee L_2\vee L_3\) and introduce the weight
Then, Itô Formula gives
The first term in the right hand side of (35) is evaluated thanks to (25). For the second term, we use the quadratic growth assumption (23). As for the Itô correction, we can estimate it similarly, using this time Assumption 2.2. With \(M_t:=\int _{0}^{t} \phi _s\left\langle X^{\mu }_s-X^{\nu }_s,\sigma (s,X^{\mu }_s,\mu _s,\alpha _s)-\sigma (s,X^{\nu }_s,\nu _s,\alpha _s)dW_s\right\rangle \) we get
Taking expectations, supremum in t, then absorbing to the left yields
Using the estimate (32) with \(p=2\), the fact that \(\exp (-2TL)\le \phi \le 1,\) inequality (13) and the boundedness of the control state space \( A \subset {\mathbb {R}}^k\), we arrive at
Considering the k-th composition of the map \(\Theta \), we get
hence contractivity of \(\Theta ^k\) follows for \(k>0\) large enough and the result then follows from Banach-fixed point theorem. \(\square \)
We now investigate some regularity of the control-to-state operator, which will be needed in the proof of the optimality principle.
Lemma 3.2
For \(p\in [2,r],\) the solution map
is well-defined and Lipschitz continuous. More precisely, there exists a constant \(C(L_1,L_2,L_3,T)>0\), such that for all \(\alpha ,\beta \in {{\mathbb {A}}}\)
Proof
That G is well-defined follows immediately from Theorem 3.1. Towards Lipschitz-continuity, the property is shown by similar considerations as in the proof of Theorem 3.1. Indeed, fixing \(\alpha ,\beta \in {{\mathbb {A}}}\) and letting M be the martingale \(M_t:=\int _0^t\langle \sigma (t,X^{\alpha },{{\,\mathrm{\mathcal {L}}\,}}(X^{\alpha }),\alpha )-\sigma (s,X^{\beta },{{\,\mathrm{\mathcal {L}}\,}}(X^{\beta }),\beta ),(X^{\alpha }-X^{\beta }) dW \rangle \), then using Itô Formula with assumptions 2.2, 2.2 and 2.2, we arrive at
Letting \(\kappa >0\) be the constant in the BDG inequality, the estimate (13) and \(ab\le \frac{a^2}{4}+ b^2\) yield
where \(C_L:=\frac{1}{2}\vee L_1\vee \frac{L_2}{2}\vee L_3.\) The result now follows from the uniform bound (32), together with Gronwall Lemma. \(\square \)
Remark 3.1
Since we have \(W_2({{\,\mathrm{\mathcal {L}}\,}}(X_s^{\alpha }),{{\,\mathrm{\mathcal {L}}\,}}(X_s^{\beta }))\le {\mathbb {E}}\left( \sup _{t\in [0,T]}|X_s^{\alpha }-X_s^{\beta }|^2\right) ^\frac{1}{2}\) we also get the Lipschitz continuity of the map
Remark 3.2
(Fokker-Planck equation) Given the settings of Example 2.2, we define
If we assume that the solution to the corresponding mean-field equation has a density p(t, x) with respect to the 3-dimensional lebesgue measure, then the McKean-Vlasov-Fokker-Planck equation is given by the nonlinear PDE:
(see [2]). It is degenerate parabolic because the matrix \(\sigma {{{\tilde{\sigma }}}}^\dagger \) is not strictly positive.
3.2 Proof of Theorem 2.1
We now prove the existence of an optimal control for (28). The strategy we use strings along the commonly named “direct method” in the calculus of variations. As a trivial consequence of the assumptions made in Sect. 2.2 and the uniform estimate (32), note at first that our control problem is indeed finite. Next, consider a sequence \((\alpha ^n)_{n\in {\mathbb {N}}}\subset {{\mathbb {A}}}\) realizing the infimum of J asymptotically, i.e.
Since \({{\mathbb {A}}}\subset L^2([0,T];{\mathbb {R}}^k)\) is bounded and closed, by Banach Alaogu Theorem there exists an \(\alpha \in L^2([0,T];{\mathbb {R}}^k)\) and a subsequence also denoted by \((\alpha ^n)_{n\in {\mathbb {N}}}\), such that
Since \({{\mathbb {A}}}\) is also convex, we get \(\alpha \in {{\mathbb {A}}}\), so \(\alpha \) is indeed an admissible control. We now divide the proof into four steps.
Step 1: tightness. In the sequel, we denote by \(X^n\) the solution of the state equation (7) with respect to the control \(\alpha ^n\), \(n\in {\mathbb {N}}.\) Adding and subtracting in (7), we have
where \(\kappa >0\) is the constant in the BDG inequality. Using the assumptions 2.2, 2.2, 2.2, the fact that \(0\in {\mathcal {C}}\) and the basic inequality (13), we obtain that
Using Hölder Inequality, Young Inequality \(ab\le \frac{q-1}{q}a^\frac{q}{q-1}+\frac{1}{q}b^q\), we arrive at the following estimate, for all \(n\in {\mathbb {N}}\) and \(0\le s\le t\le T\)
where we used the fact that \( A \) is bounded. Note that the above constants depends upon the indicated quantities, but not on \(n\in \mathbb N.\)
Making use of the uniform estimate (32), the Kolmogorov continuity criterion then asserts that the sequence of probability measures \(({\mathbb {P}}\circ (X^n)^{-1})_{n\in {\mathbb {N}}}\), defined on the space
is tight. In the same way, we can prove that the sequence on probability measures \(({\mathbb {P}}_n)_{n\in {\mathbb {N}}}:=\left( {\mathbb {P}}\circ (X^n, B^n)^{-1}\right) _{n\in {\mathbb {N}}}\), with
is tight on the product space \(E\times E,\) with respect to the product topology, where for two E-valued random variables \(Z_1,Z_2\) defined on \((\Omega ,\mathcal {A},{\mathbb {P}})\), \({\mathbb {P}}\circ (Z_1, Z_2)^{-1}\) denotes the joint law of \(Z_1\) and \(Z_2\). Thus by Prokhorov’s theorem there exists a subsequence of \(({\mathbb {P}}_n)_{n\in {\mathbb {N}}}\), which converges weakly to some probability measure \({\mathbb {P}}^*\) on \(E\times E\).
Step 2: passage to the limit in the drift. By Skorokhod’s representation theorem we can then find random variables \({\overline{X}},{\overline{B}}\), \(({\overline{X}}^n)_{n\in {\mathbb {N}}},({\overline{B}}^n)_{n\in {\mathbb {N}}}\) defined on some probability space \(({\overline{\Omega }},\overline{\mathcal {F}},{\overline{{\mathbb {P}}}})\) and with values in \(E\times E\) such that
-
\({\overline{{\mathbb {P}}}}\circ ({\overline{X}}^n,{\overline{B}}^n)^{-1}={\mathbb {P}}_n\) for all \(n\in {\mathbb {N}}\) and \({\overline{{\mathbb {P}}}}\circ ({\overline{X}},{\overline{B}})^{-1}={\mathbb {P}}^*\) and
-
\(\lim \limits _{n\rightarrow \infty }({\overline{X}}^n,{\overline{B}}^n) =({\overline{X}},{\overline{B}})\), \({\overline{{\mathbb {P}}}}\)-almost surely with respect to the uniform topology.
From (33) and by the definition of \({\mathbb {A}}\) we get for any \(p\le r\)
for some constant independent of n. Thus we can conclude by the dominated convergence theorem that
as \(n\rightarrow \infty \). This also implies \((\mathcal {L}({\overline{X}}_t))_{t\in [0,T]}\subset \mathcal {P}_2^{\mathcal {C}}\), since \( \mathcal {P}_2^{\mathcal {C}}\) is closed.
To identify the almost sure limit \({\overline{B}}\), we first claim that for each \(t\in [0,T]\)
weakly in \(L^2({\overline{\Omega }};{\mathbb {R}}^d)\). Indeed, by (22) and the dominated convergence theorem we have
Likewise, for \(h\in L^2({\overline{\Omega }};{\mathbb {R}}^d)\) we have by Assumption 2.3 and dominated convergence
as \(n\rightarrow \infty \), thus proving our claim.
The desired identification then follows from (36), the Banach-Saks theorem and the uniqueness of the almost sure limit. The processes \({{\overline{B}}}\) and \(\int _{0}^{\cdot }b(s,{\overline{X}}_s,{{\,\mathrm{\mathcal {L}}\,}}({\overline{X}}_s),\alpha _s)ds\) being both continuous pathwise, they are indistinguishable, hence the identity
for all \(t\in [0,T],\) \({\overline{{\mathbb {P}}}}\)-almost surely.
Step 3: identification of the martingale. Letting \(\sigma \sigma ^\dagger (t,x,\mu ,\alpha ):=\sigma (t,x,\mu ,\alpha )\sigma (t,x,\mu ,\alpha )^\dagger \) for short, similar arguments as above show that
weakly in \(L^2([0,T]\times {\overline{\Omega }};{\mathbb {R}}^d)\). Since the process
is, for each n, a \(\mathcal {G}_t^n:=\sigma ({X_s^n|s\le t})\) martingale under \({\mathbb {P}}\), we can conclude that
is a \(\mathcal {{\overline{G}}}_t^n:=\sigma ({{\overline{X}}_s^n|s\le t})\) martingale under \({\overline{{\mathbb {P}}}}\) with quadratic variation
From the previous considerations, we can conclude that
\({\overline{{\mathbb {P}}}}\)-almost surely for all \(t\in [0,T]\). Thus by the dominated convergence theorem the process \(({\overline{M}}_t)_{t\in [0,T]}\) is a \(\overline{\mathcal {G}}_t:=\sigma ({\overline{X}}_s|s\le t)\) martingale under \({\overline{{\mathbb {P}}}}\) and with standard arguments we also obtain, that \(({\overline{M}}_t)_{t\in [0,T]}\) has quadratic variation
By the martingale representation theorem we can find an extended probability space \(({\hat{\Omega }},\hat{\mathcal {F}},(\hat{\mathcal {F}}_t)_{t\in [0,T]},{\hat{{\mathbb {P}}}})\) with an m-dimensional brownian motion \({\hat{W}}\), such that the natural extension \({\hat{X}}\) of \({\overline{X}}\) satisfies \({\hat{{\mathbb {P}}}}\circ ({\hat{X}}^{-1})={\overline{{\mathbb {P}}}}\circ ({\overline{X}}^{-1})\) and
\({\hat{{\mathbb {P}}}}\)-almost surely for all \(t\in [0,T]\).
Step 4: end of the proof It remains to show that the infimum is attained for \(\alpha .\) Due to the uniqueness of equation (7), we have \({\mathbb {P}}\circ (X^{\alpha })^{-1}={\hat{{\mathbb {P}}}}\circ ({\hat{X}}^{-1})\). Using Fatou’s lemma, continuity of f, g, Assumption 2.3 and Remark 2.1, we obtain
This shows that \(\alpha \) has the desired properties, and hence the proof is finished. \(\square \)
4 The Maximum Principle: Proof of Theorem 2.2
In this section, it will be assumed implicitly that assumptions 2.1, 2.2, 2.3 and 2.4 hold. Hereafter, we let \(({\tilde{\Omega }},\tilde{\mathcal {A}},{\tilde{{\mathbb {P}}}})\) be a copy of the probability space \((\Omega ,\mathcal {A},{\mathbb {P}})\). The corresponding expectation map will be denoted by \({{\tilde{{\mathbb {E}}}}}\).
4.1 Gâteaux Differentiability
In this subsection we aim to complete Lemma 3.2 by showing the Gâteaux-differentiability of the control-to-state operator
The Gâteaux derivative of the solution map will be given by the solution of a mean-field equation with random coefficients. We will deal with this problem in the similar fashion as it is done in [8, Thm. 6.10 p. 544].
Lemma 4.1
The solution map G is Gâteaux-differentiable. Moreover, for each \(\alpha \in {{\mathbb {A}}}\), its derivative in the direction \(\beta \in {{\mathbb {A}}}\) is given by
where, introducing
the process \(Z=Z^{\alpha ,\beta }\) is characterized as the unique solution to
Proof
We will start by showing that (38) has a unique solution. For that purpose, we define
where \(p_1\) denotes the projector onto the first d-coordinates, namely
Clearly, if \(\mu ^n_t\) is a sequence converging weakly to \(\mu _t\) for every \(t\in [0,T]\), the constraint \(\mu _t^n\circ p_1^{-1}={{\,\mathrm{\mathcal {L}}\,}}(X_t),\forall t\) remains true for \(\mu \) itself. Since the Wasserstein distance metrizes the weak topology, we see that \(\mathcal {R}\) is closed in \(C([0,T];{\mathcal {P}}_2({\mathbb {R}}^d\times {\mathbb {R}}^d))\). Next, define
which maps \(\mu \in C([0,T];\mathcal {P}_2({\mathbb {R}}^d\times {\mathbb {R}}^d))\) to \(({{\,\mathrm{\mathcal {L}}\,}}(X_t,V_t))_{t\in [0,T]}\), where \((V_t)_{t\in [0,T]}\) is the unique solution to
For fixed \(\mu \in C([0,T];\mathcal {P}_2({\mathbb {R}}^d\times {\mathbb {R}}^d))\) we first need to check the existence of a unique solution V. But letting
we have the following properties:
for all \(t\in [0,T]\) and \({\mathbb {P}}\)-almost every \(\omega \). The first estimate is a result of Assumption 2.4 and the fact that \({\mathbb {P}}(X_t\in {\mathcal {C}},\forall t)=1\). The second estimate follows from
together with the continuity of \(t\mapsto \iint _{{\mathbb {R}}^d\times {\mathbb {R}}^d}|y|\mu _t(dx\times dy),\) and the uniform estimate (32). Using (30) we get with similar arguments
for all \(t\in [0,T]\), \({\mathbb {P}}\)-almost every \(\omega \). It follows then by classical SDE results that (39) is well-posed. Moreover, adapting the arguments yielding the moment estimates of Theorem 3.1, it is shown mutatis mutandis that for \(2\le p\le r\)
Therefore \((V_t)\) (and hence \(\Psi (\mu )\equiv {{\,\mathrm{\mathcal {L}}\,}}(X,V)\)) is uniquely determined by the probability measure \(\mu \).
We now aim to prove that \(\Psi \) is a contraction, but for that purpose it is convenient to introduce another (stronger) metric. For any \(\mu ,\nu \in {\mathcal {P}}_2({\mathbb {R}}^d\times {\mathbb {R}}^d)\) with \(\mu \circ p_1^{-1}=\nu \circ p_1^{-1}\), we let
where \(\Lambda (\mu ,\nu )\) is the set of all probability measures m on \(({\mathbb {R}}^d)^3\) such that for any \(A,B\in {\mathcal {B}}({\mathbb {R}}^d)\)
That d is stronger than \(W_2\) can be seen as follows. If m is any element in \(\Lambda (\mu ,\nu )\), one can define
where \(\delta _x\) is the Dirac mass centered at x. Clearly, \(\rho \) belongs to the set of transport plans \(\Pi (\mu ,\nu )\) between \(\mu \) and \(\nu ,\) so that in particular
Then, taking the infimum over all such m yields our conclusion.
Next, let \(m\in \Lambda (\mu ,\nu )\). Using the marginal condition on m, we have
Thus,
Since m is arbitrary, we obtain
and a similar result can be shown for \(\Sigma _\mu \). Now, if we equip \({\mathcal {R}}\) with a metric \(\delta \) inherited from d, for instance \(\delta (\mu ,\nu ):=\sup _{t\in [0,T]}e^{-\gamma t}d(\mu _t,\nu _t)\) for \(\gamma >0\) large enough, the proof that \(\Psi \) is a contraction follows with simple arguments. Since it is similar to the proof of Theorem 3.1, we omit the details.
Let now \(\alpha ,\beta \in {{\mathbb {A}}}\) and \(\epsilon >0\) small enough, such that \(\alpha +\epsilon \beta \in {{\mathbb {A}}}\). By X we denote the solution of (7) with respect to \(\alpha \) and by \(X^\epsilon \) we denote the solution to (7) with respect to \(\alpha +\epsilon \beta \). Furthermore for \(\lambda \in [0,1]\) we introduce \(X^{\lambda ,\epsilon }:=X+\lambda (X^\epsilon -X)\) and \(\alpha ^{\lambda ,\epsilon }:=\alpha +\lambda \epsilon \beta \). Note that, since \(\pi \) is convex, we have
hence \(X_t^{\lambda ,\epsilon }\) is supported in \({\mathcal {C}}\).
Next, by Lemma 3.2 we get
Thus, we can conclude that \(X^{\lambda ,\epsilon }\underset{\epsilon \rightarrow 0}{\longrightarrow } X\) in \(L^2(\Omega ,C([0,T];{\mathbb {R}}^d))\), uniformly in \(\lambda \). By a simple Taylor expansion we get
where, given \(\varphi =\varphi (t,x,\mu ,\alpha )({{\tilde{x}}})\) we use the shorthand notation
with the convention that the last input is ignored whenever \(\varphi \) does not depend on the tilde variable. Similarly, we have
Thus, for \(\varDelta ^\epsilon _t:=\dfrac{X_t^\epsilon -X_t}{\epsilon }-Z_t^{\alpha ,\beta }\) we have
By Itô formula, (40) and Assumption 2.4, we get
By Young Inequality, Jensen Inequality and assumption 2.4 we have
Since \(\epsilon >0\) is chosen in a way that \(\alpha +\epsilon \beta \in {\mathbb {A}}\), we can conclude by the a priori bound (32) and the definition of \({\mathbb {A}}\), that
for some constant \(C(T,\Vert X_0\Vert _p)>0\) which does not depend on \(\epsilon \). By the Burkholder-Davis-Gundy inequality, Young and Jensen inequalities we arrive at
for a constant \(C>0\) which does not depend on \(\epsilon \) and
and \(I_4,I_5,I_6\) are analogues for \(\sigma \). We will only show \(I_1\rightarrow 0\) as \(\epsilon \rightarrow 0\), the other terms being handled by similar arguments. By assumption 2.4 we have
Furthermore we have for any \(p\le r\) that
is bounded from above by some constant that does not depend on \(\epsilon \) for \(\epsilon >0\) small enough. Since \(X^{\lambda ,\epsilon }\rightarrow X\) in \(L^2(\Omega ;C([0,T];{\mathbb {R}}^d))\), by the a-priori bound (32), the estimate \({\mathbb {E}}\left( \sup _{t\in [0,T]}|Z_t|^4\right) <\infty \), the continuity of \(b_x\) and the dominated convergence theorem, one concludes that \(I_1\rightarrow 0\) as \(\epsilon \rightarrow 0\). Similar arguments combined with Gronwall’s lemma finish the proof. \(\square \)
As an important consequence, we obtain the following formula for the Gâteaux derivative of the cost functional. Given Lemma 4.1, the next result is proven in the same way as it is done in [8] and thus omitted.
Corollary 4.1
The cost functional
is Gâteaux differentiable and its Gâteaux derivative at \(\alpha \in {{\mathbb {A}}}\) in direction \(\beta \in {{\mathbb {A}}}\) is given by
4.2 Maximum Principle
For the reader’s convenience, we now rewrite the adjoint equation (29) using Hamiltonian formalism. Recall that for \(x,y,p\in {\mathbb {R}}^d,\) \(q\in {\mathbb {R}}^{d\times m}\) \(\mu \in {\mathcal {P}}_2\) and \(\alpha \in A,\) we introduced the quantity
Thus, given a control \(\alpha \in {{\mathbb {A}}},\) one sees that the pair \((P,Q)\in \mathcal {S}^{2,d}\times \mathcal {H}^{2,d\times m}\) solves the adjoint equation if and only if for all \(t\in [0,T]\), \({\mathbb {P}}\)-almost surely
where \(({{\tilde{X}}},{{\tilde{P}}},{{\tilde{Q}}},{{\tilde{\alpha }}})\) is an independent copy of \((X,P,Q,\alpha )\) on the space \(({{\tilde{\Omega }}},\mathcal {{{\tilde{F}}}},\mathbb {{{\tilde{P}}}}).\)
Let us point out that the above coefficients fail to satisfy [8, Assumption MKV BSDE, Chap. 4]. Hence, we first need to address the solvability of the BSDE (41) under the assumptions of Theorem 2.2.
Lemma 4.2
Under the assumptions of Theorem 2.2, there exists a unique solution \((P,Q)\in \mathcal {S}^2\times \mathcal {H}^{2,d\times m}\) of (41).
Proof
Fix \(\alpha \in {{\mathbb {A}}}\) and for simplicity, denote by \(H_x(t,\omega ,p,q):=H_x(t,X_t(\omega ),\mathcal {L}(X_t),p,q,\alpha _t)\) and by \(H_\mu (t,\omega ,x,p,q):=H_\mu (t,x,\mathcal {L}(X_t), p,q,\alpha _t)(X_t(\omega )).\) Consider the map \(\Gamma :\mathcal {H}^{2,d}\times \mathcal {H}^{2,d\times m}\rightarrow \mathcal {H}^{2,d}\times \mathcal {H}^{2,d\times m}\) which maps a given pair
to the solution (P, Q) of
where the expectation is to be understood in the following way:
In the following we drop the dependence on \(\omega \) for \(H_\mu \).
Since the above equation is a standard backward SDE with monotone coefficients, the existence of a solution is well-known by standard results. We will now show that the map \(\Gamma \) is a contraction, when the space \(\mathcal {H}^{2,d}\times \mathcal {H}^{2,d\times m}\) is equipped with the norm
for a sufficiently large parameter \(\gamma >0\). If we denote by \((P^1,Q^1),(P^2,Q^2)\) two solutions of (42) for \((Y^1,Z^1)\) and \((Y^2,Z^2)\) respectively, then by the backward Itô Formula [20, p. 356] applied to \(e^{\gamma t}|P^1_t-P^2_t|^2\) we get
From assumptions 2.4,2.4, Young’s inequality and Lemma 3.1, we infer that
and
Invoking (32), Cauchy-Schwarz and Young Inequalities, we can conclude that
For \(\gamma \) large enough this leads to
showing that \(\Gamma \) is a contraction. The conclusion follows. \(\square \)
The following corollary follows immediately by integration by parts and an application of Fubini Theorem. We therefore omit the proof and refer to [8, Lemma 6.12 p. 547].
Corollary 4.2
Let (P, Q) be a solution to (41), then it holds
Remark 4.1
An immediate consequence of (44) is the following formula for the Gâteaux derivative of the cost functional
An application of Fubini Theorem then leads to the following representation for the gradient of J:
It is hardly necessary to mention that the formula (45) is of fundamental importance for numerical purposes, see Sect. 5 below.
We are now in position to prove the maximum principle.
Proof of Theorem 2.2
Let \(\alpha \in {\mathbb {A}}\) be an optimal control for (SM), X the corresponding solution to (7) and (P, Q) the associated solution to (41). For \(\beta \in {\mathbb {A}}\) we have by the optimality of \(\alpha \)
Invoking the convexity of the Hamiltonian (see Assumption 2.3), we get
For any arbitrary measurable set \(C\subset [0,T]\) and \({{\tilde{\alpha }}}\in A\) we can define the admissible control
hence
Therefore we get
dt-almost everywhere. This proves the theorem. \(\square \)
5 Numerical Examples
In this section we focus on the FitzHugh–Nagumo model with external noise only, i.e. the system of 3N stochastic differential equations:
where we recall that \(S(v):= T_{max}/[1+\exp \lambda (v-V_T)].\)
We are interested in controlling the average membrane potential (called in the following “local field potential”) of a network of FitzHugh–Nagumo neurons into a desired state. Our cost functional is given by
where \(({\overline{v}}_t)_t\) is a certain reference profile. We should mention that the average membrane potential will only give an idea about the average activity of the network at each time. For example a high average membrane potential is an indication that a high number of neurons are in the regenerative or active phase, while a low average membrane potential means that a high number of neurons are in the absolute refractory or silent phase.
In the described case the adjoint equation is reduced to
In the following section we will give a short introduction on how to solve (48) numerically.
5.1 Numerical Approximation of the Adjoint Equation
In general we consider the following non fully coupled MFFBSDE
For the approximation of the forward component we consider an implicit Euler scheme for McKean-Vlasov equations. Since this is standard, we will not go into further details. Concerning the backward component, we consider a scheme similar to the one presented in [10]. We should mention that since we are not dealing with a fully coupled MFFBSDE, our situation is a lot easier to handle than the one treated in [10]. For a given discrete time grid \(\pi :0=t_0<t_1<...<t_N=T\), we consider the following numerical scheme:
For the approximation of the conditional expectation, we make use of the decoupling field mentioned in [8], to write
Thus we can represent the conditional expectation in terms of a function \({\tilde{u}}\) by
We approximate \({\tilde{u}}(t_{k+1},\cdot )\) with gaussian radial basis functions, by solving the following minimization problem for fixed nodes \(x_1,...,x_L\):
for \(\alpha =(\alpha _1(t_{k+1}),...,\alpha _L(t_{k+1}))^\dagger \), where \(\delta >0\) and \(L\in {\mathbb {N}}\) are fixed. Therefore we initialize our reference points \(x_1,...,x_L\) by L independent realizations of \(X_{t_k}^\pi \). For m realizations of \(Y_{t_{k+1}}^{\pi }\) and \(X_{t_k}^\pi \), denoted by \(y_{t_{k+1}}^1,...,y_{t_{k+1}}^m\) and \(x_{t_{k+1}}^1,...,x_{t_{k+1}}^m\) respectively, we then write
Thus we need to minimize
A similar approach for BSDEs can be found in [18]. There is no convergence analysis of this scheme for our assumptions on the coefficients, this should only give an idea how to solve the adjoint equation in practice. Furthermore we should mention, that in the case where only external noise is present, the duality (44) and the resulting gradient representation still holds true for any non adapted solution of (41), i.e. any solution to the random backward ODE arising from equation (41) for \(Q\equiv 0\). Thus one can also implement a numerical scheme for the adjoint equation, without any conditional expectations involved. For more general diffusion coefficient however this is not true, since the proof of the duality is based on integration by parts and the stochastic integral which appears is not defined if the integrand is not adapted.
5.2 Gradient Descent Algorithm
We will now briefly sketch our gradient descent algorithm.
Algorithm 5.1
Take an initial control \(\alpha _0\in {\mathbb {A}}\), \(s_0>0\), and recursively for \(n=0,1,\dots :\)
-
(1)
determine \(X^{\alpha _n}\) by solving the state equation with an implicit particle scheme to avoid particle corruption;
-
(2)
solve the adjoint equation for given \(X^{\alpha _n}\) in order to approximate \((P^{\alpha _n},Q^{\alpha _n})\);
-
(3)
approximate the gradient
$$\begin{aligned} \nabla J(\alpha _n)_s ={\mathbb {E}}\Big [\langle b_\alpha (s,X_s^{\alpha _n},{{\,\mathrm{\mathcal {L}}\,}}(X^{\alpha _n}_s),\alpha _s^n),P^{\alpha _n}_s\rangle + f_\alpha (s,X_s^{\alpha _n},{{\,\mathrm{\mathcal {L}}\,}}( X_s^{\alpha _n}),\alpha ^n_s) \Big ] \end{aligned}$$via Monte-Carlo method, where \((P^{\alpha _n},Q^{\alpha _n})\) solves the adjoint equation;
-
(4)
update the step size \(s_n\) according to a suitable step size rule (e.g. Armijo-rule), repeating step 5.1–5.1 if necessary
-
(5)
update the control in direction of the steepest descent: \(\alpha _{n+1}:=\alpha _n -s_n\nabla J(\alpha _n)\);
-
(6)
the algorithm stops if \(\Vert \nabla J(\alpha _n)\Vert <\epsilon \)
To compute the expectation term, one is in fact reduced to simulate the solution of the network equation itself and use the particles as samples for the Monte-Carlo simulation.
5.3 Numerical Examples for Systems of FitzHugh–Nagumo Neurons
Although the solution to the adjoint equation is a 3-dimensional process, in the following we will only plot its first variable, since the other variables are irrelevant for the gradient in our situation.
To illustrate some problems we had with the simulations, we consider the example of the deterministic uncoupled case of equation (46), where \({\overline{J}}=0\) and \(\sigma _{ext}=0\). In the given situation the membrane potential v becomes highly sensitive to small perturbations of the control at specific times, when we chose the control \(\alpha _t\equiv \alpha \) close to the bifurcation value \(\alpha _c\approx 0.33\) for the supercritical Hopf bifurcation point of the equation, where the fixed point is stable, but there is no periodic orbit, in particular we chose \(\alpha =0.3251<\alpha _c\approx 0.33\). The bifurcation point of the deterministic system can be determined in a similar way as it is done in [23]. This sensitivity can lead to high valued solutions of the corresponding adjoint equation for specific reference profiles. One example is to choose the reference profile as the v-trajectory of a solution to (46), for a control parameter \(\alpha \) in the limit cycle regime. This situation is illustrated by the Figs. 2, 3 and 4.
Membrane potential of the solution to (46) for \(\alpha \equiv 0.3251\)
Reference profile generated by solving (46) for \(\alpha \equiv 0.332\)
Solution to the corresponding adjoint equation
The same type of phenomena also occurs in the case of the coupled system of stochastic FitzHugh–Nagumo neurons. Here it can lead to high fluctuations of the sample mean for the adjoint equation, thus a high number of particles is required to compute the expectation of the solution to the adjoint equation. A small illustration is given by the Figs. 5, 6 and 7.
Local field potential of the solution to (46) for \(\alpha \equiv 0\)
Reference profile chosen as the local field potential of (46) for \(\alpha (t)= 0.8\) if \(t\le 7\)
Samples of the solution to the adjoint equation
In this example and in the following, the initial states are uniformly distributed on the orbit of a solution to (46) with \(\alpha \equiv 0\), \(\sigma _{ext}=0\) and initial conditions \(V_0=-0.828,w_0=-0.139,y_0=0.589\). The other parameters are given below in Table 1. In our example only external noise is present, so we can approximate the non adapted solution to the BSDE using the simple Euler-scheme with no conditional expectations involved, replacing \({\mathbb {E}}\left( Y_{t_{k+1}}^\pi |\mathcal {F}_{t_{k}}\right) \) by \(Y_{t_{k+1}}^\pi \) in our numerical scheme, to get a good approximation of the gradient. For the sake of completeness we mention that for the approximation of the adapted solution we would use the parameter \(L=200\) and \(\delta =2\). Furthermore we are always using \(N=1000\) particles for the particle approximation of (46).
5.3.1 Control of a Coupled System of FitzHugh–Nagumo Neurons
For our first example, we consider a parameter regime where the activity of a large number of neurons of the network at some time t leads to further activity at a later time, without any external current applied to the system. Therefore we slow down the gating variable, by decreasing the closing rate of the synaptic gates. This way its impact on the network is still high enough, when a large part of the network is excitable again. Figure 8 shows the uncontrolled local field potential in this case (i.e. when \(\alpha \equiv 0\)).
Our goal is now to increase the activity of the network up to time \(t=100\) and then control the network back into its resting potential. Up to time \(t=100\), the reference profile showed in Fig. 9 shows the local field potential of a network of coupled FitzHugh–Nagumo neurons, when a constant input current of magnitude 0.8 is applied for a time period of \(\Delta t=7\) at \(t=0\). For times \(t>100\) it shows the resting potential of a single FitzHugh–Nagumo neuron.
Uncontrolled local field potential
Reference profile
We expect the optimal control to raise the membrane potential for a small time period at \(t=0\) and then counteract the stimulating effect of the coupling around \(t=100\). However this effects should not occur in the uncoupled setting, which we will consider afterwards.
Figures 10 and 11 show the optimal control and the corresponding optimal local field potential. We remind that this might only be locally optimal, since we cannot expect to find a globally optimal control with our gradient descent algorithm.
Optimal control
Local field potential with optimal control
Since our terminal cost is always zero, i.e. \(g\equiv 0\), the solution to the adjoint equation will always be zero at terminal time T. Consequently our gradient will always be zero at time T and our gradient descent algorithm does not change the control at time T. That is why the value of our approximated optimal control will stay at the value of our initial control \(\alpha _0\) at the terminal time. We started with initial control \(\alpha _0\equiv 0\), which explains the small peak in Figs. 10 and 11.
5.3.2 Control of an Uncoupled System of FitzHugh–Nagumo Neurons
Now we investigate the control problem for the uncoupled equation (46), where \(J=0\). Since the reference profile it still the same as in example 5.3.1, we will only present the corresponding optimal control (Fig. 12).
Optimal control
As expected, the control does not need to counteract any stimulating effects for times \(t>100\). Furthermore it is not sufficient in the uncoupled case to apply an input current for a small time period at \(t=0\), to reach the desired local field potential up to time \(t=100\) (Table 1).
References
Albi, G., Choi, Y.-P., Fornasier, M., Kalise, D.: Mean field control hierarchy. Appl. Math. Optim. 76(1), 93–135 (2017)
Baladron, J., Fasoli, D., Faugeras, O., Touboul, J.: Mean-field description and propagation of chaos in networks of Hodgkin-Huxley and Fitzhugh-Nagumo neurons. J. Math. Neurosci. 2(1), 1–50 (2012)
Barbu, V., Cordoni, F., Di Persio, L.: Optimal control of stochastic Fitzhugh-Nagumo equation. Int. J. Control 89(4), 746–756 (2016)
Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory, vol. 101. Springer, Berlin (2013)
Frédéric Bonnans, J., Hadikhanloo, S., Pfeiffer, L.: Schauder estimates for a class of potential mean field games of controls. Appl. Math. Optim. 83, 1431–1464 (2019)
Bossy, M., Faugeras, O., Talay, D.: Clarification and complement to mean-field description and propagation of chaos in networks of Hodgkin-Huxley and Fitzhugh-Nagumo neurons. J. Math. Neurosci. 5(1), 1–23 (2015)
Buckdahn, R., Li, J., Ma, J.: A stochastic maximum principle for general mean-field systems. Appl. Math. Optim. 74(3), 507–534 (2016)
Carmona, R., Delarue, F.: Probabilistic Theory of Mean Field Games with Applications I-II. Springer, Berlin (2018)
Carmona, R., Delarue, F., Lachapelle, A.: Control of Mckean-Vlasov dynamics versus mean field games. Math. Financ. Econ. 7(2), 131–166 (2013)
Chassagneux, J., Crisan, D., Delarue, F.: Numerical method for FBSDE of McKean-Vlasov type. Ann. Appl. Probab. 29(3), 1640–1684 (2019)
Cordoni, F., Di Persio, L.: Optimal control for the stochastic Fitzhugh-Nagumo model with recovery variable. Evol. Equ. Control Theory 7(4), 571 (2018)
Deco, G., Rolls, E.T., Romo, R.: Stochastic dynamics as a principle of brain function. Prog. Neurobiol. 88(1), 1–16 (2009)
Destexhe, A., Mainen, Z.F., Sejnowski, T.J.: Synthesis of models for excitable membranes, synaptic transmission and neuromodulation using a common kinetic formalism. J. Comput. Neurosci. 1(3), 195–230 (1994)
Dos Reis, G., Salkeld, W., Tugaut, J.: Freidlin-Wentzell ldp in path space for Mckean-Vlasov equations and the functional iterated logarithm law. Ann. Appl. Probab. 29(3), 1487–1540 (2019)
Dudley, R.M.: Real Analysis and Probability. Chapman and Hall/CRC, London (2018)
Ermentrout, B., Terman, D.: Mathematical Foundations of Neurosciences, vol. 35. Springer, Berlin (2010)
FitzHugh, R.: Impulses and physiological states in theoretical models of nerve membrane. Biophys. J. 1(6), 445–466 (1961)
Kebiri, O., Neureither, L., Hartmann, C.: Adaptive importance sampling with forward-backward stochastic differential equations. In: International Workshop on Stochastic Dynamics Out of Equilibrium, pp. 265–281. Springer, Berlin (2017)
Nagumo, J., Arimoto, S., Yoshizawa, S.: An active pulse transmission line simulating nerve axon. Proc. IRE 50(10), 2061–2070 (1962)
Pardoux, É., Răşcanu, A.: Stochastic Differential Equations, Backward SDEs, Partial Differential Equations. Springer, Berlin (2014)
Pfeiffer, L.: Optimality conditions for mean-field type optimal control problems. SFB Rep. 15, 2015–015 (2015)
Pfeiffer, L.: Numerical methods for mean-field-type optimal control problems. arXiv preprint arXiv:1703.10001 (2017)
Phan, V.L.E.: A study of fixed points and Hopf bifurcation of Fitzhugh-Nagumo model. Can Tho Univ. J. Sci. 54(2), 112–121 (2018)
Schwalger, T., Deger, M., Gerstner, W.: Towards a theory of cortical columns: from spiking neurons to interacting neural populations of finite size. PLoS Comput. Biol. 13(4), e1005507 (2017)
Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations, vol. 43. Springer, Berlin (1999)
Acknowledgements
This work has been funded by Deutsche Forschungsgemeinschaft (DFG) through Grant CRC 910 “Control of self-organizing nonlinear systems: Theoretical methods and concepts of application”, Project (A10) “Control of stochastic mean-field equations with applications to brain networks.” Both authors are thankful to Wilhelm Stannat, Tilo Schwalger and François Delarue for helpful discussions. The anonymous referees are warmly thanked for having helped to improve this manuscript.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
Hocquet, A., Vogler, A. Optimal Control of Mean Field Equations with Monotone Coefficients and Applications in Neuroscience. Appl Math Optim 84 (Suppl 2), 1925–1968 (2021). https://doi.org/10.1007/s00245-021-09816-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-021-09816-1
Keywords
- Optimal control
- McKean–Vlasov equations
- FitzHugh–Nagumo neurons
- Stochastic differential equations
- Gradient descent