Optimal Control of Stochastic Processes

Abstract

Many control problems appearing for complex systems are subject to imperfectly known disturbances. As we have learned in the previous chapter, these disturbances originate by the hidden irrelevant degrees of freedom. Although we may formally separate the dynamics of relevant and irrelevant variables into different contributions to the equations of motion, see Sect. 1, the timedependence of the terms concerning the irrelevant degrees of freedom is still open. Unfortunately, this problem cannot solved generally, although the Mori–Zwanzig equations define a formal expression for the residual forces (6.123). To proceed, it is necessary to make some approximations about these existing, but usually ‘a priori’ unknown forces. The simplest, but most common approach is the Markov approximation discussed in Sects. 6.6 and 6.9.2. This case allows the introduction and application of the very powerful Ito calculus or alternatively the use of Fokker–Planck equations in order to describe the dynamics of the relevant variables driven by external random sources simulating the effects of the irrelevant degrees of freedom. A large number of physical processes in complex systems is sufficiently described by the Markov approach. As a standard problem we refer to the Brownian motion, originally observed by Ingenhousz and Brown [1] and first described by Einstein [2]. Therefore, we focus in the following mainly on Markov processes.