How Robust is Robust Control in Discrete Time?

Abstract

By applying robust control, the decision maker wants to make good decisions when his model is only a good approximation of the true one. Such decisions are said to be robust to model misspecification. In this paper it is shown that, in many situations relevant in economics, a decision maker applying robust control implicitly assumes that today’s worst-case adverse shock is serially uncorrelated with tomorrow’s worst-case adverse shock. Then, further investigation is needed to see how strong is the ‘immunization against uncertainty’ provided by these popular frameworks.

1 Introduction

A characteristic “feature of most robust control theory”, observes Bernhard (2002, p. 19), “is that the a priori information on the unknown model errors (or signals) is nonprobabilistic in nature, but rather is in terms of sets of possible realizations. Typically, though not always, the errors are bounded in some way…. As a consequence, robust control aims at synthesizing control mechanisms that control in a satisfactory fashion (e.g., stabilize, or bound, an output) a family of models”.¹ Then “standard control theory tells a decision maker how to make optimal decisions when his model is correct (whereas) robust control theory tells him how to make good decisions when his model approximates a correct one” (Hansen and Sargent 2007a, p. 25). In other words, by applying robust control the decision maker makes good decisions when it is statistically difficult to distinguish between his approximating model and the correct one using a time series of moderate size. “Such decisions are said to be robust to misspecification of the approximating model” (Hansen and Sargent 2007a, p. 27).²

Tucci (2006, p. 538) argues that “the true model in Hansen and Sargent (2007a) … is observationally equivalent to a model with a time-varying intercept.” In the sense that, unless some prior information is available, it is impossible to distinguish between the two models by simply observing the output. Then he goes on showing that, when the same worst-case adverse shock and objective functional are used in both procedures, robust control is identical to the optimal control associated with time-varying parameters, or TVP-control, only when the transition matrix in the law of motion of the parameters is zero. He concludes that this decision maker implicitly assumes that today’s worst-case adverse shock is serially uncorrelated with tomorrow’s worst-case adverse shock.

This is a relevant conclusion because it applies to a robust control set up widely used in economics. Moreover, as commonly understood, the robust control choice accounts for all possible kinds of persistence of worst-case shocks, which may take a very general form. Then, it is not immediately obvious why they look linearly independent when the decision maker cares only of the induced distributions under the approximating model and is indifferent between utility processes with identical induced distributions. Namely, when He/She is assumed having preferences defined by using a single constraint, or penalty, on the adverse shocks.

At this stage however, it is unclear if this result holds when a more general framework is considered. For instance, when the decision maker has a different constraint for each type of adverse shocks. This may be the case when He/She is looking for decisions robust to perturbations in a situation where parts of the state vector are unobservable. Then two types of statistical perturbations are considered. One that distorts the adopted model conditional on the knowledge of hidden state and the other that distorts the distribution of the hidden state. This decision maker is sometimes referred to as the non-“probabilistically sophisticated” decision maker and contrasted with the “probabilistically sophisticated” decision maker, who considers only one kind of perturbation, described above.³ Alternatively robust control may be applied to situations where the decision maker wants to be “immunized against uncertainty” related to unknown structural parameters as in Giannoni (2002, 2007).⁴

The goal of this paper is to carry on the comparison between TVP-control and robust control for a much larger class of models in discrete-time.⁵ Namely, the case of a non-“probabilistically sophisticated” decision makers who want to make decisions robust with respect to unstructured uncertainty à la Hansen and Sargent, i.e. a nonparametric set of additive mean-distorting model perturbations. And the class of models where uncertainty is related to unknown structural parameters. This is a necessary step to determine if Tucci’s (2006) result holds only in the simplest case or is valid for a much larger class of models. In the former case it may be treated as an interesting special case of limited, or no, practical relevance because it does not affect the most commonly used robust control frameworks. In the latter it is a clear indication that further investigation, outside the scope of the present work, is needed to see how large is the uncertainty set associated with these frameworks. In other words, how strong is the ‘immunization against uncertainty’ provided by the linear-quadratic robust control set up widely used in economics in discrete-time.⁶

The remainder of the paper is organized as follows. Section 2 reviews the simplest robust control problem with unstructured uncertainty à la Hansen and Sargent. An example of a non-“probabilistically sophisticated” decision maker is discussed in Sect. 3. In Sect. 4 both problems are reformulated as linear quadratic tracking control problems where the system equations have a time-varying intercept following a mean reverting, or ‘Return to Normality’, model and the associated TVP-controls are derived. Then the optimizing model for monetary policy used in Giannoni (2002, 2007) is presented (Sect. 5). Section 6 reports some numerical results and the main conclusions are summarized in Sect. 7. For the reader’s sake, the major result of each section is stated as a proposition and its proof confined to the “Appendix”.

2 Robust Control à la Hansen and Sargent: The Standard Case

Hansen and Sargent (2007a, p. 140) consider a decision maker “who has a unique explicitly specified approximating model but concedes that the data might actually be generated by an unknown member of a set of models that surround the approximating model”.⁷ Then the linear system

$${\mathbf{y}}_{t + 1} = {\mathbf{Ay}}_{t} { + }{\mathbf{Bu}}_{t} { + }{\mathbf{C}}{\varvec{\upvarepsilon }}_{t + 1} \quad {\text{for}}\quad t = 0, \ldots ,\infty ,$$

(1)

with y_t the n × 1 vector of state variables at time t, u_t the m × 1 vector of control variables and ε_t+1 an l × 1 identically and independently distributed (iid) Gaussian vector process with mean zero and an identity contemporaneous covariance matrix, is viewed as an approximation to the true unknown model

$${\mathbf{y}}_{t + 1} = {\mathbf{Ay}}_{t} { + }{\mathbf{Bu}}_{t} { + }{\mathbf{C}}({\varvec{\upvarepsilon }}_{t + 1} + {\varvec{\upomega }}_{t + 1} )\quad {\text{for}}\quad t = 0, \ldots ,\infty .$$

(2)

The matrices of coefficients A, B and C are assumed known and y₀ given.⁸

In Eq. (2) the vector ω_t+1 denotes an unknown l × 1 process, that can feed back in a possibly nonlinear way on the history of y, and is introduced because the iid random process ε_t+1 can represent only a very limited class of approximation errors. In particular it cannot depict the kind of misspecified dynamics characterizing models with nonlinear and time-dependent feedback of y_t+1 on past states.⁹ To express the idea that (1) is a good approximation of (2) the ω’s are restrained by

$$E_{0} \left[ {\sum\limits_{t = 0}^{\infty } {\beta^{t + 1} {\varvec{\upomega }}_{t + 1}^{{\prime }} {\varvec{\upomega }}_{t + 1} } } \right] \le \eta_{0} \quad {\text{with}}\quad 0 <\beta < 1$$

(3)

where E₀ denotes mathematical expectation evaluated with respect to model (2) and conditioned on y₀ and η₀ measures the set of models surrounding the approximating model.¹⁰

The decision maker’s looking for good decisions over a set of models (2) satisfying (3) is indeed solving a constraint problem or a multiplier problem. The constraint robust control problem is defined as¹¹

$$\mathop {\hbox{max} }\limits_{{\mathbf{u}}} \, \mathop {\hbox{min} }\limits_{{\varvec{\upomega }}} - E_{0} \left[ {\sum\limits_{t = 0}^{\infty } {\beta^{t} } r({\mathbf{y}}_{t} ,{\mathbf{u}}_{t} )} \right],$$

(4)

with r(y_t, u_t) the one-period loss function, subject to (2)–(3) where η^* > η₀ and η^* measures the largest feasible set of perturbations. The multiplier robust control problem is formalized as

$$\mathop {\hbox{max} }\limits_{{\mathbf{u}}} \, \mathop {\hbox{min} }\limits_{{\varvec{\upomega }}} \, - E_{0} \left\{ {\sum\limits_{t = 0}^{\infty } {\beta^{t} } \left[ {r({\mathbf{y}}_{t} ,{\mathbf{u}}_{t} ) - \, \theta \beta {\varvec{\upomega }}_{t + 1}^{{\prime }} {\varvec{\upomega }}_{t + 1} } \right]} \right\}$$

(5)

subject to (2) with θ, 0 < θ^* < θ ≤ ∞, a penalty parameter restraining the minimizing choice of the {ω_t+1} sequence. The “breakdown point” θ^*represents a lower bound on θ needed to keep the objective of the two-person zero-sum game convex in ω_t+1 and concave in u_t.¹² Both problems can be reinterpreted as two-player zero-sum games where one player is the decision maker maximizing the objective functional by choosing the sequence for u and the other player is a malevolent nature choosing a feedback rule for a model-misspecification process ω to minimize the same criterion function.¹³ For this reason, the constraint and the multiplier robust control problem are also referred to as the constraint and multiplier game, respectively.

Hansen and Sargent (2007a, p. 139) notice that if the parameters η₀ and θ are appropriately related the two “games have equivalent outcomes.” Equivalent in the sense that if there exists a solution u^*, ω^* to the multiplier robust control problem, that u^* also solves the constraint robust control problem with $\eta_{0} = \eta_{0}^{*} =E_{0} [\sum\nolimits_{t = 0}^{\infty } {\beta^{t + 1} {\varvec{\upomega }}_{t + 1}^{{*{\prime }}} {\varvec{\upomega }}_{t + 1}^{*} } ]$.¹⁴ Then, in Appendix C of Ch. 7, two sets of formulae to compute the robust decision rule are provided and it is pointed out that the Riccati equation for the robust control problem (5) looks like the Riccati equation for an ordinary optimal linear regulator problem (also known as the linear quadratic control problem) with controls $({\mathbf{u^{\prime}}}_{t} \, {\varvec{\upomega}}^{\prime}_{t + 1} )^{\prime}$ and penalty matrix defined as $diag({\mathbf{R}}, - \beta \theta {\mathbf{I}}_{l} )$.¹⁵

Therefore the robust rules for u_t and the worst-case shock ω_t+1 can be directly computed from the associated ordinary linear regulator problem. In particular, when the one-period loss function r(y_t, u_t) is specified as¹⁶

$$\left( {{\mathbf{y}}_{t} - {\tilde{\mathbf{y}}}_{t}^{d} } \right)^{{\prime }} {\mathbf{Q}}\left( {{\mathbf{y}}_{t} - {\tilde{\mathbf{y}}}_{t}^{d} } \right) + 2\left( {{\mathbf{y}}_{t} - {\tilde{\mathbf{y}}}_{t}^{d} } \right)^{{\prime }} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{W} }}\left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right) + \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right)^{{\prime }} {\mathbf{R}}\left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right),$$

(6)

with Q a positive semi-definite matrix, R a positive definite matrix, W an n × m array, ${\mathbf{y}}_{t}^{d}$ and ${\mathbf{u}}_{t}^{d}$ the desired values of the states and controls, respectively, for period t, the robust control rule is derived by extremizing, i.e. maximizing with respect to u_t and minimizing with respect to ω_t+1, the objective function¹⁷

$$- E_{0} \left[ {\sum\limits_{t = 0}^{\infty } {\beta^{t} r({\mathbf{y}}_{t} ,{\tilde{\mathbf{u}}}_{t} )} } \right]$$

(7)

with

$$r({\mathbf{y}}_{t} ,{\tilde{\mathbf{u}}}_{t} ) = \left( {{\mathbf{y}}_{t} - {\mathbf{y}}_{t}^{d} } \right)^{{\prime }} {\mathbf{Q}}\left( {{\mathbf{y}}_{t} - {\mathbf{y}}_{t}^{d} } \right) + 2\left( {{\mathbf{y}}_{t} - {\mathbf{y}}_{t}^{d} } \right)^{{\prime }} {\tilde{\mathbf{W}}}\left( {{\tilde{\mathbf{u}}}_{t} - {\tilde{\mathbf{u}}}_{t}^{d} } \right) + \left( {{\tilde{\mathbf{u}}}_{t} - {\tilde{\mathbf{u}}}_{t}^{d} } \right)^{{\prime }} {\tilde{\mathbf{R}}}\left( {{\tilde{\mathbf{u}}}_{t} - {\tilde{\mathbf{u}}}_{t}^{d} } \right)$$

(8)

subject to

$${\mathbf{y}}_{t + 1} = {\mathbf{Ay}}_{t} + {\mathbf{\tilde{B}\tilde{u}}}_{t} + {\mathbf{C}}{\varvec{\upvarepsilon }}_{t + 1} \quad {\text{for}}\quad t = \, 0, \ldots ,\infty$$

(9)

where¹⁸

$${\tilde{\mathbf{R}}} = \left[ {\begin{array}{*{20}c} {\mathbf{R}} & {\mathbf{O}} \\ {\mathbf{O}} & -{ \beta \theta {\mathbf{I}}_{l} } \\ \end{array} } \right],\quad {\tilde{\mathbf{u}}}_{t} = \left[ {\begin{array}{*{20}c} {{\mathbf{u}}_{t} } \\ {{\varvec{\upomega }}_{t + 1} } \\ \end{array} } \right] ,\quad {\tilde{\mathbf{B}}} = \left[ {\begin{array}{*{20}c} {\mathbf{B}} & {\mathbf{C}} \\ \end{array} } \right],\quad {\tilde{\mathbf{u}}}^{d}_{t} = \left[ {\begin{array}{*{20}c} {{\mathbf{u}}_{t}^{d} } \\ {\mathbf{0}} \\ \end{array} } \right]$$

(10)

and ${\tilde{\mathbf{W}}} = [\begin{array}{*{20}c} {\mathbf{W}} & {\mathbf{O}} \\ \end{array} ]$ with O and 0 null arrays of appropriate dimension.

At this point the following result can be stated:

Proposition 1

Extremizing the objective function (7) subject to (9), with definitions as in (8) and (10), yields the θ-constrained worst-case control for the decision maker¹⁹

$${\mathbf{u}}_{t} = - \left( {{\mathbf{R}}_{t} + {\mathbf{B}}^\prime {\mathbf{P}}_{t + 1}^{*} {\mathbf{B}}} \right)^{ - 1} \left[ {\left( {{\mathbf{B}}^\prime {\mathbf{P}}_{t + 1}^{*} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} } \right){\mathbf{y}}_{t} + {\mathbf{B}}^\prime {\mathbf{p}}_{t + 1}^{*} + {\mathbf{r}}_{t} } \right]$$

and for the malevolent nature²⁰

$$\begin{aligned} {\varvec{\upomega }}_{t + 1} & = \left( {\beta \theta {\mathbf{I}}_{l} - {\mathbf{C}}^\prime {\mathbf{P}}_{t + 1} {\mathbf{C}}} \right)^{ - 1} {\mathbf{C}}^\prime {\mathbf{P}}_{t + 1} \left\{ {\left[ {{\mathbf{A}} - {\mathbf{B}}\left( {{\mathbf{R}}_{t} + {\mathbf{B}}^\prime {\mathbf{P}}_{t + 1}^{*} {\mathbf{B}}} \right)^{ - 1} \left( {{\mathbf{B}}^\prime {\mathbf{P}}_{t + 1}^{*} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} } \right)} \right]{\mathbf{y}}_{t} } \right. \\& \quad \left. { - {\mathbf{B}}\left( {{\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1}^{*} {\mathbf{B}}} \right)^{ - 1} \left( {{\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1}^{*} + {\mathbf{r}}_{t} } \right) + {\mathbf{P}}_{t + 1}^{ - 1} {\mathbf{p}}_{t + 1} } \right\} \\ \end{aligned}$$

with²¹

$$\begin{aligned} {\mathbf{P}}_{t + 1}^{*} & ={\mathbf{P}}_{t + 1} { + }{\mathbf{P}}_{t + 1} {\mathbf{C}} (\beta \theta {\mathbf{I}}_{l} - \, {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{C}} )^{ - 1} {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} \\ {\mathbf{p}}_{t + 1}^{*} & =\left[ { \, {\mathbf{I}}_{n} { + }{\mathbf{P}}_{t + 1} {\mathbf{C}} (\beta \theta {\mathbf{I}}_{l} - \, {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{C}} )^{ - 1} {\mathbf{C}}^{\prime} } \right]{\mathbf{p}}_{t + 1} . \\ \end{aligned}$$

where the “robust” Riccati arrays ${\mathbf{P}}_{t + 1}^{*}$ and ${\mathbf{p}}_{t + 1}^{*}$ are always greater or equal to P_t+1 and p_t+1, respectively, because it is assumed that, in the “admissible” region, the parameter θ is large enough to make $(\beta \theta {\mathbf{I}}_{l} - \, {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{C}} )$ positive definite.²² They are equal when θ = ∞.²³

3 Robust Filtering Without Commitment

The previous section has considered the case where the decision maker is “probabilistically sophisticated” in the sense that He/She is indifferent between utility processes with identical induced distributions. However robust control can be applied also to situations where there are multiple penalty functions (i.e. more than one θ), in other words cases where the decision maker is not “probabilistically sophisticated.” This is the case when the decision maker does not observe parts of the state useful to forecast relevant variables. Then the approximating model includes an ordinary (i.e. non robust) Kalman filter estimator of this hidden portion of the state. To obtain “decision rules that are robust with respect to perturbations of the conditional distributions associated with the approximating model, the decision maker imagines a malevolent agent who perturbs the distribution of future states conditional on the entire state as well as the distribution of the hidden state conditional on the history of signals” (Hansen and Sargent 2007a, p. 383). This is sometimes referred to as the “robust filtering without commitment” problem.

The law of motion for the states in the approximating model is

$${\mathbf{y}}_{t + 1} = {\mathbf{Ay}}_{t} + {\mathbf{Bu}}_{t} + {\mathbf{C}}{\varvec{\upvarepsilon }}_{t + 1}$$

(11)

with

$${\mathbf{y}}_{t + 1} = \left[ {\begin{array}{*{20}c} {{\mathbf{y}}_{1} } \\ {{\mathbf{y}}_{2} } \\ \end{array} } \right]_{t + 1} ,\quad {\mathbf{A}}=\left[ {\begin{array}{*{20}c} {{\mathbf{A}}_{11} } & {{\mathbf{A}}_{12} } \\ {{\mathbf{A}}_{21} } & {{\mathbf{A}}_{22} } \\ \end{array} } \right],\quad {\mathbf{B}} = \left[ {\begin{array}{*{20}c} {{\mathbf{B}}_{1} } \\ {{\mathbf{B}}_{2} } \\ \end{array} } \right],\quad {\mathbf{C}} = \left[ {\begin{array}{*{20}c} {{\mathbf{C}}_{1} } \\ {{\mathbf{C}}_{2} } \\ \end{array} } \right],\quad {\varvec{\upvarepsilon }}_{t + 1} =\left[ {\begin{array}{*{20}c} {{\varvec{\upvarepsilon }}_{1} } \\ {{\varvec{\upvarepsilon }}_{2} } \\ \end{array} } \right]_{t + 1}$$

where now the state vector is partitioned into two parts with y₁ containing the n₁ observed variables and y₂ the n₂ hidden state variables, with n₁ + n₂ = n, and u_t and ε_t+1 are as in the previous section.²⁴ The decision maker ranks sequences of states and controls according to

$$- E_{0} \left[ {\sum\limits_{t = 0}^{\infty } {\beta^{t} U\left( {{\mathbf{y}}_{1,t} ,{\mathbf{y}}_{2,t} ,{\mathbf{u}}_{t} } \right)} } \right]$$

(12)

with the one-period utility function U defined as

$$\begin{aligned} U\left( {{\mathbf{y}}_{1,t} ,{\mathbf{y}}_{2,t} ,{\mathbf{u}}_{t} } \right) & = \left[ {\begin{array}{*{20}c} {{\mathbf{y^{\prime}}}_{1,t} } & {{\mathbf{y^{\prime}}}_{2,t} } & {{\mathbf{u^{\prime}}}_{t} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {{\mathbf{Q}}_{11} } & {{\mathbf{Q}}_{12} } & {{\mathbf{W}}_{1} } \\ {{\mathbf{Q^{\prime}}}_{12} } & {{\mathbf{Q}}_{22} } & {{\mathbf{W}}_{2} } \\ {{\mathbf{W^{\prime}}}_{1} } & {{\mathbf{W^{\prime}}}_{2} } & {\mathbf{R}} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {{\mathbf{y}}_{1,t} } \\ {{\mathbf{y}}_{2,t} } \\ {{\mathbf{u}}_{t} } \\ \end{array} } \right] \\ & \quad \equiv \left[ {\begin{array}{*{20}c} {{\mathbf{y^{\prime}}}_{t} } & {{\mathbf{u^{\prime}}}_{t} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\mathbf{Q}} & {\mathbf{W}} \\ {{\mathbf{W^{\prime}}}} & {\mathbf{R}} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {{\mathbf{y}}_{t} } \\ {{\mathbf{u}}_{t} } \\ \end{array} } \right] \\ \end{aligned}$$

(13)

with the matrices Q, R and W as in the previous section and y_1,0, the observed portion of the state vector at time 0, given.

Assuming that the decision maker believes that the distribution of the initial value of the unobserved part of the state is ${\mathbf{y}}_{2,0} \sim \varvec{N}({\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,0} ,{\varvec{\Delta }}_{0} )$ and taking into account that y₁ is observed, the ordinary Kalman filter gives the projected value of y_1,t+1 conditional on all the available information at time t, i.e. $E\left( {{\mathbf{y}}_{1,t + 1} |I_{t} } \right)$, and the updated value of y_2,t+1 conditional on all the available information at time t + 1, namely ${\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t + 1} \equiv E\left( {{\mathbf{y}}_{2,t + 1} |I_{t + 1} } \right)$.²⁵ Under the approximating model, y_2,t is distributed as $\varvec{N}({\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t} ,{\varvec{\Delta }}_{t} )$, with ${\varvec{\Delta }}_{t} = E[({\mathbf{y}}_{2,t} - {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t} )({\mathbf{y}}_{2,t} - {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t} )^{\prime}]$, and the mean and variance of the state represent sufficient statistics for the distribution of the unobserved part of the state at time t.²⁶ Equation (11) is then rewritten with the system equations for y₂ replaced by the associated ordinary Kalman filter updating equation and the law of motion for the observed subvector expressed in terms of the updated estimate of the hidden state and the discrepancy between this value and the true one, i.e.²⁷

$${\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t + 1} = {\mathbf{A\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} + {\mathbf{Bu}}_{t} + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{1} ({\varvec{\Delta }}_{t} ){\varvec{\upvarepsilon }}_{t + 1} + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{2} ({\varvec{\Delta }}_{t} )\left( {{\mathbf{y}}_{2,t} - {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t} } \right)$$

(14)

with²⁸

$${\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t + 1} = \left[ {\begin{array}{*{20}c} {{\mathbf{y}}_{1} } \\ {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2} } \\ \end{array} } \right]_{t + 1} ,\quad {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{1} ({\varvec{\Delta }}_{t} ) = \left[ {\begin{array}{*{20}c} {{\mathbf{C}}_{1} } \\ {{\varvec{\Xi }}({\varvec{\Delta }}_{t} ){\mathbf{C}}_{1} } \\ \end{array} } \right],\quad {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{2} ({\varvec{\Delta }}_{t} ) = \left[ {\begin{array}{*{20}c} {{\mathbf{A}}_{12} } \\ {{\varvec{\Xi }}({\varvec{\Delta }}_{t} ){\mathbf{A}}_{12} } \\ \end{array} } \right],$$

where ${\varvec{\Xi }} ({\varvec{\Delta }}_{t} )= ({\mathbf{A}}_{22} {\varvec{\Delta }}_{t} {\mathbf{A^{\prime}}}_{12} + {\mathbf{C}}_{2} {\mathbf{C^{\prime}}}_{1} )({\mathbf{A}}_{12} {\varvec{\Delta }}_{t} {\mathbf{A^{\prime}}}_{12} + {\mathbf{C}}_{1} {\mathbf{C^{\prime}}}_{1} )^{ - 1} ,{\varvec{\upvarepsilon }}_{t + 1} \sim {\varvec{N}}({\mathbf{0}},{\mathbf{I}}_{l} )$ and ${\mathbf{y}}_{2,t} \sim \varvec{N}({\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t} ,{\varvec{\Delta }}_{t} )$.

In this approximating model appear two random vectors: ε_t+1 and ${\mathbf{y}}_{2,t} - {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t}$. Let ω_1,t and ω_2,t represent the perturbation to the distribution of ε_t+1 and of the hidden state conditional on (${\mathbf{y}}_{1,t} ,{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t}$), respectively.²⁹ Then the misspecified model is written as³⁰

$${\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t + 1} = {\mathbf{A\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} + {\mathbf{Bu}}_{t} + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{1} ({\varvec{\Delta }}_{t} )\left( {{\varvec{\upvarepsilon }}_{t + 1} + {\varvec{\upomega }}_{1,t} } \right) + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{2} ({\varvec{\Delta }}_{t} )\left( {{\varvec{\upomega }}_{2,t} + {\mathbf{y}}_{2,t} - {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t} } \right)$$

(15)

and the associated return function is

$$U\left( {{\mathbf{y}}_{1,t} ,{\mathbf{y}}_{2,t} ,{\mathbf{u}}_{t} } \right) - \theta_{1} \left| {{\varvec{\upomega }}_{1,t} } \right|^{2} - \theta_{2} {\varvec{\upomega}}^{\prime}_{2,t} \Delta_{t}^{ - 1} {\varvec{\upomega }}_{2,t}$$

(16)

where θ₁ and θ₂ penalize distortions ω_1,t and ω_2,t, respectively.³¹

When ${\mathbf{y}}_{t}^{d}$ and ${\mathbf{u}}_{t}^{d}$ denote the vectors of desired values of the states and controls, respectively, for period t, Eq. (16) can be rewritten as³²

$$\begin{aligned} r\left( {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} ,{\tilde{\mathbf{u}}}_{t} } \right) & = \left( {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} - {\mathbf{y}}_{t}^{d} } \right)^{\prime } {\mathbf{Q}}\left( {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} - {\mathbf{y}}_{t}^{d} } \right) + \left( {{\tilde{\mathbf{u}}}_{t} - {\tilde{\mathbf{u}}}_{t}^{d} } \right)^{\prime } {\tilde{\mathbf{R}}}({\varvec{\Delta }}_{t} )\left( {{\tilde{\mathbf{u}}}_{t} - {\tilde{\mathbf{u}}}_{t}^{d} } \right) \\ & \quad + 2\left( {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} - {\mathbf{y}}_{t}^{d} } \right)^{\prime } {\tilde{\mathbf{W}}}\left( {{\tilde{\mathbf{u}}}_{t} - {\tilde{\mathbf{u}}}_{t}^{d} } \right) \\ \end{aligned}$$

(17)

where³³

$${\tilde{\mathbf{R}}}({\varvec{\Delta }}_{t} ) = \left[ {\begin{array}{*{20}c} {\mathbf{R}} & {{\mathbf{E}}^{*} } \\ {{\mathbf{E}}^{{*{\prime }}} } & {{\varvec{\Delta }}_{{\varvec{\upomega }}} ({\varvec{\Delta }}_{t} )} \\ \end{array} } \right]\quad {\text{with}}\quad \mathop {{\mathbf{E}}^{*} }\limits_{{m \times (l + n_{2} )}} = \left[ {\begin{array}{*{20}c} {\mathop {{\mathbf{O}}_{{}} }\limits_{m \times l} } & {\mathop {{\mathbf{W^{\prime}}}_{2} }\limits_{{m \times n_{2} }} } \\ \end{array} } \right],\quad {\varvec{\Delta }}_{{\varvec{\upomega }}} ({\varvec{\Delta }}_{t} ) = \left[ {\begin{array}{*{20}c} { - \theta_{1} {\mathbf{I}}_{l} } & {\mathbf{O}} \\ {\mathbf{O}} & {{\mathbf{R}}_{22} - \theta_{2} {\varvec{\Delta }}_{t}^{ - 1} } \\ \end{array} } \right],$$

${\tilde{\mathbf{W}}} = \left( {{\mathbf{W}}\quad {\mathbf{M}}^{*} } \right)$ with ${\mathbf{M}}^{*} = [{\mathbf{O}}\quad {\mathbf{Q}}_{2} ]$, O being a null matrix of dimension n × l and Q₂ the matrix of dimension n × n₂ obtained deleting the first n₁ columns of matrix Q in (13), ${\tilde{\mathbf{u}}}_{t}^{d} = \left( {{\mathbf{u}}_{t}^{d\prime } \, {\mathbf{0}}^{\prime } } \right)^{{\prime }}$ and 0 a null (l + n₂)-dimensional vector. As stressed in Hansen and Sargent (2007a, p. 389) “assigning different values to θ… lets the decision maker to focus more or less on misspecifications of one or the other of the two distributions being perturbed.”

For the linear quadratic problem at hand, $({\mathbf{y}}_{1} ,{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2} )$-contingent distortions ω_1,t and ω_2,t and the associated robust rule for u can be computed by solving the deterministic, certainty equivalent, problem³⁴

$$\mathop {\hbox{max} }\limits_{{{\mathbf{u}}_{t} }} \, \mathop {\hbox{min} }\limits_{{{\varvec{\upomega }}_{1} ,{\varvec{\upomega }}_{2} }} \, \left[ { - \sum\limits_{t = 0}^{\infty } {\beta^{t} r\left( {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} ,{\tilde{\mathbf{u}}}_{t} } \right)} } \right]$$

(18)

subject to

$${\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t + 1} = {\mathbf{A\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} + {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ){\tilde{\mathbf{u}}}_{t}$$

(19)

where

$$\begin{aligned} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t + 1} & = \left[ {\begin{array}{*{20}c} {{\mathbf{y}}_{1} } \\ {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2} } \\ \end{array} } \right]_{t + 1} ,\quad {\tilde{\mathbf{u}}}_{t} = \left[ {\begin{array}{*{20}c} {\mathbf{u}} \\ {\varvec{\upomega }} \\ \end{array} } \right]_{t} ,\quad {\varvec{\upomega }}_{t} =\left[ {\begin{array}{*{20}c} {{\varvec{\upomega }}_{1} } \\ {{\varvec{\upomega }}_{2} } \\ \end{array} } \right]_{t} , \\ {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ) & = \left[ {\begin{array}{*{20}c} {\mathbf{B}} & {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{1} ({\varvec{\Delta }}_{t} )} & {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}_{2} ({\varvec{\Delta }}_{t} )} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {\mathbf{B}} & {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \\ \end{array} } \right] \\ \end{aligned}$$

and the Gaussian random vectors with mean zero have been dropped as in the previous section.³⁵

This result is stated as:

Proposition 2

Extremizing the objective function (18) subject to (19), with definitions as in (17), yields the (θ₁, θ₂)-constrained worst-case control for the decision maker³⁶

$$\begin{aligned} {\mathbf{u}}_{t} & = - \left\{ {{\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{B}} + \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right)^{\prime } } \right\}^{ - 1} \\ & \quad \times \left\{ {\left[ {{\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} + \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{M}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{A}}} \right)} \right]{\mathbf{y}}_{t} } \right. \\ & \quad\left. { + \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\Theta }}_{t}^{ - 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{p}}_{t + 1} + {\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1} + {\mathbf{r}}_{t} } \right\}. \\ \end{aligned}$$

In this general case, where the arrays ${\mathbf{E}}_{t}^{*}$ and ${\mathbf{M}}_{t}^{*}$ are not necessarily null matrices, Proposition 2 cannot be written as Proposition 1 with

$${\mathbf{P}}_{t + 1}^{*} = {\mathbf{P}}_{t + 1} + {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ){\varvec{\Theta }}_{t}^{ - 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} .$$

In any case the following relations

$${\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{B}} + \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right)^{\prime } \ge {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{B}},$$

(20a)

$${\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{A}} + \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{M}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{A}}} \right) \ge {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{A}}$$

(20b)

$$\left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\Theta }}_{t}^{ - 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{p}}_{t + 1} + {\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1} \ge {\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1}$$

(20c)

always hold because it is assumed that θ₁ and θ₂ are large enough to make Θ_t positive definite.³⁷ The equality signs prevail when θ₁ = θ₂ = ∞.

4 Optimal Control of a Linear System with Time-Varying Parameters

Tucci (2006) argues that the model used by a “probabilistically sophisticated’ decision maker to represent dynamic misspecification, i.e. Equation (2), is observationally equivalent to a model with a time-varying intercept. Namely, unless some prior information is available, it is impossible to distinguish between the two models by simply observing the output. When this intercept is restricted to follow a mean reverting, or ‘Return to Normality’,³⁸ model and the symbols are as in Sect. 2, Eq. (2) can be rewritten as

$${\mathbf{y}}_{t + 1} = {\mathbf{A}}_{1} {\mathbf{y}}_{t} + {\mathbf{Bu}}_{t} + {\mathbf{C}}\left( {{\varvec{\upalpha }}_{t + 1} + {\varvec{\upvarepsilon }}_{t + 1} } \right)\quad {\text{for}}\quad t = 0, \ldots ,\infty ,$$

(21)

with

$${\varvec{\upalpha }}_{t + 1} = {\mathbf{a}} + {\varvec{\upnu }}_{t + 1} \quad {\text{for}}\quad t = 0, \ldots ,\infty ,$$

(22a)

$${\varvec{\upnu }}_{t + 1} = {\varvec{\Phi}} {\varvec{\nu}}_{t} + {\varvec{\upzeta }}_{t + 1} \quad {\text{for}}\quad t = 0, \ldots ,\infty ,$$

(22b)

where a is the unconditional mean vector of ${\varvec{\upalpha }}_{t + 1} ,\, {\varvec{\Phi }}$ the l × l transition matrix with eigenvalues strictly less than one in absolute value to guarantee stationarity and ${\varvec{\upzeta }}_{t + 1}$ is a Gaussian iid vector process with mean zero and an identity covariance matrix. The matrix A₁ is such that ${\mathbf{A}}_{1} {\mathbf{y}}_{t} + {\mathbf{Ca}}$ in (21) is equal to Ay_t in (2).³⁹ Obviously, the robust control formulation is more general than model (21)–(22) because in (2) the vector ω_t+1 can represent a very general, and possibly complicated, process.

Then the approach discussed in Kendrick (1981) and Tucci (2004) can be used to find the set of controls u_t which maximizes⁴⁰

$$J = E_{0} \left[ { - \sum\limits_{t = 0}^{\infty } {L_{t} \left( {{\mathbf{y}}_{t} , \, {\mathbf{u}}_{t} } \right)} } \right],$$

(23)

where

$$L_{t} \left( {{\mathbf{y}}_{t} ,{\mathbf{u}}_{t} } \right) = \beta^{t} \left[ {\left( {{\mathbf{y}}_{t} - {\mathbf{y}}_{t}^{d} } \right)^{\prime } {\mathbf{Q}}\left( {{\mathbf{y}}_{t} - {\mathbf{y}}_{t}^{d} } \right) + 2\left( {{\mathbf{y}}_{t} - {\mathbf{y}}_{t}^{d} } \right)^{\prime } {\mathbf{W}}\left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right) + \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right)^{\prime } {\mathbf{R}}\left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right)} \right],$$

(24)

subject to (21)–(22). This control problem can be solved treating the stochastic parameters as additional state variables. If the same objective functional used in the robust control problem is optimized, the expression in square bracket is identical to the one-period loss function defined in (6).

When the hyper-structural parameters a and Φ are known, the original problem is restated in terms of an augmented state vector z_t as: find the controls u_t maximizing⁴¹

$$J = E_{0} \left[ { - \sum\limits_{t = 0}^{\infty } {L_{t} \left( {{\mathbf{z}}_{t} , \, {\mathbf{u}}_{t} } \right)} } \right]$$

(25)

subject to⁴²

$${\mathbf{z}}_{t + 1} = {\mathbf{f}}({\mathbf{z}}_{t} ,{\mathbf{u}}_{t} ) + {\varvec{\upvarepsilon }}_{t + 1}^{*} \quad {\text{for}}\quad t = 0, \ldots ,\infty ,$$

(26)

with⁴³

$${\mathbf{z}}_{t}=\left[\begin{array}{*{20}c}{{\mathbf{y}}_{t} } \\ {{\varvec{\upalpha }}_{t + 1} } \\ \end{array} \right] , { }{\mathbf{f}}\left( {{\mathbf{z}}_{t} , { }{\mathbf{u}}_{t} } \right) = \left[ \begin{array}{*{20}c} {{\mathbf{A}}_{1} {\mathbf{y}}_{t} + {\mathbf{Bu}}_{t} + {\mathbf{C}}{\varvec{\upalpha }}_{t + 1} } \\ {\varvec{\Phi}} {\varvec{\upalpha}}_{t + 1} + \left( {{\mathbf{I}}_{l} - {\varvec{\Phi }}} \right){\mathbf{a}} \\ \end{array} \right]{\text{ and }}{\varvec{\upvarepsilon }}_{t}^{*} = \left[\begin{array}{*{20}c} {{\varvec{\upvarepsilon }}_{t} } \\ {{\varvec{\upzeta }}_{t + 1} } \\ \end{array} \right].$$

(27)

and the arrays z_t and ${{{\mathbf{f}}({\mathbf{z}}_{t} ,{\mathbf{u}}_{t} )}}$ having dimension n + l, i.e. the number of original states plus the number of stochastic parameters. For this ‘augmented’ control problem the L’s in Eq. (25) are defined as

$$L_{t} \left( {{\mathbf{z}}_{t} ,{\mathbf{u}}_{t} } \right) = \left( {{\mathbf{z}}_{t} - {\mathbf{z}}_{t}^{d} } \right)^{\prime } {\mathbf{Q}}_{t}^{*} \left( {{\mathbf{z}}_{t} - {\mathbf{z}}_{t}^{d} } \right) + 2\left( {{\mathbf{z}}_{t} - {\mathbf{z}}_{t}^{d} } \right)^{\prime } {\mathbf{W}}_{t}^{*} \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right) + \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right)^{\prime } {\mathbf{R}}_{t} \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{d} } \right)$$

(28)

with ${\mathbf{Q}}_{t}^{*} = \beta^{t} {\mathbf{Q}}^{*} ,{\mathbf{Q}}^{*} = diag\left( {{\mathbf{Q}}, \, - \beta \theta {\mathbf{I}}_{l} } \right),{\mathbf{W}}_{t}^{*} = \beta^{t} [\begin{array}{*{20}c} {{\mathbf{W^{\prime}}}} & {{\mathbf{O^{\prime}}}} \\ \end{array} ]^{\prime}$ and ${\mathbf{R}}_{t} = \beta^{t} {\mathbf{R}}$ and this proposition can be stated:

Proposition 3

Maximizing the objective function (25) subject to (26), with definitions as in (27) and (28), yields a TVP-control equal to

$${\mathbf{u}}_{t} = - ({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1}^{ + } {\mathbf{B}})^{ - 1} [({\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1}^{ + } {\mathbf{A}} + {\mathbf{W}}_{t} \prime ){\mathbf{y}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1}^{ + } {\mathbf{K}}_{{ 1 1 , {\text{t + }}1}}^{ - 1} {\mathbf{k}}_{1,t + 1} + {\mathbf{r}}_{t} ]$$

with

$${\mathbf{K}}_{11,t + 1}^{ + } = [{\mathbf{I}}_{n} + ({\mathbf{K}}_{11,t + 1} {\mathbf{C}} + {\mathbf{K}}_{12,t + 1} {\varvec{\Phi }})(\beta \theta {\mathbf{I}}_{l} - {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{C}})^{ - 1} {\mathbf{C}}^{\prime} ]{\mathbf{K}}_{11,t + 1} .$$

when ${\varvec{\upnu }}_{t + 1} \equiv {\varvec{\upomega }}_{t + 1}$. Namely, the same shock is used to determine both robust control and TVP-control.

The quantity ${\mathbf{K}}_{11,t + 1}^{ + }$ in Proposition 3 collapses to the ‘robust’ Riccati matrix ${\mathbf{P}}_{t + 1}^{*}$ in Proposition 1 when ${\mathbf{P}}_{t + 1} = {\mathbf{K}}_{11,t + 1}$ and ${\mathbf{K}}_{12,t + 1}$ is a null matrix. This means that robust is control is insensitive to the true value of Φ appearing in the law of motion for the stochastic parameters. This is due to the fact that when the same objective functional is optimized both in the robust and TVP-control problems, the only difference between the associated Bellman Eqs. is that the former, implicitly, sets ${\mathbf{P}}_{t} = {\mathbf{K}}_{11,t} ,{\mathbf{p}}_{t} = {\mathbf{k}}_{1,t}$ and K_12,t, K_21,t, K_22,t and k_2,t equal to null arrays. Therefore, by construction, the control applied by the decision maker who wants to be “robust to misspecifications of the approximating model” implicitly assumes that the ω’s in (2) are serially uncorrelated.

The framework laid out in this section can be used also to study the case of robust control without commitment discussed in Sect. 3. Then, the following result holds:

Proposition 4

Maximizing the objective function (25) subject to (26), with definitions as in (27) when

$${\varvec{\upalpha }}_{t + 1} = \left[ {\begin{array}{*{20}c} {{\varvec{\upalpha }}_{1,t + 1} } \\ {{\varvec{\upalpha }}_{2,t + 1} } \\ \end{array} } \right],\quad {\varvec{\Phi }} = \left[ {\begin{array}{*{20}c} {\mathop {{\varvec{\Phi }}_{11} }\limits_{l \times l} } & {\mathbf{O}} \\ {\mathbf{O}} & {\mathop {{\varvec{\Phi }}_{22} }\limits_{{n_{2} \times n_{2} }} } \\ \end{array} } \right],{\mathbf{C}}\,{\text{isreplaced}}\,{\text{by}}\quad {\mathbf{C}}^{ * } = \left[ {\begin{array}{*{20}c} {\mathop {\mathbf{C}}\limits_{n \times l} } & {\mathop {\mathbf{O}}\limits_{{n \times n_{2} }} } \\ \end{array} } \right]$$

and the arrays z_t and ${{{\mathbf{f}}({\mathbf{z}}_{t} ,{\mathbf{u}}_{t} )}}$ have dimension $n + (l + n_{2} )$ and as in Eq. (28) with ${\mathbf{Q}}_{t}^{*} = \beta^{t} {\mathbf{Q}}^{*} ,{\mathbf{R}}_{t} = \beta^{t} {\mathbf{R}}$ and ${\mathbf{W}}_{t}^{*} = \beta^{t} {\mathbf{W}}_{{}}^{*}$ where

$${\mathbf{Q}}^{*} = \left[ {\begin{array}{*{20}c} {\mathop {\mathbf{Q}}\limits_{n \times n} } & {\mathop {{\mathbf{M}}^{*} }\limits_{{n \times (l + n_{2} )}} } \\ {{\mathbf{M}}^{{*^{\prime}}} } & {\mathop {{\varvec{\Delta }}_{{\varvec{\upomega }}} ({\varvec{\Delta }}_{t} )}\limits_{{(l + n_{2} ) \times (l + n_{2} )}} } \\ \end{array} } \right],\quad {\mathbf{W}}^{*} = \left[ {\begin{array}{*{20}c} {\mathop {\mathbf{W}}\limits_{n \times m} } \\ {\mathop {{\mathbf{E}}^{*\prime } }\limits_{{(l + n_{2} ) \times m}} } \\ \end{array} } \right]\quad {\text{and}}\quad \mathop {{\mathbf{E}}^{*} }\limits_{{m \times (l + n_{2} )}} = \left[ {\begin{array}{*{20}c} {\mathop {\mathbf{O}}\limits_{m \times l} } & {\mathop {\mathbf{E}}\limits_{{m \times n_{2} }} } \\ \end{array} } \right]$$

yields a TVP-control equal to

$$\begin{aligned} {\mathbf{u}}_{t} & = - \left\{ {{\mathbf{R}}_{t} + {\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{B}} + \left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ) + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{E}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{B}}} \right)} \right\}^{ - 1} \\ & \quad \times \left\{ {\left[ {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} + \left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ) + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{M}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{A}}} \right)} \right]} \right.{\mathbf{y}}_{t} \\ & \quad \left. { + \left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ) + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\Theta }}_{t}^{ - 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{p}}_{t + 1} + {\mathbf{B^{\prime}k}}_{1,t + 1} + {\mathbf{r}}_{t} } \right\}. \\ \end{aligned}$$

when the worst-case adverse shock determined in Sect. 3 is used.

The quantities

$${\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{B}} + \left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ) + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{E}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{B}}} \right)$$

(29a)

$${\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{A}} + \left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ) + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\Theta }}_{t}^{ - 1} \left( {{\mathbf{M}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{A}}} \right)$$

(29b)

$$\left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} ) + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\Theta }}_{t}^{ - 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{p}}_{t + 1} + {\mathbf{B^{\prime}k}}_{1,t + 1}$$

(29c)

in Proposition 4 are identical to the corresponding quantities in Proposition 2 when ${\mathbf{K}}_{11,t + 1} \equiv {\mathbf{P}}_{t + 1}$ and ${\mathbf{k}}_{11,t + 1} \equiv {\mathbf{p}}_{t + 1}$ and ${\mathbf{K}}_{12,t + 1}$ is a null matrix.⁴⁴ Therefore even this non “probabilistically sophisticated” decision maker implicitly assumes that ω_1,t and ω_2,t, i.e. the perturbation to the distribution of ε_t+1 and of the hidden state conditional on (${\mathbf{y}}_{1,t} ,{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t}$), respectively, in (15) are both independent and serially uncorrelated.

Before leaving this section, it is worth it to emphasize two things. First of all, the results in Propositions 3 and 4 do not imply that robust control is implicitly based on a very specialized type of time-varying parameter model or that one of the two approaches is better than the other. Robust control and TVP-control represent two alternative ways of dealing with the problem of not knowing the true model ‘we’ want to control and are generally characterized by different solutions. In general, when the same objective functional and terminal conditions are used, the main difference is due the fact that the former is determined assuming for ω_t+1 the worst-case value, whereas the latter is computed using the expected conditional mean of ${\varvec{\upnu }}_{t + 1}$ and taking into account its relationship with next period conditional mean. As a side effect even the Riccati matrices common to the two procedures, named P and p in the robust control case and K₁₁ and k₁₁ in the TVP-case, are different. The use of identical Riccati matrices and of an identical shock in the two alternative approaches, i.e. setting ${\mathbf{K}}_{11,t + 1} \equiv {\mathbf{P}}_{t + 1} ,{\mathbf{k}}_{11,t + 1} \equiv {\mathbf{p}}_{t + 1}$ and ${\varvec{\upnu }}_{t + 1} \equiv {\varvec{\upomega }}_{t + 1}$ or ${\varvec{\upnu }}_{t + 1} \equiv [\begin{array}{*{20}c} {\varvec{\upomega}}^{\prime}_{1,t} & {\varvec{\upomega}}^{\prime}_{2,t} \\ \end{array} ]^{\prime}$, has the sole purpose of investigating some of the implicit assumptions of these procedures.

Secondly the results of this section do not claim that the worst-case adverse shocks are serially uncorrelated or that the perturbation to the distribution of ε_t+1 and of the hidden state conditional on (${\mathbf{y}}_{1,t} ,{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{2,t}$) in (15) are both independent and serially uncorrelated. It simply shows that in all models where the agent is assumed to behave both in a “probabilistically sophisticated” and in a probabilistically ‘unsophisticated’ manner robust control implicitly assumes that these shocks are serially uncorrelated. This follows from the Bellman Equation associated with this type of problem.

5 Robust Control in the Presence of Uncertain Parameters in the Structural Model

The robust control problems discussed in the previous sections deal with unstructured uncertainty à la Hansen and Sargent. However, sometimes robust control is applied to situations where uncertainty is related to unknown structural parameters. Giannoni (2002, 2007) considers an optimizing model for monetary policy. This is a structural forward-looking model where the constant structural parameters are unknown to the policymaker but are known to agents in the private sector. It “is composed of a monetary policy rules and two structural equations—an intertemporal IS equation and an aggregate supply equation—that are based on explicit microeconomic foundations… (namely, they) can be derived as log-linear approximations to equilibrium conditions of an underlying general equilibrium model with sticky prices” (Giannoni 2002, pp. 112–114).⁴⁵

In Giannoni (2007), the demand side of the economy is written as⁴⁶

$$x_{t} = E_{t} x_{t + 1} + \sigma^{ - 1} E_{t} \pi_{t + 1} - \sigma^{ - 1} \hat{\iota }_{t} + \frac{\varpi }{{\left( {\varpi + \sigma } \right)\sigma }}\delta_{t} + \frac{1}{{\left( {\varpi + \sigma } \right)}}\eta_{t}$$

(30)

where E_t denotes the expectation formed at time t, x_t the output gap, π_t the rate of inflation, $\hat{\iota }_{t}$ the percentage deviation of the nominal interest rate from its constant steady state value, δ_t a demand shock and η_t an “adverse efficient supply shock”. By output gap is meant the percentage deviation of actual output from its constant steady state value minus the percentage deviation of the efficient rate of output.⁴⁷ The aggregate supply curve takes the form⁴⁸

$$\pi_{t} = \beta E_{t} \pi_{t + 1} + \kappa x_{t} + \frac{\kappa }{\varpi + \sigma }\mu_{t}$$

(31)

with μ_t the percent deviation of the desired markup from steady state,⁴⁹κ a parameter greater than zero and β the discount factor.⁵⁰ As pointed out in Giannoni (2007, p. 187), the parameters σ and ϖ represent “the inverse of the intertemporal elasticity of substitution in private expenditure (and) … the elasticity of each firm’s real marginal cost with respect to its own supply”, respectively.⁵¹ Finally, it is assumed that the exogenous shocks δ_t, η_t and μ_t have zero (unconditional) mean, are independent of the parameters σ, κ and ϖ and follow an AR(1) process,⁵² i.e.

$$\left[ {\begin{array}{*{20}c} \delta \\ \eta \\ \mu \\ \end{array} } \right]_{t + 1} = \left[ {\begin{array}{*{20}c} {\rho_{\delta } } & 0 & 0 \\ 0 & {\rho_{\eta } } & 0 \\ 0 & 0 & {\rho_{\mu } } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} \delta \\ \eta \\ \mu \\ \end{array} } \right]_{t} + \left[ {\begin{array}{*{20}c} {\xi_{\delta } } \\ {\xi_{\eta } } \\ {\xi_{\mu } } \\ \end{array} } \right].$$

(32)

In this model “monetary policy has real effects … because prices do not respond immediately to perturbations … only a fraction … of suppliers may change their prices at the end of each period” (Giannoni 2007, p. 187). The controller determines the optimal monetary policy optimizing the following penalty function⁵³

$$L_{0} = E_{0} \left\{ {\left( {1 - \beta } \right)\sum\limits_{t = 0}^{\infty } {\beta^{t} \left[ {\pi_{t}^{2} + \lambda_{x} \left( {x_{t} - x^{*} } \right)^{2} + \lambda_{i} \hat{\iota }_{t}^{2} } \right]} } \right\}$$

(33)

where λ_x and λ_y, both positive, “are weights placed on the stabilization of the output gap and the nominal interest rate and where x^* ≥ 0 represents some optimal level of output gap” (Giannoni 2007, p. 189).

Assuming rational expectations, the system (30)–(32) can be rewritten as (1) when ${\mathbf{y}}_{t} = \left( {x_{t} \, \pi_{t} \, \delta_{t} \, \eta_{t} \, \mu_{t} } \right)^{{\prime }}$ and ${\mathbf{u}}_{t} = \hat{\iota }_{t}$. Then matrix A looks like

$${\mathbf{A}} = \left[ {\begin{array}{*{20}c} {{\tilde{\mathbf{A}}}} & {\mathbf{D}} \\ {\mathbf{O}} & {\mathbf{T}} \\ \end{array} } \right],$$

(34)

where

$${\tilde{\mathbf{A}}} = \left[ {\begin{array}{ccc} {1 + \left( {{\kappa \mathord{\left/ {\vphantom {\kappa {\beta \sigma }}} \right. \kern-0pt} {\beta \sigma }}} \right)} & { - {1 \mathord{\left/ {\vphantom {1 {\beta \sigma }}} \right. \kern-0pt} {\beta \sigma }}} \\ { - {\kappa \mathord{\left/ {\vphantom {\kappa \beta }} \right. \kern-0pt} \beta }} & {{1 \mathord{\left/ {\vphantom {1 \beta }} \right. \kern-0pt} \beta }} \\ \end{array} } \right] , { }{\mathbf{D}} = \left[ {\begin{array}{cccc} {\begin{array}{*{20}c} { - {\varpi \mathord{\left/ {\vphantom {\varpi {\left( {\varpi + \sigma } \right)\sigma }}} \right. \kern-0pt} {\left( {\varpi + \sigma } \right)\sigma }}} & { - {1 \mathord{\left/ {\vphantom {1 {\left( {\varpi + \sigma } \right)}}} \right. \kern-0pt} {\left( {\varpi + \sigma } \right)}}} & {{\kappa \mathord{\left/ {\vphantom {\kappa {\left( {\varpi + \sigma } \right)\beta \sigma }}} \right. \kern-0pt} {\left( {\varpi + \sigma } \right)\beta \sigma }}} \\ \end{array} } \\ {\begin{array}{*{20}c} & & & { \, 0 \, } & & & & & { \, 0 \, } & &{ \, - {\kappa \mathord{\left/ {\vphantom {\kappa {\left( {\varpi + \sigma } \right)\beta }}} \right. \kern-0pt} {\left( {\varpi + \sigma } \right)\beta }}} \\ \end{array} } \\ \end{array} } \right] ,$$

(35)

T is the 3 × 3 diagonal matrix on the right hand side of Eq. (32) and O is a null 3 × 2 array, and B is defined as ${\mathbf{B}} = (\begin{array}{*{20}c} {\sigma^{ - 1} } & 0 & 0 & 0 & 0 \\ \end{array} )$. The vector of disturbances ${\varvec{\upvarepsilon }}_{t} = \left( {\varepsilon_{x} \, \varepsilon_{\pi } \, \varepsilon_{\delta } \, \varepsilon_{\eta } \, \varepsilon_{\mu } } \right)^{{\prime }}$ has mean zero and identity covariance matrix and C is appropriately defined. Namely, it is such that ${\mathbf{CC}}^{\prime} = E({\varvec{\upxi }}_{t} {\varvec{\upxi}}^{\prime}_{t} )$ where ${\varvec{\upxi}}^{\prime}_{t} = \left( {\xi_{x} \, \xi_{\pi } \, \xi_{\delta } \, \xi_{\eta } \, \xi_{\mu } } \right)$, with ξ_x and ξ_π the errors associated with the output gap and inflation, respectively, and $E({\varvec{\upxi }}_{t} ) = {\mathbf{0}},E({\varvec{\upxi }}_{t} {\varvec{\upxi}}^{\prime}_{t} ) =\Sigma$. Similarly, the one-period loss function implicit in (33) can be put in the format (6) when W is a null matrix, ${\mathbf{Q}} = (1 - \beta )diag(\begin{array}{*{20}c} {\lambda_{x} } & 1 & 0 & 0 & 0 \\ \end{array} )$ and ${\mathbf{R}} = (1 - \beta )\lambda_{\iota }$.

In the presence of uncertain parameters, the worst-case parameter vector results in worst-case matrices which can be viewed as the algebraic sum of the ‘baseline case matrices’, A and B, and the ‘worst-case discrepancies’, A_ω and B_ω.⁵⁴ It follows that the model in the worst case scenario can be written as⁵⁵

$${\mathbf{y}}_{t + 1} = \left( {{\mathbf{A}} + {\mathbf{A}}_{{\varvec{\upomega }}} } \right){\mathbf{y}}_{t} + \left( {{\mathbf{B}} + {\mathbf{B}}_{{\varvec{\upomega }}} } \right){\mathbf{u}}_{t} + {\mathbf{C}}{\varvec{\upvarepsilon }}_{\tau + 1} = {\mathbf{Ay}}_{t} + {\mathbf{Bu}}_{t} + {\mathbf{C}}{\varvec{\upvarepsilon }}_{\tau + 1} + \left( {{\mathbf{A}}_{{\varvec{\upomega }}} {\mathbf{y}}_{t} + {\mathbf{B}}_{{\varvec{\upomega }}} {\mathbf{u}}_{t} } \right)$$

(36)

where the term in parenthesis on the right-hand side of the second equality sign plays the role of Cω_t+1 in Eq. (2). More precisely, the quantity ${\mathbf{A}}_{{\varvec{\upomega }}} {\mathbf{y}}_{t} + {\mathbf{B}}_{{\varvec{\upomega }}} {\mathbf{u}}_{t}$ replaces the worst-case adverse shock used to derive the result in Proposition 1, premultiplied by the volatility matrix C, in a robust control model where uncertainty is à la Hansen and Sargent. Then, robust control for Giannoni’s model case looks like

$${\mathbf{u}}_{t} = - ({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{B}}_{{\mathbf{W}}} )^{ - 1} [({\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{A}}_{{\mathbf{W}}} + {\mathbf{W}}_{t} \prime ){\mathbf{y}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1} + {\mathbf{r}}_{t} ]$$

(37)

where ${\mathbf{A}}_{{\mathbf{w}}} = {\mathbf{A}} + {\mathbf{A}}_{{\varvec{\upomega }}} {\text{ and }}{\mathbf{B}}_{{\mathbf{w}}} = {\mathbf{B}} + {\mathbf{B}}_{{\varvec{\upomega }}}$. The same worst-case adverse shock can be used to compute the associated TVP-control.

6 Some Numerical Results

The permanent income model is a popular model in the robust control literature (see, e.g., Hansen and Sargent 2001, 2003, 2007b; Hansen et al. 1999, 2002). It is a linear quadratic stochastic growth model with a habit where a “probabilistically sophisticated” planner values a scalar process s of consumption services according to⁵⁶

$$E_{0} \left[ { - \sum\limits_{t = 0}^{\infty } {\beta^{t} \left( {s_{t} - \mu_{b} } \right)^{2} } } \right]$$

(38)

with μ_b a preference parameter governing the curvature of the utility function.⁵⁷ The service s is produced by the scalar consumption process c_t via the household technology

$$s_{t} = (1 + \lambda )c_{t} - \lambda h_{t - 1}$$

(39a)

$$h_{t} = \delta_{h} h_{t - 1} + (1 - \delta_{h} )c_{t}$$

(39b)

where $\lambda \ge 0,0 < \delta_{h} < 1$ and h_t is a stock of households habits given by a geometric weighted average of present and past consumption. Then a linear technology converts an exogenous (scalar) stochastic endowment d_tinto consumption or capital, i.e.

$$k_{t} = \delta_{k} k_{t - 1} + i_{t}$$

(40a)

$$c_{t} + i_{t} = \gamma k_{t - 1} + d_{t}$$

(40b)

where k_t and i_t represent the capital stock and gross investment, respectively, at time t, γ the constant marginal product of capital and δ_k the depreciation factor for capital. The endowment is specified as the sum of two orthogonal AR(2) components, namely

$$d_{t + 1} = \mu_{d} + d_{1,t + 1} + d_{2,t + 1}$$

(41)

where d_1,t+1 and d_2,t+1 are the permanent and transitory component, respectively, and

$$\begin{aligned} d_{1,t + 1} & = g_{1} d_{1,t} + g_{2} d_{1,t - 1} + c_{1} \varepsilon_{1,t + 1} \\ d_{2,t + 1} & = a_{1} d_{2,t} + a_{2} d_{2,t - 1} + c_{2} \varepsilon_{2,t + 1} \\ \end{aligned}$$

with $\varepsilon_{1,t + 1}$ and $\varepsilon_{2,t + 1}$ as in Sect. 2.⁵⁸

Rewriting (40b) in terms of c_t and substituting it into (39a) yields

$$s_{t} = (1 + \lambda )[\gamma k_{t - 1} - i_{t} + d_{t} ] - \lambda h_{t - 1} .$$

(42)

Then the one-period loss function $(s_{t} - \mu_{b} )^{2}$ in (38) can be expressed as in (6) when ${\mathbf{u}}_{t} = i_{t} ,{\mathbf{u}}_{t}^{d} = 0, \, {\mathbf{y}}_{t} = (\begin{array}{*{20}c} {h_{t - 1} } & {k_{t - 1} } & {d_{t - 1} } & 1 & {d_{t} } & {d_{1,t} } & {d_{1,t - 1} } \\ \end{array} )^{\prime}, \, {\mathbf{y}}_{t}^{d} = { (}\begin{array}{*{20}c} { - {{\mu_{b} } \mathord{\left/ {\vphantom {{\mu_{b} } \lambda }} \right. \kern-0pt} \lambda }} & 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} )^{\prime}$⁵⁹, Q = diag(Q^†,O₂) where

$${\mathbf{Q}}^{\dag } = \left[ {\begin{array}{lllllll} {\lambda^{2} } & \bullet & \bullet & & \bullet & \bullet \\ { - \left( {1 + \lambda } \right)\lambda \gamma } & {\left( {1 + \lambda } \right)^{2} \gamma^{2} } & \bullet & & \bullet & \bullet \\ 0 & 0 & 0 & & \bullet & \bullet \\ 0 & 0 & 0 & & 0 & \bullet \\ { - \left( {1 + \lambda } \right)\lambda } & {\left( {1 + \lambda } \right)^{2} \gamma } & 0 & & 0 & {\left( {1 + \lambda } \right)^{2} } \\ \end{array} } \right]$$

(43)

and only the lower portion is reported because the matrix is symmetric, O₂is a square null matrix of dimension 2, ${\mathbf{R}} = \left( {1 + \lambda } \right)^{{\mathbf{2}}}$ and ${\mathbf{W}} = (\begin{array}{*{20}c} {w_{1} } & {w_{2} } & 0 & 0 & {w_{5} } & 0 & 0 \\ \end{array} )^{{\prime }}$ with $w_{1} = (1 + \lambda )\lambda ,w_{2} = - (1 + \lambda )^{2} \gamma$ and $w_{5} = - (1 + \lambda )^{2}$.

When model misspecification is not ruled out, the equations for the permanent and transitory components of the endowment process are rewritten adding the quantities $c_{1} \omega_{1,t + 1}$ and $c_{2} \omega_{2,t + 1}$, respectively. Then problem (5) is solved subject to Eq. (2) with the initial condition y₀ given and the matrices of coefficients defined as

$${\mathbf{A}} = \left[ {\begin{array}{*{20}c} {\delta_{h} } & {\left( {1 - \delta_{h} } \right)\gamma } & 0 & 0 & {\left( {1 - \delta_{h} } \right)} & 0 & 0 \\ 0 & {\delta_{k} } & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & {a_{2} } & {\mu_{d}^{ * } } & {a_{1} } & {g_{1} - a_{1} } & {g_{2} - a_{2} } \\ 0 & 0 & 0 & 0 & 0 & {g_{1} } & {g_{2} } \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 \\ \end{array} } \right],\quad {\mathbf{C}} = \left[ {\begin{array}{*{20}c} 0 & 0 \\ 0 & 0 \\ 0 & 0 \\ 0 & 0 \\ {c_{1} } & {c_{2} } \\ {c_{1} } & 0 \\ 0 & 0 \\ \end{array} } \right]$$

(44)

and ${\mathbf{B}} = \left( {b_{1} \quad 1\quad 0\quad 0\quad 0\quad 0\quad 0} \right)^{{\prime }}$, where $\mu_{d}^{*} = \mu_{d} \left( {1 - a_{1} - a_{2} } \right)$ and $b_{1} = - \left( {1 - \delta_{h} } \right)$.⁶⁰

Using the parameter estimates in Hansen et al. (2002), robust control for the permanent income model is computed for μ_b = 32 and different values of θ’s.⁶¹ The initial condition is set at y₀ = (100 100 13.7099 1.0 13.7099 0 0)′, and it is assumed a time horizon of 2 periods.⁶² As observed in Hansen and Sargent (2007a, p. 47), a preference for robustness “leads the consumer to engage in a form of precautionary savings that … tilts his consumption profile toward the future relative to what it would be without a concern about misspecification of (the endowment) process.” This is confirmed by the results reported in Table 1 where gross investment, the control variable, increases as θ gets smaller.⁶³

Table 1 Linear quadratic control (QLP) versus robust control at time 0

When the observationally equivalent model of Sect. 4 is used, the endowment process and its permanent component have time-varying intercepts following a mean reverting model, namely

$$d_{t + 1} = \mu_{d,t + 1}^{*} + a_{1} d_{t} + a_{2} d_{t - 1} + (g_{1} - a_{1} )d_{1,t} + (g_{2} - a_{2} )d_{1,t - 1}$$

(45a)

$$d_{1,t + 1} = \mu_{d1,t + 1}^{*} + g_{1} d_{1,t} + g_{2} d_{1,t - 1}$$

(45b)

with $\mu_{d,t + 1}^{*} = \mu_{d}^{*} + c_{1} \nu_{1,t + 1} + c_{2} \nu_{2,t + 1}$ and $\mu_{d1,t + 1}^{*} = c_{1} \nu_{1,t + 1}$ with $\nu_{i,t + 1} \equiv \varepsilon_{i,t + 1} + \omega_{i,t + 1}$ for i = 1, 2. Then the time-invariant portion of the intercepts can be interpreted as⁶⁴

$$\left[ {\begin{array}{*{20}c} {\mu_{d}^{*} } \\ 0 \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {c_{1} } & {c_{2} } \\ {c_{1} } & 0 \\ \end{array} } \right] \, \left[ {\begin{array}{*{20}c} {a_{1} } \\ {a_{2} } \\ \end{array} } \right]$$

(46a)

and the stochastic component takes the form

$$\left[ {\begin{array}{*{20}c} {\nu_{1} } \\ {\nu_{2} } \\ \end{array} } \right]_{t + 1} = \left[ {\begin{array}{*{20}c} {\phi_{11} } & {\phi_{12} } \\ {\phi_{21} } & {\phi_{22} } \\ \end{array} } \right] \, \left[ {\begin{array}{*{20}c} {\nu_{1} } \\ {\nu_{2} } \\ \end{array} } \right]_{t} + \left[ {\begin{array}{*{20}c} {\varepsilon_{1} } \\ {\varepsilon_{2} } \\ \end{array} } \right]_{t + 1} .$$

(46b)

In this case, setting $\varepsilon_{i,t + 1} = 0$ and $\nu_{i,t + 1} \equiv \omega_{i,t + 1}$ for i = 1, 2, the results reported in Sect. 4 hold.

The relationship between the value of Φ and the optimal control at various θ’s is shown in Fig. 1 where Φ = ϕI and several values of ϕ are used. As shown in Sect. 4, the TVP-control derived assuming that the intercept follows a ‘Return to Normality’ model and ϕ = 0 is identical to robust control when the same malevolent shocks are used. On the other hand, knowing that tomorrow’s shocks are negatively correlated with today’s ones would make the household, facing a negative ‘malevolent nature’ shock, to save less for a given β. For instance, when θ = 100, savings decrease from − 51.1894 at ϕ = 0 to − 57.7563 at ϕ = −.1. Then the controls at the various θ’s associated with negative ϕ’s are always below the corresponding robust controls and they go farther and farther from them as the absolute value of ϕ increases. The opposite occurs for positive values of ϕ. Again, the line farther from the ‘robust control line’ is that associated with a higher absolute value of ϕ.

Fig. 1

First period robust control and TVP-control, for various values of Φ, at different levels of θ for the permanent income model

A meaningful example of robust control applied to situations where uncertainty is related to unknown structural parameters of the model has been discussed in Sect. 5. When the parameter values are as in Giannoni (2007, pp. 189–191 and 200) both for the baseline case and for the worst case, the matrices in (36) look like⁶⁵

$${\mathbf{A}} = \left[ {\begin{array}{*{20}c} {1.1530} & { - 6.4297} & { - 4.7781} & { - 1.5873} & {.2429} \\ { - .0240} & {1.0101} & 0 & 0 & { - .0382} \\ 0 & 0 & {.35} & 0 & 0 \\ 0 & 0 & 0 & {.35} & 0 \\ 0 & 0 & 0 & 0 & {.35} \\ \end{array} } \right], \, {\mathbf{B}}=\left[ {\begin{array}{*{20}c} {6.3654} \\ 0 \\ 0 \\ 0 \\ 0 \\ \end{array} } \right]$$

and

$${\mathbf{A}}_{{\varvec{\upomega }}} = \left[ {\begin{array}{*{20}c} {.1870} & { - 4.6097} & { - 3.4856} & { - 1.0779} & {.6633} \\ { - .0071} & 0 & 0 & 0 & { - .0448} \\ 0 & 0 & {.45} & 0 & 0 \\ 0 & 0 & 0 & {.45} & 0 \\ 0 & 0 & 0 & 0 & {.45} \\ \end{array} } \right],\quad {\mathbf{B}}_{{\varvec{\upomega }}} =\left[ {\begin{array}{*{20}c} {4.5636} \\ 0 \\ 0 \\ 0 \\ 0 \\ \end{array} } \right].$$

with⁶⁶

$${\mathbf{C}}=\left[ {\begin{array}{*{20}c} { 2. 1 8 8 6} & 0& 0& 0& 0\\ { 0. 0 0 6 0} & { 1. 5 1} & 0& 0& 0\\ 0& 0& { 1. 7 3 6 4} & 0& 0\\ 0& 0& 0& 0& 0\\ 0& 0& { 1 6. 2 5 5 8} & 0& { 1 5. 0 7 9 3} \\ \end{array} } \right].$$

Setting the initial condition y₀ = (.03 .05 1.0 0 .01)′,⁶⁷ the desired path for the output gap equal to .01 and the parameters in the penalty matrices equal to λ_x = .0483, λ_t = .2364 and β = .99 yields the results in Table 2, for a time horizon of 2 periods.⁶⁸ In this example, robust control is more active than that associated with the familiar linear regulator problem (or quadratic linear problem) and it is identical to the TVP-control when the transition matrix Φ is equal to zero. For this problem specification, the TVP-control is higher than robust control when Φ is positive. The opposite is true for negative values of Φ. As already noticed the difference between the two controls gets larger and larger as Φ gets farther from the null matrix.

Table 2 Linear quadratic control (QLP), robust control and TVP-control at time 0

7 Conclusion

In this paper a robust control problem with unstructured uncertainty à la Hansen and Sargent, i.e. uncertainty that takes the form of a nonparametric set of additive mean-distorting model perturbations, and a decision maker who cares only of the induced distributions under the approximating model has been introduced. Then the more complicated case of a decision maker with a different constraint for each type of adverse shocks has been discussed. At this point both problems have been reformulated as linear quadratic tracking control problems where the system equations have a time-varying intercept following a mean reverting, or ‘Return to Normality’, model. By comparing the robust control solution with the associated TVP-control, when the same worst-case adverse shock and objective functional are used in both procedures, it is shown that in all these cases a decision maker who wants to be robust against misspecification of the approximating model implicitly assumes that today’s worst-case adverse shock is serially uncorrelated with tomorrow’s worst-case adverse shock. Moreover, the same result holds when uncertainty is not unstructured but is related with unknown structural parameters of the model.

This is a relevant conclusion because it applies to a large set of popular robust control frameworks in economics. As commonly understood, the robust control choice accounts for all possible kinds of persistence of malevolent shocks, which may take a much more general form than the VAR(1) assumed in these pages. Then, it is not immediately obvious why they look linearly independent in this set of models. The fact that the transition matrix does not appear in the relevant expression does not necessarily mean that the decision maker does not contemplate very persistent model misspecification shocks. For instance, the robust control in the worst case may not depend upon the transition matrix simply because the persistence of the misspecification shock does not affect the worst case. In other words, the robust decision maker accounts for the possible persistence of the misspecification shocks, and that persistence may affect the evolution of the control variables in other equilibria, but it happens that transition matrix does not play a role in the worst-case equilibrium. While for many possible “models”, these misspecification shocks may be very persistent, such models happen to result in lower welfare losses than the worst-case model. These are some of the aspects that need to be further investigated in order to assess more precisely how strong is the ‘immunization against uncertainty’ provided by the linear-quadratic robust control frameworks widely used in economics in discrete-time.

Appendices

Appendix 1

Proof of Proposition 1

Setting ε_t+1 = 0 and writing the optimal value of (7) as $- {\mathbf{y^{\prime}}}_{t} {\mathbf{P}}_{t} {\mathbf{y}}_{t} - 2{\mathbf{y^{\prime}}}_{t} {\mathbf{p}}_{t} ,$⁶⁹ the Bellman equation looks like⁷⁰

$$\begin{aligned} - {\mathbf{y^{\prime}}}_{t} {\mathbf{P}}_{t} {\mathbf{y}}_{t} - 2{\mathbf{y^{\prime}}}_{t} {\mathbf{p}}_{t} & = \mathop {ext}\limits_{{{\tilde{\mathbf{u}}}}} - \left[ {{\mathbf{y^{\prime}}}_{t} {\mathbf{Q}}_{t} {\mathbf{y}}_{t} } \right. + {\mathbf{u^{\prime}}}_{t} {\mathbf{R}}_{t} {\mathbf{u}}_{t} - \beta \theta {\varvec{\upomega}}^{\prime}_{t + 1} {\varvec{\upomega }}_{t + 1} + 2{\mathbf{y^{\prime}}}_{t} {\mathbf{W}}_{t} {\mathbf{u}}_{t} + 2{\mathbf{y^{\prime}}}_{t} {\mathbf{q}}_{t} \\ & \quad + \left. {2{\mathbf{u^{\prime}}}_{t} {\mathbf{r}}_{t} + {\mathbf{y^{\prime}}}_{t + 1} {\mathbf{P}}_{t + 1} {\mathbf{y}}_{t + 1} + 2{\mathbf{y^{\prime}}}_{t + 1} {\mathbf{p}}_{t + 1} } \right] \\ \end{aligned}$$

(47)

with ${\mathbf{P}}_{t + 1} = \beta {\mathbf{P}}_{t} ,{\mathbf{Q}}_{t} = \beta^{t} {\mathbf{Q}},{\mathbf{W}}_{t} = \beta^{t} {\mathbf{W}},{\mathbf{R}}_{t} = \beta^{t} {\mathbf{R}},{\mathbf{q}}_{t} = - \left( {{\mathbf{Q}}_{t} {\mathbf{y}}_{t}^{d} + {\mathbf{W}}_{t} {\mathbf{u}}_{t}^{d} } \right)$ and ${\mathbf{r}}_{t} = - ({\mathbf{R}}_{t} {\mathbf{u}}_{t}^{d} + {\mathbf{W^{\prime}}}_{t} {\mathbf{y}}_{t}^{d} )$.⁷¹ Then expressing the right-hand side of (47) only in terms of y_t and ${\tilde{\mathbf{u}}}_{t}$ and extremizing it yields the optimal control for the decision maker

$${\mathbf{u}}_{t} = - \left( {{\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{B}}} \right)^{ - 1} \left[ {\left( {{\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{A}} + {\mathbf{W}}_{t}^{\prime } } \right){\mathbf{y}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{C}}{\varvec{\upomega }}_{t + 1} + {\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1} + {\mathbf{r}}_{t} } \right]$$

(48)

and the optimal control for the malevolent nature

$${\varvec{\upomega }}_{t + 1} = (\beta \theta {\mathbf{I}}_{l} - {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{C}})^{ - 1} ({\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{Ay}}_{t} + {\mathbf{C}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{Bu}}_{t} + {\mathbf{C}}^{\prime} {\mathbf{p}}_{t + 1} ).$$

(49)

By substituting (49) into (48) and (48) into (49) gives the θ-constrained worst-case controls in Proposition 1.

As noted in Hansen and Sargent (2007a, p. 11), the first order conditions for problem (47) subject to (9) imply the matrix Riccati equation

$${\mathbf{P}}_{t} = {\mathbf{Q}}_{t} + {\mathbf{A^{\prime}P}}_{t + 1} {\mathbf{A}} - \left( {{\mathbf{A^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}} + {\tilde{\mathbf{W}}}_{t} } \right)\left( {{\mathbf{\tilde{B}^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}} + {\tilde{\mathbf{R}}}_{t} } \right)^{ - 1} \left( {{\mathbf{A^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}} + {\tilde{\mathbf{W}}}_{t} } \right)^{\prime } ,$$

(50)

where ${\tilde{\mathbf{R}}}_{t} = \beta^{t} {\tilde{\mathbf{R}}}$ and ${\tilde{\mathbf{W}}}_{t} = \beta^{t} {\tilde{\mathbf{W}}}$, and the Riccati equation for the p vector⁷²

$${\mathbf{p}}_{t} = {\mathbf{q}}_{t} + {\mathbf{A^{\prime}p}}_{t + 1} - \left( {{\mathbf{A^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}} + {\tilde{\mathbf{W}}}_{t} } \right)\left( {{\mathbf{\tilde{B^{\prime}}P}}_{t + 1} {\tilde{\mathbf{B}}} + {\tilde{\mathbf{R}}}_{t} } \right)^{ - 1} \left( {{\mathbf{\tilde{B^{\prime}}p}}_{t + 1} + {\tilde{\mathbf{r}}}_{t} } \right)$$

(51)

where ${\tilde{\mathbf{R}}}_{t} = \beta^{t} {\tilde{\mathbf{R}}},{\tilde{\mathbf{W}}}_{t} = \beta^{t} {\tilde{\mathbf{W}}}$ and ${\tilde{\mathbf{r}}}_{t} = (\begin{array}{*{20}c} {{\mathbf{r^{\prime}}}_{t} } & {{\mathbf{0^{\prime}}}_{l} } \\ \end{array} )^{\prime}$. It is straightforward to show that the right-hand side of Eqs. (50)–(51) can be rewritten as

$${\mathbf{Q}}_{t} + {\mathbf{A^{\prime}P}}_{t + 1}^{*} {\mathbf{A}} - \left( {{\mathbf{A^{\prime}P}}_{t + 1}^{*} {\mathbf{B}} + {\mathbf{W}}_{t} } \right)\left( {{\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1}^{*} {\mathbf{B}}} \right)^{ - 1} \left( {{\mathbf{A^{\prime}P}}_{t + 1}^{*} {\mathbf{B}} + {\mathbf{W}}_{t} } \right)^{\prime }$$

and

$${\mathbf{q}}_{t} + {\mathbf{A^{\prime}p}}_{t + 1}^{*} - \left( {{\mathbf{A^{\prime}P}}_{t + 1}^{*} {\mathbf{B}} + {\mathbf{W}}_{t} } \right)\left( {{\mathbf{B^{\prime}P}}_{t + 1}^{*} {\mathbf{B}} + {\mathbf{R}}_{t} } \right)^{ - 1} \left( {{\mathbf{B^{\prime}p}}_{t + 1}^{*} + {\mathbf{r}}_{t} } \right),$$

respectively. Equations (50) and (51) reduce to the usual Riccati equations of the linear quadratic tracking control problem when ω_t+1 = 0.⁷³

Appendix 2

Proof of Proposition 2

As in “Appendix 1”, writing the optimal value of (18) as $- {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{P}}_{t} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} - 2{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}p}}_{t} ,$ the Bellman equation looks like

$$\begin{aligned} - {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{P}}_{t} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} - 2{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{p}}_{t} & = \mathop {ext}\limits_{{{\mathbf{u}},{\varvec{\upomega }}}} - \left[ {{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{Q}}_{t} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} + {\mathbf{u^{\prime}}}_{t} {\mathbf{R}}_{t} {\mathbf{u}}_{t} } \right. + 2{\mathbf{u^{\prime}}}_{t} {\mathbf{E}}_{t}^{*} {\varvec{\upomega }}_{t} - {\varvec{\upomega}}^{\prime}_{t} {\varvec{\Delta }}_{{{\varvec{\upomega }},t}} ({\varvec{\Delta }}_{t} ){\varvec{\upomega }}_{t} \\ & \quad + 2{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{W}}_{t} {\mathbf{u}}_{t} + 2{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{M}}_{t}^{*} {\varvec{\upomega }}_{t} + 2{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y}^{\prime}}}_{t} {\mathbf{q}}_{t} \left. { + 2{\mathbf{u^{\prime}}}_{t} {\mathbf{r}}_{t} + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y^{\prime}} }}_{t + 1} {\mathbf{P}}_{ + 1t} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t + 1} + 2{\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y^{\prime}} }}_{t + 1} {\mathbf{p}}_{t + 1} } \right] \\ \end{aligned}$$

(52)

with ${\mathbf{P}}_{t + 1} = \beta {\mathbf{P}}_{t} ,{\mathbf{Q}}_{t} = \beta^{t} {\mathbf{Q}},{\mathbf{R}}_{t} = \beta^{t} {\mathbf{R}},{\mathbf{E}}_{t}^{*} = \beta^{t} {\mathbf{E}}^{*} ,{\varvec{\Delta }}_{{{\varvec{\upomega }},t}} ({\varvec{\Delta }}_{t} ) = \beta^{t} {\varvec{\Delta }}_{{\varvec{\upomega }}} ({\varvec{\Delta }}_{t} ),{\mathbf{W}}_{t} = \beta^{t} {\mathbf{W}},{\mathbf{M}}_{t}^{*} = \beta^{t} {\mathbf{M}}^{*} ,{\mathbf{q}}_{t} = - \left( {{\mathbf{Q}}_{t} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t}^{d} + {\mathbf{W}}_{t} {\mathbf{u}}_{t}^{d} } \right)$ and ${\mathbf{r}}_{t} = - ({\mathbf{R}}_{t} {\mathbf{u}}_{t}^{d} + {\mathbf{W}}_{t} \prime {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t}^{d} ).$ Then expressing the right-hand side of (52) only in terms of ${\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t}$ and ${\tilde{\mathbf{u}}}_{t}$ and extremizing it yields the optimal control for the decision maker, i.e.

$$\begin{aligned} {\mathbf{u}}_{t} & = - \left( {{\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{B}}} \right)^{ - 1} \\ & \quad \times \left[ {\left( {{\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} } \right){\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} + \left( {{\mathbf{E}}_{t}^{*} + {\mathbf{B}}^{\prime} {\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )} \right){\varvec{\upomega }}_{t} + {\mathbf{B}}^{\prime} {\mathbf{p}}_{t + 1} + {\mathbf{r}}_{t} } \right], \\ \end{aligned}$$

(53)

and the optimal control for the malevolent nature

$${\varvec{\upomega }}_{t} = {\varvec{\Theta }}_{t}^{ - 1} \left[ {\left( {{\mathbf{M}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{A}}} \right){\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{y} }}_{t} + \left( {{\mathbf{E}}_{t}^{*\prime } + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{B}}} \right){\mathbf{u}}_{t} + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{p}}_{t + 1} } \right]$$

(54)

with ${\varvec{\Theta }}_{t} = - [{\varvec{\Delta }}_{{{\varvec{\upomega }},t}} ({\varvec{\Delta }}_{t} ) + {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )^{\prime}{\mathbf{P}}_{t + 1} {\mathbf{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\smile}$}}{C} }}({\varvec{\Delta }}_{t} )]$. By substituting (54) into (53) yields the result in Proposition 2.

As in Proposition 1, the first order conditions for problem (18) subject to (19) imply the matrix Riccati equation

$$\begin{aligned} {\mathbf{P}}_{t} & = {\mathbf{Q}}_{t} + {\mathbf{A^{\prime}P}}_{t + 1} {\mathbf{A}} \\ & \quad - \left[ {{\mathbf{A^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ) + {\tilde{\mathbf{W}}}_{t} } \right]\left[ {{\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} )^{\prime } {\mathbf{P}}_{t + 1} {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ) + {\tilde{\mathbf{R}}}_{t} ({\varvec{\Delta }}_{t} )} \right]^{ - 1} \left[ {{\mathbf{A^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ) + {\tilde{\mathbf{W}}}_{t} } \right]^{\prime } \\ \end{aligned}$$

(55)

where ${\tilde{\mathbf{R}}}_{t} ({\varvec{\Delta }}_{t} ) = \beta^{t} {\tilde{\mathbf{R}}}({\varvec{\Delta }}_{t} )$ and ${\tilde{\mathbf{W}}}_{t} = \beta^{t} {\tilde{\mathbf{W}}}$. The Riccati equation for the p vector looks like⁷⁴

$$\begin{aligned} {\mathbf{p}}_{t} & = {\mathbf{q}}_{t} + {\mathbf{A^{\prime}p}}_{t + 1} \\ & \quad - \left[ {{\mathbf{A^{\prime}P}}_{t + 1} {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ) + {\tilde{\mathbf{W}}}_{t} } \right]\left[ {{\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} )^{\prime } {\mathbf{P}}_{t + 1} {\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} ) + {\tilde{\mathbf{R}}}_{t} ({\varvec{\Delta }}_{t} )} \right]^{ - 1} \left[ {{\tilde{\mathbf{B}}}({\varvec{\Delta }}_{t} )^{\prime } {\mathbf{p}}_{t + 1} + {\mathbf{r}}_{t} } \right]. \\ \end{aligned}$$

(56)

Both Eqs. (55) and (56) reduce to the usual Riccati equations of the linear quadratic tracking control problem when ${\varvec{\upomega }}_{t} = {\mathbf{0}}$.

Appendix 3

Proof of Proposition 3

By replacing ${\mathbf{A}}_{1} {\mathbf{y}}_{t} + {\mathbf{C}}{\varvec{\upalpha}}_{t + 1}$ with ${\mathbf{Ay}}_{t} + {\mathbf{C}}{\varvec{\upnu}}_{t + 1}$ in (27), treating the vector of stochastic components ν_t+1 as additional state variables, setting ${\varvec{\upvarepsilon}}_{t + 1}^{*} = {\mathbf{0}}$ and using the deterministic counterpart to (25)–(28), namely

$${\mathbf{z}}_{t + 1} = {\mathbf{A}}^{*} {\mathbf{z}}_{t} + {\mathbf{B}}^{*} {\mathbf{u}}_{t} \quad {\text{for}}\quad t = 0, \ldots ,\infty ,$$

(57)

with

$${\mathbf{z}}_{t} = \left[ {\begin{array}{*{20}c} {{\mathbf{y}}_{t} } \\ {{\varvec{\upnu }}_{t + 1} } \\ \end{array} } \right],\quad {\mathbf{A}}^{*} = \left[ {\begin{array}{*{20}c} {\mathbf{A}} & {\mathbf{C}} \\ {\mathbf{O}} & {\varvec{\Phi }} \\ \end{array} } \right]\quad {\text{and}}\quad {\mathbf{B}}^{*} = \left[ {\begin{array}{*{20}c} {\mathbf{B}} \\ {\mathbf{O}} \\ \end{array} } \right] ,$$

(58)

the optimal value of (25) can be written as $- {\mathbf{z^{\prime}}}_{t} {\mathbf{K}}_{t} {\mathbf{z}}_{t} - 2{\mathbf{z^{\prime}}}_{t} {\mathbf{k}}_{t}$ and it satisfies the Bellman equation

$$\begin{aligned} - {\mathbf{z^{\prime}}}_{t} {\mathbf{K}}_{t} {\mathbf{z}}_{t} - 2{\mathbf{z^{\prime}}}_{t} {\mathbf{k}}_{t} & = \mathop {\hbox{max} }\limits_{{{\mathbf{u}}_{t} }} - \left[ {\left( {{\mathbf{z}}_{t} - {\mathbf{z}}_{t}^{*} } \right)^{\prime } {\mathbf{Q}}_{t}^{*} \left( {{\mathbf{z}}_{t} - {\mathbf{z}}_{t}^{*} } \right)} \right. + \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{*} } \right)^{\prime } {\mathbf{R}}_{t} \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{*} } \right) \\ & \quad \left. { + 2\left( {{\mathbf{z}}_{t} - {\mathbf{z}}_{t}^{*} } \right)^{\prime } {\mathbf{W}}_{t}^{*} \left( {{\mathbf{u}}_{t} - {\mathbf{u}}_{t}^{*} } \right) + {\mathbf{z^{\prime}}}_{t + 1} {\mathbf{K}}_{t + 1} {\mathbf{z}}_{t + 1} + 2{\mathbf{z^{\prime}}}_{t + 1} {\mathbf{k}}_{t + 1} } \right] \\ \end{aligned}$$

(59)

with ${\mathbf{K}}_{t + 1} = \beta {\mathbf{K}}_{t}$. Again, expressing the right-hand side of (59) only in terms of z_t and u_t and maximizing it yields the optimal control in the presence of time-varying parameters (or TVP-control), i.e.⁷⁵

$$\begin{aligned} {\mathbf{u}}_{t} & = - \left( {{\mathbf{R}}_{t} + {\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{B}}} \right)^{ - 1} \\ & \quad \times \left[ {\left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} } \right){\mathbf{y}}_{t} } \right. + {\mathbf{B^{\prime}}}\left( {{\mathbf{K}}_{11,t + 1} {\mathbf{C}} + {\mathbf{K}}_{12,t + 1} {\varvec{\Phi }}} \right){\varvec{\upnu }}_{t + 1} \left. { + \left( {{\mathbf{B^{\prime}k}}_{1,t + 1} + {\mathbf{r}}_{t} } \right)} \right]. \\ \end{aligned}$$

(60)

The matrices K₁₁ and K₁₂ in (60) denote the n × n North-West block and the n × l North-East block, respectively, of the Riccati matrix

$$\begin{aligned} {\mathbf{K}}_{t} & = {\mathbf{Q}}_{t}^{*} + {\mathbf{A}}^{*} \prime {\mathbf{K}}_{t + 1} {\mathbf{A}}^{*} \\ & \quad - \left( {{\mathbf{B}}^{*} \prime {\mathbf{K}}_{t + 1} {\mathbf{A}}^{*} + {\mathbf{W}}_{t}^{*} } \right)^{\prime } \left( {{\mathbf{R}}_{t} + {\mathbf{B}}^{*} \prime {\mathbf{K}}_{t + 1} {\mathbf{B}}^{*} } \right)^{ - 1} \left( {{\mathbf{B}}^{*} \prime {\mathbf{K}}_{t + 1} {\mathbf{A}}^{*} + {\mathbf{W}}_{t}^{*} } \right) \\ \end{aligned}$$

(61)

Consequently they are defined as⁷⁶

$$\begin{aligned} {\mathbf{K}}_{11,t} & = {\mathbf{Q}}_{t} + {\mathbf{A}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{A}} \\ & \quad - ({\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W}}_{t} \prime )\prime ({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{B}})^{ - 1} ({\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W}}_{t} \prime ) \\ \end{aligned}$$

(62a)

and

$$\begin{aligned} {\mathbf{K}}_{12,t} & = ({\mathbf{A}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{C}} + {\mathbf{A}}^{\prime} {\mathbf{K}}_{12,t + 1} {\varvec{\Phi }}) \\ & \quad - ({\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} )\prime ({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{B}})^{ - 1} {\mathbf{B}}^{\prime} ({\mathbf{K}}_{11,t + 1} {\mathbf{C}} + {\mathbf{K}}_{12,t + 1} {\varvec{\Phi }}). \\ \end{aligned}$$

(62b)

It is apparent from (62b) that even when K_12,t+1 is zero, it is sufficient that the same is not true for K_11,t+1, a condition typically met, to have a non zero K_12,t matrix. This means that even when the terminal condition for K₁₂ is a null matrix, this array will not vanish in all the time periods in the planning horizon except for the last one. Consequently, only the last control, namely the control applied at the ‘final period minus 1’ of the planning horizon, will be independent of the transition matrix characterizing the time-varying parameters.

The optimal control (60) is independent of the parameter θ which enters the l × l South-East block of K, namely

$$\begin{aligned} {\mathbf{K}}_{22,t} & = - (\beta \theta {\mathbf{I}}_{l} - {\mathbf{C}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{C}}) + 2{\varvec{\Phi }}^{\prime} {\mathbf{K}}_{21,t + 1} {\mathbf{C}} + {\varvec{\Phi }}^{\prime} {\mathbf{K}}_{22,t + 1} {\varvec{\Phi }} \\ & \quad - ({\mathbf{C}}^{\prime} {\mathbf{K}}_{11,t + 1} + {\varvec{\Phi }}^{\prime} {\mathbf{K}}_{21,t + 1} ){\mathbf{B}}({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{B}})^{ - 1} {\mathbf{B}}^{\prime} ({\mathbf{K}}_{11,t + 1} {\mathbf{C}} + {\mathbf{K}}_{12,t + 1} {\varvec{\Phi }}). \\ \end{aligned}$$

(62c)

However it depends upon the vector k_t which can be partitioned as ${\mathbf{k}}_{t} = [\begin{array}{*{20}c} {{\mathbf{k^{\prime}}}_{1,t} } & {{\mathbf{k^{\prime}}}_{2,t} } \\ \end{array} ]^{{\prime }}$ with

$${\mathbf{k}}_{1,t} = {\mathbf{q}}_{t} + {\mathbf{A}}^{\prime} {\mathbf{k}}_{1,t + 1} - ({\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} )\prime ({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{B}})^{ - 1} ({\mathbf{B}}^{\prime} {\mathbf{k}}_{1,t + 1} + {\mathbf{r}}_{t} )$$

(63a)

$$\begin{aligned} {\mathbf{k}}_{2,t} & = {\mathbf{C}}^{\prime} {\mathbf{k}}_{1,t + 1} + {\varvec{\Phi }}^{\prime} {\mathbf{k}}_{2,t + 1} \\ & \quad - ({\mathbf{C}}^{\prime} {\mathbf{K}}_{11,t + 1} + {\varvec{\Phi }}^{\prime} {\mathbf{K}}_{21,t + 1} ){\mathbf{B}}({\mathbf{R}}_{t} + {\mathbf{B}}^{\prime} {\mathbf{K}}_{11,t + 1} {\mathbf{B}})^{ - 1} ({\mathbf{B}}^{\prime} {\mathbf{k}}_{1,t + 1} + {\mathbf{r}}_{t} ). \\ \end{aligned}$$

(63b)

When ${\varvec{\upnu }}_{t + 1} \equiv {\varvec{\upomega }}_{t + 1}$, i.e. the same shock is used to determine both robust control and TVP-control, the latter is as defined in Proposition 3.

Appendix 4

Proof of Proposition 4

Proceeding as in the proof of Proposition 3 it follows that for this problem the TVP-control looks like

$$\begin{aligned} {\mathbf{u}}_{t} & = - \left( {{\mathbf{R}}_{t} + {\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{B}}} \right)^{ - 1} \\ & \quad \times \left[ {\left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{A}} + {\mathbf{W^{\prime}}}_{t} } \right){\mathbf{y}}_{t} } \right. + \left( {{\mathbf{B^{\prime}K}}_{11,t + 1} {\mathbf{C}} + {\mathbf{B^{\prime}K}}_{12,t + 1} {\varvec{\Phi }} + {\mathbf{E}}_{t}^{*} } \right){\varvec{\upnu }}_{t + 1} \left. { + \left( {{\mathbf{B^{\prime}k}}_{1,t + 1} + {\mathbf{r}}_{t} } \right)} \right]. \\ \end{aligned}$$

Then setting ${\varvec{\upnu }}_{t + 1} \equiv [\begin{array}{*{20}c} {\varvec{\upomega}}^{\prime}_{1,t} & {\varvec{\upomega}}^{\prime}_{2,t} \\ \end{array} ]^{\prime}$ with ω_1,t and ω_2,t as in Sect. 3 yields Proposition 4.