The Rational Expectation Hypothesis, Time-Varying Parameters and Adaptive Control pp 71-90 | Cite as

# Adaptive Control in the Presence of Time-Varying Parameters

## Abstract

In the previous chapters two strands of literature have been brought together. The first strand is the research on time-varying parameters following a ‘Return to Normality’ model, viz. Harvey and Phillips (1982) and Swamy and Tinsley (1980).^{1} The second strand is the work on adaptive control in the tradition of Tse and Bar-Shalom (1973) and Kendrick (1981). As indicated in Chapter 2, given a time horizon of T periods, the goal was to find the values of control variables for period 0, period 1 and so on until period T−1 which minimized the objective functional, or cost-to-go, in an adaptive control framework. For each period in the time horizon, say period t, several trial, or search, values of the controls were tried.^{2} The optimal cost-to-go associated with each ‘search’ control was then calculated. Following Kendrick (1981, Ch. 10) the computations performed at time t were organized in three steps. First the nominal value for the parameters for period t through T was computed using their estimates for time t−1 and their ‘expected’ law of motion. The first ‘search’ control was also selected. Then the search for the optimal control was carried out. This step included: a) use the ‘search’ control and the nominal value for the parameters to get the projected states and covariances in period t+1; b) get the nominal path for the states and controls for the period t+1 through T by solving the certainty equivalence (CE) problem with the projected states as the initial condition; c) compute the Riccati matrices for periods t+1, ..., T; d) update and project the covariances from period t+1 given t, i.e. the covariance seen at point a) above, to T−1; e) calculate the approximate cost-to-go for the period t through T; f) choose a new ‘search’ control and repeat a)-f) until all `search’ points are evaluated; g) select the control which yields the minimum cost. After the selected control was applied to the system and the process moved one step forward in time, the estimates of the states and parameters were updated.

## Keywords

Time Horizon Adaptive Control Riccati Equation Certainty Equivalent Previous Chapter## Preview

Unable to display preview. Download preview PDF.

## Notes

- 1.See also, among the others, Belsley and Kuh (1973), Brown et al. (1975), Cooley and Prescott (1976), Goldfeld and Quandt (1973), Hildreth and Houck (1968), Quandt (1972) and Swamy (1970) for their pioneering works on the use of time-varying parameters in econometrics and Abutaleb and Papaioannou (2000), Blangiewicz and Charemza (2001), Chang et al. (2002) and Shen (2000) for some recent applications.Google Scholar
- 2.Here and in the following the singular ‘control’ is sometimes used even when a ‘vector of control variables’ is implied.Google Scholar
- 3.Equation (6) in Tucci (1997, p. 42) is clearly wrong. It should be replaced by [5–8] below.Google Scholar
- 4.A critical introduction to time-varying parameters can be found in Tucci (1995).Google Scholar
- 5.When the states are measured without error x
_{0}is given. Again**0**denotes a null vector and**O**a null matrix unless otherwise noted.Google Scholar - 6.The results of this chapter hold even when
**H**is time-varying but independent of β_{t}.Google Scholar - 7.See Section 2.2 for details.Google Scholar
- 8.See Section 3.2 for details.Google Scholar
- 9.See Section 4.2 for details.Google Scholar
- 10.Matrix
**H**^{Z}is defined as**H**^{Z}=[**H O**],**H**^{Z}=[**H O O**] and**H**^{Z}=[**H O O O O**] in case i), ii) and iii), respectively. Similarly, the definition of v_{t}is, respectively,**v**_{t}[ρ_{t}′ 0′]′,**v**_{t}[ρ_{t}′**0**′**0**′]′ and**v**_{t}[ρ_{t}′**0**′**0**′**0**′**0**′]′. The variance of ε_{t}, namely**Q**^{z}, is assumed time invariant.Google Scholar - 11.As in the previous chapters, the vector e
^{i}has the effect of ‘placing’ the scalar quantity tr[.] in the*i*-th row of the vector on the left-hand side of the equality sign.Google Scholar - 12.As usual by now all derivatives are evaluated along the nominal path.Google Scholar
- 13.As in the previous chapters, A
_{β}^{i}is the n×k matrix of the derivatives of the*i*-th row of**A**, transposed, with respect to β′ and Σ_{t|t}^{βx}is the covariance between the subvectors β and**x**. When only the intercept and/or the parameters associated with the controls are assumed time-varying, the last term on the right-hand side of [5–18] disappears.Google Scholar - 14.To be consistent with the notation adopted in [5–16] the superscripts and subscripts Φ and Ξ should be replaced by vec(Φ) and vech(Ξ), respectively.Google Scholar
- 15.See Chapter 4 for details. The array I
^{k}is of dimension k×n_{Φ}and can be partitioned as with n_{Φ}^{k}the number of estimates in the*k*-th row of Φ, Σ*k*_{=}1^{K}, n_{Φ}^{k}=n_{Φ}, the**O**’s null matrices of appropriate dimension and a matrix k×n_{Φ}^{k}with each column having 1 in the position corresponding to the parameter associated with the estimated element of the*k*-th row of Φ and zero elsewhere. When Φ is diagonal, I_{k}is an identity matrix of order k.Google Scholar - 16.As in Chapter 4, the matrix
**B**_{ot}= diag(**B**^{1}′,**B**^{2}′,**B**^{k}′) is of dimension k×n_{Φ}with. Some of the covariances may be zero as argued in Section 3.2 and 4.2.Google Scholar - 17.As stressed in the previous chapters, the nominal path for the parameters is generated during the first step of the search procedure performed at each time period t.Google Scholar
- 18.This is to remind the reader that
**K**matrix take the form the recursions for the where the time subscript is j unless otherwise specified.Google Scholar - 19.As reported in the previous chapters the submatrices of K associated with the new random variables included in the augmented state vector have a form similar to which is the same in case ii) and iii), with all quantities having subscript j unless otherwise specified and with
**A**,**B**and Φ set at their nominal values.Google Scholar - 20.A penalty on an element of the b vector would discourage, ceteris paribus, the use of the control associated with that parameter whereas a penalty attached to an element of E would discourage the use of controls whose effect is highly uncertain.Google Scholar
- 21.See the previous chapters for details.Google Scholar
- 22.As in the previous chapters, the subscript ‘d’ is used to indicate the portion of the approximate cost-to-go which depends on the control applied at time t.Google Scholar
- 23.The relationship between Θ
_{j}and**K**j_{+1}is briefly described in footnote 18. See the previous chapters for details.Google Scholar - 24.When
**b**,Φ and Ξ are assumed unknown but fixed, as in the text, the second term on the right-hand side of [5–38] is the same in all cases.Google Scholar - 25.Analogously the cautionary component in case ii) is likely to be larger than in case i) because of the
**K**and Σ submatrices relative to the vector**b**and for reasons similar to those mentioned in the remainder of the paragraph.Google Scholar - 26.Analogous considerations apply to the relationship between case ii) and i).Google Scholar
- 27.DUALTVP is the time-varying parameter version, coded by the author, of the original active learning DUAL code developed by Kendrick (1981).Google Scholar
- 28.See Kendrick (1981, chapters 5–7) for a nice introduction to OLF methods.Google Scholar
- 29.See also next chapter.Google Scholar
- 30.Except for the ‘time-varying’ assumption, this is the first experiment in Amman and Kendrick (1994).Google Scholar
- 31.Indeed Kendrick (1981) calls the transition matrix
**D**rather than Φ as in the text.Google Scholar - 32.This case corresponds to Kendrick’s (1981) case discussed in Chapter 2.Google Scholar
- 33.
- 34.The standard deviation of the mean, say
*sdev*, is s/n^{l/2}with s the standard deviation reported in the tables and n the number of Monte Carlo runs. Therefore a 95% confidence interval for the mean cost is approximately mean cost ± 2*sdev*.Google Scholar