# Adaptive Control in the Presence of Time-Varying Parameters

• Marco P. Tucci
Part of the Advances in Computational Economics book series (AICE, volume 19)

## Abstract

In the previous chapters two strands of literature have been brought together. The first strand is the research on time-varying parameters following a ‘Return to Normality’ model, viz. Harvey and Phillips (1982) and Swamy and Tinsley (1980).1 The second strand is the work on adaptive control in the tradition of Tse and Bar-Shalom (1973) and Kendrick (1981). As indicated in Chapter 2, given a time horizon of T periods, the goal was to find the values of control variables for period 0, period 1 and so on until period T−1 which minimized the objective functional, or cost-to-go, in an adaptive control framework. For each period in the time horizon, say period t, several trial, or search, values of the controls were tried.2 The optimal cost-to-go associated with each ‘search’ control was then calculated. Following Kendrick (1981, Ch. 10) the computations performed at time t were organized in three steps. First the nominal value for the parameters for period t through T was computed using their estimates for time t−1 and their ‘expected’ law of motion. The first ‘search’ control was also selected. Then the search for the optimal control was carried out. This step included: a) use the ‘search’ control and the nominal value for the parameters to get the projected states and covariances in period t+1; b) get the nominal path for the states and controls for the period t+1 through T by solving the certainty equivalence (CE) problem with the projected states as the initial condition; c) compute the Riccati matrices for periods t+1, ..., T; d) update and project the covariances from period t+1 given t, i.e. the covariance seen at point a) above, to T−1; e) calculate the approximate cost-to-go for the period t through T; f) choose a new ‘search’ control and repeat a)-f) until all `search’ points are evaluated; g) select the control which yields the minimum cost. After the selected control was applied to the system and the process moved one step forward in time, the estimates of the states and parameters were updated.

## Keywords

Time Horizon Adaptive Control Riccati Equation Certainty Equivalent Previous Chapter
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## Notes

1. 1.
See also, among the others, Belsley and Kuh (1973), Brown et al. (1975), Cooley and Prescott (1976), Goldfeld and Quandt (1973), Hildreth and Houck (1968), Quandt (1972) and Swamy (1970) for their pioneering works on the use of time-varying parameters in econometrics and Abutaleb and Papaioannou (2000), Blangiewicz and Charemza (2001), Chang et al. (2002) and Shen (2000) for some recent applications.Google Scholar
2. 2.
Here and in the following the singular ‘control’ is sometimes used even when a ‘vector of control variables’ is implied.Google Scholar
3. 3.
Equation (6) in Tucci (1997, p. 42) is clearly wrong. It should be replaced by [5–8] below.Google Scholar
4. 4.
A critical introduction to time-varying parameters can be found in Tucci (1995).Google Scholar
5. 5.
When the states are measured without error x0 is given. Again 0 denotes a null vector and O a null matrix unless otherwise noted.Google Scholar
6. 6.
The results of this chapter hold even when H is time-varying but independent of βt.Google Scholar
7. 7.
See Section 2.2 for details.Google Scholar
8. 8.
See Section 3.2 for details.Google Scholar
9. 9.
See Section 4.2 for details.Google Scholar
10. 10.
Matrix HZ is defined as HZ=[H O], HZ=[H O O] and HZ=[H O O O O] in case i), ii) and iii), respectively. Similarly, the definition of vt is, respectively, vtt′ 0′]′, vtt00′]′ and vtt0000′]′. The variance of εt, namely Qz, is assumed time invariant.Google Scholar
11. 11.
As in the previous chapters, the vector ei has the effect of ‘placing’ the scalar quantity tr[.] in the i-th row of the vector on the left-hand side of the equality sign.Google Scholar
12. 12.
As usual by now all derivatives are evaluated along the nominal path.Google Scholar
13. 13.
As in the previous chapters, Aβi is the n×k matrix of the derivatives of the i-th row of A, transposed, with respect to β′ and Σt|tβx is the covariance between the subvectors β and x. When only the intercept and/or the parameters associated with the controls are assumed time-varying, the last term on the right-hand side of [5–18] disappears.Google Scholar
14. 14.
To be consistent with the notation adopted in [5–16] the superscripts and subscripts Φ and Ξ should be replaced by vec(Φ) and vech(Ξ), respectively.Google Scholar
15. 15.
See Chapter 4 for details. The array Ik is of dimension k×nΦ and can be partitioned as with nΦk the number of estimates in the k-th row of Φ, Σk=1K, nΦk =nΦ, the O’s null matrices of appropriate dimension and a matrix k×nΦk with each column having 1 in the position corresponding to the parameter associated with the estimated element of the k-th row of Φ and zero elsewhere. When Φ is diagonal, Ik is an identity matrix of order k.Google Scholar
16. 16.
As in Chapter 4, the matrix Bot= diag(B1′, B2′, Bk′) is of dimension k×nΦ with. Some of the covariances may be zero as argued in Section 3.2 and 4.2.Google Scholar
17. 17.
As stressed in the previous chapters, the nominal path for the parameters is generated during the first step of the search procedure performed at each time period t.Google Scholar
18. 18.
This is to remind the reader that K matrix take the form the recursions for the where the time subscript is j unless otherwise specified.Google Scholar
19. 19.
As reported in the previous chapters the submatrices of K associated with the new random variables included in the augmented state vector have a form similar to which is the same in case ii) and iii), with all quantities having subscript j unless otherwise specified and with A, B and Φ set at their nominal values.Google Scholar
20. 20.
A penalty on an element of the b vector would discourage, ceteris paribus, the use of the control associated with that parameter whereas a penalty attached to an element of E would discourage the use of controls whose effect is highly uncertain.Google Scholar
21. 21.
See the previous chapters for details.Google Scholar
22. 22.
As in the previous chapters, the subscript ‘d’ is used to indicate the portion of the approximate cost-to-go which depends on the control applied at time t.Google Scholar
23. 23.
The relationship between Θj and Kj+1 is briefly described in footnote 18. See the previous chapters for details.Google Scholar
24. 24.
When b,Φ and Ξ are assumed unknown but fixed, as in the text, the second term on the right-hand side of [5–38] is the same in all cases.Google Scholar
25. 25.
Analogously the cautionary component in case ii) is likely to be larger than in case i) because of the K and Σ submatrices relative to the vector b and for reasons similar to those mentioned in the remainder of the paragraph.Google Scholar
26. 26.
Analogous considerations apply to the relationship between case ii) and i).Google Scholar
27. 27.
DUALTVP is the time-varying parameter version, coded by the author, of the original active learning DUAL code developed by Kendrick (1981).Google Scholar
28. 28.
See Kendrick (1981, chapters 5–7) for a nice introduction to OLF methods.Google Scholar
29. 29.
30. 30.
Except for the ‘time-varying’ assumption, this is the first experiment in Amman and Kendrick (1994).Google Scholar
31. 31.
Indeed Kendrick (1981) calls the transition matrix D rather than Φ as in the text.Google Scholar
32. 32.
This case corresponds to Kendrick’s (1981) case discussed in Chapter 2.Google Scholar
33. 33.
When V(βt)=1, the variance of b0 is set equal to.498 and V(ξ)=.01.Google Scholar
34. 34.
The standard deviation of the mean, say sdev, is s/nl/2 with s the standard deviation reported in the tables and n the number of Monte Carlo runs. Therefore a 95% confidence interval for the mean cost is approximately mean cost ± 2sdev.Google Scholar