Using Revisions as a Measure of Price Index Quality in Repeat-Sales Models

Abstract

Repeat-sales indexes are the most widely used type of transaction based property price indexes. However, such indexes are particularly prone to revision. When a new period of transaction data becomes available and is used to update the repeat-sales model, all past index values can potentially be revised. These revisions are especially problematical for commercial real estate (as compared to housing), due to the relative scarcity of transaction data and the heterogeneity of the underlying properties. From a methodological perspective, the magnitude of the revisions is a particularly useful measure of the index quality, as it directly reflects both the precision of the index and its practical usefulness in economic and business applications. This paper focuses on index revisions in thin, commercial property markets, the type of market that is most challenging. We present multiple specifications of the repeat-sales model (both existing and new), seeking to reduce revisions. We are able to reduce overall index revisions by more than 50%, compared to more traditional repeat-sales models.

Appendices

Appendix A: Revisions in Repeat-Sales Models

A.1 Revisions in the Standard Repeat-Sales Model

The way indexes are revised over time is illustrated by Bailey et al. (1963) and Clapp and Giaccotto (1999). Suppose we have only three periods, t = 0, 1, 2, and we are interested in the revision subsequent to time 1 of an index at time 1, originally estimated from transactions in period 0 and 1. That is, consider the revision of the index history when the transactions for period 2 become available. Let $\bar {r}_{st}$ be the average log price change between the period of buy s and the period of sell t and let $\hat {\delta }_{T|\tau }$ be the estimated log index return for period T conditional on all information (transactions) up to time τ, where τ ≥ T. The initial estimate of the log index return for period 1 is simply the average return of the properties in period 1, $\hat {\delta }_{1|1} = \bar {r}_{01}$ with variance $2\sigma ^{2}_{\epsilon } / n_{01}$, where n_st is the number of transaction pairs with time of buy s and time of sell t. Now the estimated log index return for period 1 conditional on all information up to time 2 is a weighted average of the original estimate $\hat {\delta }_{1|1}$ and new information with respect to δ₁ based on the difference $(\bar {r}_{02} - \bar {r}_{12})$, with variance $2 \sigma ^{2}_{\epsilon } (1/n_{02} + 1 / n_{12})$,

$$\hat{\delta}_{1|2} = \frac{(n_{02}+n_{12})n_{01} \bar{r}_{01} + n_{02} n_{12} (\bar{r}_{02} - \bar{r}_{12})} {n_{01}(n_{02}+n_{12}) + n_{12}n_{02}} =w r_{01} + (1 - w)(\bar{r}_{02} - \bar{r}_{12}), $$

where the weight w = (n₀₁(n₀₂ + n₁₂)) / (n₀₁(n₀₂ + n₁₂) + n₀₂n₁₂) depends on the variances of $\bar {r}_{01}$ and $(\bar {r}_{02} - \bar {r}_{12})$. The revision $\hat {\delta }_{1|2} - \hat {\delta }_{1|1}$ can be expressed as $(1-w)(\bar {r}_{02} - \bar {r}_{12} - \bar {r}_{01})$.

Let us illustrate the revision by a numerical example. Assume that $\bar {r}_{01}= 0.08, \bar {r}_{02} = 0.15$ and $\bar {r}_{12}= 0.10$, and n₀₁ = n₀₂ = n₁₂ = 25. The initial estimate $\hat {\delta }_{1|1} =\bar {r}_{01}= 0.08$. The estimate of δ₁ after transactions in period 2 are available, is a weighted average of $\bar {r}_{01}= 0.08$ and $\bar {r}_{02} - \bar {r}_{12}= 0.05$, with weight w = 2/3, so $\hat {\delta }_{1|2} = 0.07$, and the revision is − 0.01.

A.2 Revisions in a Random Walk Repeat-Sales Model

Revisions in structural time series repeat-sales models are different from the standard (Bailey et al. 1963) model described in A.1. In this appendix we provide an algebraic derivation of how indexes are revised over time for a random walk repeat-sales model. For more advanced models the revision expressions become very complicated.

The random walk repeat-sales model for 3 observations $\bar {r}_{01}, \bar {r}_{02}$ and $\bar {r}_{12}$ can be expressed as

$$\left( \begin{array}{c} \bar{r}_{01} \\ \bar{r}_{02} \\ \bar{r}_{12} \\ 0 \\ 0 \end{array}\right) = \left( \begin{array}{cc} 1 & 0 \\ 1 & 1 \\ 0 & 1 \\ 1 & 0 \\ 0 & 1 \end{array}\right) \left( \begin{array}{c} \delta_{1} \\ \delta_{2} \end{array}\right) + \left( \begin{array}{c} \bar{\epsilon}_{01} \\ \bar{\epsilon}_{02} \\ \bar{\epsilon}_{12} \\ \eta_{1} \\ \eta_{2} \end{array}\right), $$

where δ_t = Δμ_t, Var$(\bar {\epsilon }_{st}) = 2\sigma ^{2}_{\epsilon } / n_{st}$, and Var$(\eta _{t}) = \sigma ^{2}_{\eta }$. Rows 1 – 3 are the observation equations, and rows 4 and 5 represent the random walk specification. Define the signal-to-noise ratio q as $\sigma ^{2}_{\eta } / (2\sigma ^{2}_{\epsilon })$. If only the transactions up to time 1 are available, the model consists only of rows 1 and 4. Note that for the standard Bailey et al. repeat-sales model only rows 1 – 3 are applicable.

The model can be estimated by weighted least squares, giving

$$\begin{array}{@{}rcl@{}} \hat{\delta}_{1|1} &=& \frac{n_{01}}{n_{01}+ 1/q} \bar{r}_{01}, \text{ based on transactions up to time 1}\\ \hat{\delta}_{1|2} &=& \frac{(n_{02}+n_{12})n_{01} \bar{r}_{01} + n_{02} n_{12} (\bar{r}_{02} - \bar{r}_{12}) + 1/q(n_{01} \bar{r}_{01} + n_{02} \bar{r}_{02})} {(n_{01} + 1/q)(n_{02}+n_{12} + 1/q) + (n_{12}+ 1/q)n_{02}}, \end{array} $$

the last equation based on on transactions up to time 2. Note that if q →∞, the expressions are identical to the ones in the previous section.

If we assume that the signal-to-noise ratio is equal to q = 0.1, and using the same numbers as in the previous example, then $\hat {\delta }_{1|1} = 0.057$ (instead of 0.08), and $\hat {\delta }_{1|2} = 0.063$ (instead of 0.07). The revision is equal to 0.006, and is smaller in absolute magnitude than the revision in the standard repeat sales model (0.006 versus 0.01).

A.3 Calculating Revisions for Different Signal-to-Noise Ratios

In this Section we calculate revisions for different values of the signal-to-noise ratios q in the random walk repeat-sales model as described in A.2. Our purpose here is simply to demonstrate the large role that q plays in revisions. We use typical values for the signal ($\sigma ^{2}_{\eta }$) and noise ($\sigma ^{2}_{\epsilon }$).

We use the LAI repeat-sales data, see “Data and Descriptive Statistics” to compute the indexes starting from 34 quarters back. We have 338 transaction observation pairs in the final dataset, or about 4.5 per quarter on average. With each new quarter of history, approximately 10 new pairs are added, including second-sales in the new quarter as well as additional pairs earlier in the history. In the calculations we use signal-to-noise ratios q in the range between 0.02 and than 0.6. This range covers the ratios that we do indeed estimate.

Figure 5 provides revision statistics and the volatility and the autocorrelation of the index returns for different values of the signal-to-noise ratio q. In order to conserve space we only look at the absolute mean revisions in levels and returns, for both the total sample and specific for the final period.

Fig. 5

Revisions in LAI for different simulated values of the signal-to-noise ratio q

It can be seen that lower values of q give less revisions. Note that in an extreme case with q = 0 there would be no revisions; the indexes would be flat. In this case there would also be a first-order autocorrelation of 1, and no volatility. The increase in levels revisions is not linear with an increase in q. For example, with q = 0.1 the average (absolute) revision is 7 bps, whereas with q = 0.5, the average (absolute) revision is ‘just’ 10 bps. However, linearity does seem to hold in the average (absolute) return revisions (Fig. 5d). More interesting is that the final period return revision behaves like a concave function of the assumed signal-to-noise ratio (Fig. 5e). Our results show that at q = 0.16 and q = 0.60 the final period index level revisions are equally large (equal cumulative historical change). For the returns revisions the concavity seems to be mostly absent.

Appendix B: Technical Estimation Details

This appendix contains some technical details on the model estimation. We apply the No-U-Turn-Sampler (NUTS) developed by Hoffman and Gelman (2014). It is a generalization of the Hamiltonian Monte Carlo (HMC) algorithm. The HMC avoids the random walk behaviour and sensitivity to correlated parameters that plague other Markov chain Monte Carlo methods by taking a series of steps informed by first-order gradient information. These features allow it to converge to high-dimensional target distributions much more quickly compared to for example the Metropolis-Hastings algorithm and Gibbs sampling. In addition, the NUTS algorithm avoids setting the parameters for the step size and the desired numbers of steps, which the HMC is so sensitive to. In case of many parameters, the manual setting of step sizes and desired number of steps, becomes intractable. The NUTS algorithm has been introduced in R via the software package called ‘Rstan’ (Carpenter et al. 2017).

The difference in convergence between NUTS and Gibbs sampling is clearly illustrated by the the more elaborate ARAG model. In our applications Gibbs sampling, by using the program JAGS (Plummer 2003), could take hours to converge, while NUTS only needed 10 – 20 seconds.

We apply the ‘Matt trick’ to sample μ_t by specifying μ_t = μ_t− 1 + η_t, where the increments η_t ∼Normal(0, 1) are independent of each other. The effective sample size is increased considerably by doing so, simply because there is less correlation in the index returns compared to index levels.¹⁰ Intuitively, without the ‘Matt trick’, if an ‘extreme’ value is sampled for price level in period t, this will affect the price level sampled in period t + 1, t + 2, etc. By sampling increments, this relationship is gone. A more technical description is given by Betancourt and Girolami (2015).

We sample 2, 000 times, of which we discard the first 1, 000 when analysing the chains (i.e. the warm-up stage), in parallel over 6 chains (so 6 × 1, 000 = 6, 000 samples in total). We provide different initial values for each variable in each chain and we do not thin the chains (see Link and Eaton 2012, for our reasons to shy away from thinning). We subsequently evaluate whether or not the model converges based on $\bar {R}$, the potential scale reduction statistic.¹¹ We use very strict convergence criteria, i.e. average $\bar {R} \leq 1.01$, which is more strict than usual. The fact that the models for our granular indices converge even with such strict criteria, shows the power of the NUTS algorithm. As shown in “Results” and “Results”, the effective and total sample size are almost identical for most models. We also keep track of the Monte Carlo (MC) error (Koehler et al. 2009), but these are not presented in this paper to conserve space. Even with a sample size of only 6,000, the MC error divided by the mean or standard deviation of the posterior, remains very low.

Appendix C: Parameter Estimates and Credible Intervals

Table 9 Summary of Posterior Distributions for LAI

Table 10 Summary of Posterior Distributions for SFO

Appendix D: Revisions for a Selection of Indexes

Fig. 6

Revisions for a selection of the indexes in LAI and SFO

Fig. 7

Cumulative revisions of index levels at different periods

Fig. 8

Cumulative revisions of index returns at different periods

Appendix E: Revision Results of our Robustness Section

Table 11 Revisions in our ‘robustness’ markets