A hierarchical approach to scalable Gaussian process regression for spatial data

Abstract

Large scale and highly detailed geospatial datasets currently offer rich opportunities for empirical investigation, where finer-level investigation of spatial spillovers and spatial infill can now be done at the parcel level. Gaussian process regression (GPR) is particularly well suited for such investigations, but is currently limited by its need to manipulate and store large dense covariance matrices. The central purpose of this paper is to develop a more efficient version of GPR based on the hierarchical covariance approximation proposed by Chen et al. (J Mach Learn Res 18:1–42, 2017) and Chen and Stein (Linear-cost covariance functions for Gaussian random fields, arXiv:1711.05895, 2017). We provide a novel probabilistic interpretation of Chen’s framework, and extend his method to the analysis of local marginal effects at the parcel level. Finally, we apply these tools to a spatial dataset constructed from a 10-year period of Oklahoma County Assessor databases. In this setting, we are able to identify both regions of possible spatial spillovers and spatial infill, and to show more generally how this approach can be used for the systematic identification of specific development opportunities.

Appendix

To show that expressions (34) and (37) are indeed the actual covariances of the random vectors in expression (33) of the text, it is convenient to introduce further simplifying notation. For each possible root path, $i_{1} \, \to \,i_{2} \, \to \, \cdots \, \to \,i_{m - 1} \, \to \,i_{m} \to \,r$, let $H_{{i_{1} \,i_{2} \, \cdots \,i_{m} \,r}}$ be defined recursively for paths of length one by

$$H_{{i_{1} \,r}} \, = \,Z_{{i_{1} \,|\,r}} \, + \,A_{{i_{1} \,r}} \,Z_{r}$$

(A.1)

[as in (18) of the text] and for longer paths by

$$H_{{i_{1} \,i_{2} \, \cdots \,i_{m} \,r}} \, = \,Z_{{i_{1} \,|\,i_{2} }} + \,A_{{i_{1} \,i_{2} \,}} H_{{i_{2} \cdots \,i_{m} \,r\,}}$$

(A.2)

Then, (A.1) together with the argument in (14) through (17) of the text again shows that for paths of length one,

$${\text{cov}} \left( {H_{{i_{1} \,r}} } \right)\, = \,K_{{i_{1} \,i_{1} }}$$

(A.3)

So if it hypothesized that

$${\text{cov}} (H_{{i_{1} \, \cdots \,i_{m} \,r}} ) = K_{{i_{1} \,i_{1} }}$$

(A.4)

holds for all paths of length m, then for paths of length $m + 1$ it follows from the independence of $Y_{{i_{1} \,|\,i_{2} }}$ and $H_{{i_{2} \cdots \,i_{m + 1} \,r\,}}$, together with (A.4) that [again from the argument in (14) through (17) in the text],

$$\begin{aligned} {\text{cov}} \left( {H_{{i_{1} \,i_{2} \, \cdots \,i_{m} \,i_{m + 1} \,r}} } \right) & = {\text{cov}} \left( {Y_{{i_{1} \,|\,i_{2} }} + A_{{i_{1} \,i_{2} \,}} H_{{i_{2} \cdots \,i_{m + 1} \,r\,}} } \right) \\ & = {\text{cov}} \left( {Y_{{i_{1} \,|\,i_{2} }} } \right) + A_{{i_{1} \,i_{2} }} {\text{cov}} \left( {H_{{i_{2} \cdots \,i_{m + 1} \,r}} } \right)A_{{i_{2} \,i_{1} }} \\ & = K_{{i_{1} \,i_{1} }} - K_{{i_{1} \,i_{2} }} K_{{i_{2} \,i_{2} }}^{ - 1} K_{{i_{2} \,i_{1} }} + \left( {K_{{i_{1} \,i_{2} }} K_{{i_{2} \,i_{2} }}^{ - 1} } \right)\left[ {K_{{i_{2} \,i_{2} }} } \right]\left( {K_{{i_{2} \,i_{2} }}^{ - 1} K_{{i_{2} \,i_{1} }} } \right) \\ & = K_{{i_{1} \,i_{1} }} - K_{{i_{1} \,i_{2} }} K_{{i_{2} \,i_{2} }}^{ - 1} K_{{i_{2} \,i_{1} }} + K_{{i_{1} \,i_{2} }} K_{{i_{2} \,i_{2} }}^{ - 1} K_{{i_{2} \,i_{1} }} = K_{{i_{1} \,i_{1} }} \\ \end{aligned}$$

(A.5)

So by induction, (A.4) must hold for all m. But for any leaf, $i$, with root path, $i\, \to i_{1} \, \to \cdots \, \to i_{m} \to \,r$, this implies at once that

$${\text{cov}} \left( {H_{i} } \right) = {\text{cov}} \left( {H_{{i\,\,i_{1} \cdots i_{m} \,r}} } \right) = K_{i\,i}$$

(A.6)

and thus that expression (39) in the text must hold.

It remains to establish expression (37) in the text for any distinct leaves, $i$ and $j$ with root paths as in (35) and (36) (where again this taken to include the case, $s = r$). To do so, we first expand $H_{i} = H_{{i\,\,i_{1} \cdots i_{p} \,s\,h_{\,1} \, \cdots \,h_{m} \,r}}$ and $H_{j} = H_{{j\,\,j_{1} \cdots j_{q} \,s\,h_{\,1} \, \cdots \,h_{m} \,r}}$ as follows:

$$H_{i} = Y_{{i\,|\,i_{1} }} + A_{{i\,i_{1} }} Y_{{i_{1} \,|\,i_{2} }} + \left( {A_{{i\,i_{1} }} A_{{i_{1} \,i_{2} }} } \right)Y_{{i_{2} \,|\,i_{3} }} + \cdots + \left( {A_{{i\,\,i_{1} }} A_{{i_{1} \,i_{2} }} \cdots A_{{i_{p - 1\,p} }} } \right)Y_{{i_{p} \,s}} + \left( {A_{{i\,i_{1} }} A_{{i_{1} \,i_{2} }} \cdots A_{{i_{p} \,s}} } \right)H_{{s\,h_{\,1} \cdots h_{m} \,r}}$$

(A.7)

$$H_{j} = Y_{{j\,|\,j_{1} }} + A_{{j\,j_{1} }} Y_{{j_{1} \,|\,j_{2} }} + \left( {A_{{j\,j_{1} }} A_{{j_{1} \,j_{2} }} } \right)Y_{{j_{2} \,|\,j_{3} }} + \cdots + \left( {A_{{j\,j_{1} }} A_{{j_{1} \,j_{2} }} \cdots A_{{j_{q - 1\,q} }} } \right)Y_{{j_{q} \,s}} + \left( {A_{{j\,j_{1} }} A_{{j_{1} \,j_{2} }} \cdots A_{{j_{q} \,s}} } \right)H_{{s\,h_{\,1} \cdots h_{m} \,r}}$$

(A.8)

Next recall that since the random variables $(Y_{{i\,|\,i_{1} }} ,Y_{{i_{1} \,|\,i_{2} }} ,\, \ldots ,\,Y_{{i_{p} \,|\,s}} ,\,Y_{{j\,|\,j_{1} }} ,Y_{{j_{1} \,|\,j_{2} }} ,\, \ldots ,\,Y_{{j_{q} \,|\,s}} ,H_{{s\,h_{\,1} \cdots h_{m} \,r}} )$ are all independent, it follows [as for example in (31) of the text] that all covariance terms between $H_{i}$ and $H_{j}$ are zero except for the shared term involving $H_{{s\,h_{\,1} \cdots h_{m} \,r}}$, so that,

$$\begin{aligned} {\text{cov}} \left( {H_{i} ,H_{j} } \right) & = {\text{cov}} [(A_{{i\,i_{1} }} A_{{i_{1} \,i_{2} }} \cdots A_{{i_{p} \,s}} )\,H_{{s\,h_{\,1} \cdots h_{m} \,r}} \,,\,\,(A_{{j\,j_{1} }} A_{{j_{1} \,j_{2} }} \cdots A_{{j_{q} \,s}} )\,H_{{s\,h_{\,1} \cdots h_{m} \,r}} ] \\ & = \left( {A_{{i\,i_{1} }} A_{{i_{1} \,i_{2} }} \cdots A_{{i_{p} \,s}} } \right){\text{cov}} \left( {H_{{s\,h_{\,1} \cdots h_{m} \,r}} } \right)\left( {A_{{s\,j_{q} }} \cdots A_{{j_{2} \,j_{1} }} \cdots A_{{j_{1} \,j}} } \right) \\ \end{aligned}$$

(A.9)

But this implies at once from (A.5) that

$${\text{cov}} \left( {H_{i} ,H_{j} } \right) = A_{{i\,i_{1} }} A_{{i_{1} \,i_{2} }} \cdots A_{{i_{p} \,s}} \left( {K_{ss} } \right)A_{{s\,j_{q} }} \cdots A_{{j_{2} \,j_{1} }} \cdots A_{{j_{1} \,j}}$$

(A.10)

and thus that expression (37) must hold.