Automatic estimation of spatial spectra via smoothing splines

Abstract

Spectra are frequently used to depict the dependence features of a second-order stationary process. In this paper, the spatial log-spectral density is expressed by a new type of smoothing splines in the form of the summation of a linear expression of univariate bases and two quadratic forms of univariate bases. Based on this new type of smoothing splines, a Bayesian nonparametric method is proposed to estimate the spectral density of spatial data observed on a lattice. The proposed Bayesian approach uses a Hamiltonian Monte Carlo-within-Gibbs technique to fit smoothing splines to the spatial periodogram. Our technique produces an automatically smoothed spatial spectral estimate along with samples from the posterior distributions of the parameters to facilitate inference.

Appendices

Appendix A. The Hamiltonian Monte Carlo step in Sect. 3

In what follows, the current and proposed values of $\varvec{\beta }$, $\mathbf {B}_1$ and $\mathbf {B}_2$ are denoted by the superscripts c and p, respectively. Suppose the chains for $\varvec{\beta }$, $\mathbf {B}_1$ and $\mathbf {B}_2$ are currently at $\varvec{\beta }^c$, $\mathbf {B}_1^c$ and $\mathbf {B}_2^c$, respectively. As an illustration, we only state the details of the steps for sampling $\varvec{\beta }^p$ and $\mathbf {B}_1^p$. Given $\mathbf {B}_1^c$, $\mathbf {B}_2^c$, $\lambda _0$ and $I_{n}(\varvec{\omega })$, $\varvec{\omega }\in \Omega _n$, the Hamiltonian Monte Carlo (HMC) is designed to sample $\varvec{\beta }^p$ jointly from $p\big (\varvec{\beta } \big |\lambda _0,\mathbf {B}_1^c,\mathbf {B}_2^c, I_{n}(\varvec{\omega }), \varvec{\omega }\in \Omega _n\big )$. Given $\varvec{\beta }^c$, $\mathbf {B}_2^c$, $\lambda _1$ and $I_{n}(\varvec{\omega })$, $\varvec{\omega }\in \Omega _n$, the Hamiltonian Monte Carlo (HMC) is designed to sample $\mathbf {B}_1^p$ jointly from $p\big (\mathbf {B}_1 \big |\lambda _1,\varvec{\beta }^c,\mathbf {B}_2^c, I_{n}(\varvec{\omega }), \varvec{\omega }\in \Omega _n\big )$.

1.1 Appendix A.1. The gradients

For notational brevity, we omit the arguments of p in the following. By taking the derivative of $\log p$ with respect to $\varvec{\beta }$, we obtain

$$\begin{aligned} \frac{\partial \log p}{\partial \varvec{\beta }}=&-\frac{1}{2}\sum _{(\omega _1,\omega _2) \in \Omega _n} \Big \{\Big [1-I_{n}(\omega _1,\omega _2) \exp \big (-F(\omega _1,\omega _2)\big )\Big ] \nonumber \\&\quad \Big (\varvec{\varsigma } \odot \varvec{\varphi }_{J_1,J_2}(\omega _1,\omega _2)\Big )\Big \}-\Sigma ^{-1} \varvec{\beta }. \end{aligned}$$

(A.1)

By taking the derivative of $\log p$ with respect to the matrices $\mathbf {B}_1$ and $\mathbf {B}_2$ (cf. Horn and Johnson 2013), respectively, we obtain

$$\begin{aligned} \frac{\partial \log p}{\partial \mathbf {B}_1}=&-\frac{1}{2}\sum _{(\omega _1,\omega _2) \in \Omega _n} \Big \{\Big [1-I_{n}(\omega _1,\omega _2) \exp \big (-F(\omega _1,\omega _2)\big )\Big ] \nonumber \\&\quad \Big (\Lambda \odot \varvec{\psi }_{J_1}\big (\omega _1\big ) \varvec{\psi }_{J_2}\big (\omega _2\big )^T\Big )\Big \}-\lambda _1^{-1} \mathbf {B}_1, \end{aligned}$$

(A.2)

and

$$\begin{aligned} \frac{\partial \log p}{\partial \mathbf {B}_2}=&-\frac{1}{2}\sum _{(\omega _1,\omega _2) \in \Omega _n} \Big \{\Big [1-I_{n}(\omega _1,\omega _2) \exp \big (-F(\omega _1,\omega _2)\big )\Big ] \\&\quad \Big (\Lambda \odot \varvec{\phi }_{J_1}\big (\omega _1\big ) \varvec{\phi }_{J_2}\big (\omega _2\big )^T\Big )\Big \}-\lambda _2^{-1} \mathbf {B}_2. \end{aligned}$$

1.2 Appendix A.2. The HMC step for sampling $\varvec{\beta }$

Let $\mathbf{m}$ be the momentum vector with the same dimension as $\varvec{\beta }$. We give $\mathbf{m}$ a multivariate normal distribution, $N(\mathbf {0},\kappa ^2\, I_{J_1+J_2+1})$, where $\kappa >0$.

Recall that the current value of $\varvec{\beta }$ is $\varvec{\beta }^c$. The HMC begins by drawing $\mathbf{m}$ from $N(\mathbf {0},\kappa ^2\, I_{J_1+J_2+1})$, say $\mathbf{m} ^c$. Then it proceeds by updating $\mathbf{m}$ and $\varvec{\beta }$ simultaneously, with L ‘leapfrog steps’ scaled by a factor $\epsilon$. Each leapfrog step consists of three parts as follows.

(a) Use the gradient (A.1) to make a half-step of $\mathbf{m}$:

$$\begin{aligned} \mathbf{m} \longleftarrow \mathbf{m} +\frac{1}{2}\epsilon \frac{\partial \log p}{\partial \varvec{\beta }}. \end{aligned}$$

(b) Use the momentum matrix $\mathbf{m}$ to update $\varvec{\beta }$:

$$\begin{aligned} \varvec{\beta } \longleftarrow \varvec{\beta }+\epsilon \,\kappa ^{-2} \mathbf{m} . \end{aligned}$$

(c) Again use the gradient (A.1) to half-update $\mathbf{m}$:

$$\begin{aligned} \mathbf{m} \longleftarrow \mathbf{m} +\frac{1}{2}\epsilon \frac{\partial \log p}{\partial \varvec{\beta }}. \end{aligned}$$

We label $\mathbf{m} ^\prime$ and $\varvec{\beta }^\prime$ as the value of $\mathbf{m}$ and $\varvec{\beta }$ after the L leapfrog steps. Let

$$\begin{aligned} A=\frac{p\big (\varvec{\beta }^\prime |\lambda _0, \mathbf {B}_1^c,\mathbf {B}_2^c, I_{n}(\varvec{\omega }), \varvec{\omega }\in \Omega _n\big )p(M^\prime )}{p\big (\varvec{\beta }^c|\lambda _0, \mathbf {B}_1^c,\mathbf {B}_2^c, I_{n}(\varvec{\omega }), \varvec{\omega }\in \Omega _n\big )p(M^c)}, \end{aligned}$$

where the unconditional density $p(\cdot )$ is the probability density function of $N(\mathbf {0},\kappa ^2\, I_{J_1+J_2+1})$. The updating of $\varvec{\beta }$ is completed by setting $\varvec{\beta }^{p}=\varvec{\beta }^\prime$ with probability $\min (1,A)$, and $\varvec{\beta }^{p}=\varvec{\beta }^c$, otherwise.

1.3 Appendix A.3. The HMC step for sampling $\mathbf{B} _1$

Let M be the momentum matrix with the same dimension as $\mathbf {B}_1$. We give M a matrix-valued normal distribution, $\mathcal {MN}_{J_1,J_2}(\mathbf {O},\widetilde{U},\widetilde{V})$. To keep it simple, we assume $\widetilde{U}=\kappa I_{J_1}$ and $\widetilde{V}=\kappa I_{J_2}$, where $\kappa >0$.

Recall that the current value of $\mathbf {B}_1$ is $\mathbf {B}_1^c$. The HMC begins by drawing M from $\mathcal {MN}_{J_1,J_2}(\mathbf {O},\widetilde{U},\widetilde{V})$, say $M^c$. Then it proceeds by updating M and $\mathbf {B}_1$ simultaneously, with L ‘leapfrog steps’ scaled by a factor $\epsilon$. Each leapfrog step consists of three parts as follows.

(a) Use the gradient (A.2) to make a half-step of M:

$$\begin{aligned} M\longleftarrow M+\frac{1}{2}\epsilon \frac{\partial \log p}{\partial \mathbf {B}_1}. \end{aligned}$$

(b) Use the momentum matrix M to update $\mathbf {B}_1$:

$$\begin{aligned} \mathbf {B}_1 \longleftarrow \mathbf {B}_1+\epsilon \,\widetilde{U}^{-1} M (\widetilde{V}^{-1})^T. \end{aligned}$$

In our settings, it holds that $\widetilde{U}^{-1} M (\widetilde{V}^{-1})^T=\kappa ^{-2} M$.

(c) Again use the gradient (A.2) to half-update M:

$$\begin{aligned} M\longleftarrow M+\frac{1}{2}\epsilon \frac{\partial \log p}{\partial \mathbf {B}_1}. \end{aligned}$$

We label $M^\prime$ and $\mathbf {B}_1^\prime$ as the value of M and $\mathbf {B}_1$ after the L leapfrog steps. Let

$$\begin{aligned} A=\frac{p\big (\mathbf {B}_1^\prime \big |\lambda _1,\varvec{\beta }^c,\mathbf {B}_2^c, I_{n}(\varvec{\omega }), \varvec{\omega }\in \Omega _n\big )p(M^\prime )}{p\big (\mathbf {B}_1^c \big |\lambda _1,\varvec{\beta }^c,\mathbf {B}_2^c, I_{n}(\varvec{\omega }), \varvec{\omega }\in \Omega _n\big )p(M^c)}, \end{aligned}$$

where the unconditional density $p(\cdot )$ is the pdf of $\mathcal {MN}_{J_1,J_2}(\mathbf {O},\widetilde{U},\widetilde{V})$. The updating of $\mathbf {B}_1$ is completed by setting $\mathbf {B}_1^{p}=\mathbf {B}_1^\prime$ with probability $\min (1,A)$, and $\mathbf {B}_1^{p}=\mathbf {B}_1^c$, otherwise.

1.4 Appendix A.4. Choice of tuning parameters

As suggested by Gelman et al. (2014), we set the product $\epsilon \, L$ to 1. Then the HMC requires only two tuning parameters: the scale parameter $\kappa$ and the number of leapfrog steps L.

Actually, it produces no clear difference for the updating ratios of $\mathbf {B}=\{\varvec{\beta },\mathbf {B}_1,\mathbf {B}_2\}$ in the iterative steps if ranging L from 5 to 20 in our simulation and data analysis. In this article, we set L to 10, which can ensure that the updating ratios of $\mathbf {B}$ in the iterative steps are 90% in all our simulation examples, so that we recommend 10 to be the default setting of L in applications.

The rest of this appendix pertains to setting the scale parameter $\kappa$. For fixed $L=10$, the proposed approach produces indistinguishable results if ranging $\kappa$ from 5 to 50 in our simulation and data analysis. However, we find that for the autocorrelation of Markov Chain Monte Carlo (MCMC) sample of each posterior parameter decays more rapidly with smaller 5. Therefore, we set $\kappa$ to 5 in all the examples and applications in this article.

Appendix B. Supplementary material

Supplementary materials related to this article, including a supplementary file and some R programs, are available at https://doi.org/10.1007/s00180-021-01141-z.