Strict positive definiteness in geostatistics

Abstract

Geostatistical modeling is often based on the use of covariance functions, i.e., positive definite functions. However, when interpolation problems have to be solved, it is advisable to consider the subset of strictly positive definite functions. Indeed, it will be argued that ensuring strict positive definiteness for a covariance function is convenient from a theoretical and practical point of view. In this paper, an extensive analysis on strictly positive definite covariance functions has been given. The closure of the set of strictly positive definite functions with respect to the sum and the product of covariance functions defined on the same Euclidean dimensional space or on factor spaces, as well as on partially overlapped lower dimensional spaces, has been analyzed. These results are particularly useful (a) to extend strict positive definiteness in higher dimensional spaces starting from covariance functions which are only defined on lower dimensional spaces and/or are only strictly positive definite in lower dimensional spaces, (b) to construct strictly positive definite covariance functions in space–time as well as (c) to obtain new asymmetric and strictly positive definite covariance functions.

Appendices

Appendix

Proof. Theorem 2. point 1

Given $n=2$, let $C_1$ and $C_2$ be covariance functions, both defined on ${\mathbb {R}}^m$ and

$$\begin{aligned} C({\mathbf{h }})=C_1({\mathbf{h }})C_2({\mathbf{h }}), \quad {\mathbf{h }} \in {\mathbb {R}}^m, \end{aligned}$$

(48)

a covariance function defined on ${\mathbb {R}}^m$. Assume that $C_1$ is a SPD covariance function and $C_2$ is not identically zero (i.e. $C_2(\mathbf{0})>0$) positive definite covariance function. For any $k\in N_+$ and any choice of distinct points $\mathbf{s}_i \in {\mathbb {R}}^{m}, i=1,\ldots ,k$ and ${\varvec{\lambda }}=[\lambda _1,\ldots ,\lambda _k]^T\in {\mathbb {R}}^k$, not zero, the quadratic form can be written as follows ${\varvec{\lambda }}{} {\mathbf{C}} {\varvec{\lambda }}^T$, where the generic element of ${\mathbf{C}}$ is

$$\begin{aligned} C({\mathbf{s }}_i-{\mathbf{s }}_j)=C_1({\mathbf{s }}_i-{\mathbf{s }}_j)C_2({\mathbf{s }}_i-{\mathbf{s }}_j). \end{aligned}$$

(49)

The matrix ${\mathbf{C}}$ is the Hadamard product between the matrices ${\mathbf{C}} _1$, whose generic element ${\mathbf{C}} _1[ij]$ is $C_1({\mathbf{s }}_i-{\mathbf{s }}_j)$, and ${\mathbf{C}} _2$, whose generic element ${\mathbf{C}} _2[ij]$ is $C_2({\mathbf{s }}_i-{\mathbf{s }}_j)$. Since ${\mathbf{C}} _1$ is SPD and ${\mathbf{C}} _2$ has no diagonal entry equal to 0, then by recalling Horn and Johnson (1991), C defined in (49) is SPD. Thus model in (48) is SPD. For $n=3$, assume that $C_1$ is a SPD covariance function and $C_2$, $C_3$ are not identically zero positive definite covariance functions. Since $C_2C_3$ is still a not identically zero positive definite covariance function then the following covariance function,

$$\begin{aligned} C({\mathbf{h }})=C_1({\mathbf{h }})C_2( {\mathbf{h }})C_3({\mathbf{h }}), \end{aligned}$$

(50)

is SPD. The proof can be completed by induction for any n.

Proof. Theorem 2. point 2

Part 1 (only if)

The proof of this implication (from the right to the left) follows from the quadratic form in (2).

Part 2 (if)

The proof of this implication (from the left to the right) follows from the Bochner characterization as clarified hereafter. However, before proving the theorem, it is worth introducing the following Lemma.

Lemma 2

C is SPD if and only if any trigonometric polynomial $P(\omega )=\displaystyle \sum _{j=1}^{n}\lambda _{j}\exp ({i{\mathbf{s }}_{j}^{T}\omega })$, such that

$$\begin{aligned} \int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF(\omega )=0, \end{aligned}$$

is identically zero, that is $P(\omega )=0\quad \forall \omega \in {\mathbb {R}}^m.$

Proof of Lemma 2

Lemma-Part 1 (if)

Let C be a SPD covariance function and

$$\begin{aligned} P(\omega )=\displaystyle \sum _{j=1}^{n}\lambda _{j}\exp \left( {i{\mathbf{s }}_{j}^{T}\omega }\right) . \end{aligned}$$

Let ${\mathbf{s }}_{i}, \; i=1,2,\ldots , n$ be distinct points in ${\mathbb {R}}^m.$

Then

$$\begin{aligned} 0= & {} \int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF(\omega )=\int _{{\mathbb {R}}^m}\sum _{i=1}^{n}\sum _{j=1}^{n}\lambda _i\lambda _j\,\exp \left( {i({\mathbf{s }}_i-{\mathbf{s }}_j)^{T}\omega }\right) dF(\omega )\\= & {} \displaystyle \sum _{i=1}^{n}\sum _{j=1}^{n}\lambda _i\lambda _j\, C({\mathbf{s }}_i-{\mathbf{s }}_j)\; \Rightarrow \; \lambda _{i}=0,\; \forall i=1,\ldots ,n \hbox { and } P\equiv 0 \hbox { on } {\mathbb {R}}^m. \end{aligned}$$

Lemma-Part 2 (only if)

Let ${\mathbf{s }}_{1}, {\mathbf{s }}_{2},\ldots , {\mathbf{s }}_{n}$ be distinct points on ${\mathbb {R}}^m$ and

$$\begin{aligned} (\lambda _1, \ldots , \lambda _n)\ne (0, \dots , 0). \end{aligned}$$

The trigonometric polynomial

$$\begin{aligned} P(\omega )=\displaystyle \sum _{j=1}^{n}\lambda _{j} \exp \left( {i{\mathbf{s }}_{j}^{T}\omega }\right) \end{aligned}$$

is not identically zero on ${\mathbb {R}}^m$, because the exponential functions $\exp ({i{\mathbf{s }}_{j}^{T}\omega }), j=1,\ldots ,n,$ are linearly independent on ${\mathbb {R}}^m$.

Then, by hypothesis, $\displaystyle 0<\int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF(\omega )=\sum _{i=1}^{n}\sum _{j=1}^{n}\lambda _i\lambda _j\, C({\mathbf{s }}_i-{\mathbf{s }}_j).$ $\square$

Thus, the proof of the second part of Theorem 2 follows. Given $n=2$, by contradiction let

$$\begin{aligned} C(\mathbf{h}) = \int _{{\mathbb {R}}^m} \exp \left( {i\mathbf{h}^T \omega }\right) dF(\omega ), \quad C_k(\mathbf{h}) = \int _{{\mathbb {R}}^m} \exp \left( {i\mathbf{h}^T \omega }\right) dF_k(\omega ), \quad k=1,2, \end{aligned}$$

with $C(\mathbf{h})=C_1(\mathbf{h})+C_2(\mathbf{h}),$ and $F(\omega )=F_1(\omega )+F_2(\omega ),$ where C is a SPD covariance function, while $C_1$ and $C_2$ are only positive definite covariance functions. Because of Lemma 2, there exist the trigonometric polynomials $P_1$ and $P_2$ on ${\mathbb {R}}^m$

$$\begin{aligned} P_k(\omega )=\displaystyle \sum _{j=1}^{n_k}\lambda _{j}^{(k)} \exp \left( {i{\mathbf{s }}_{j}^{T}\omega }\right) \quad k=1,2 \end{aligned}$$

both not identically zero such that:

$$\begin{aligned} \int _{{\mathbb {R}}^m}|P_k(\omega )|^2\, dF_k(\omega )=0,\quad k=1,2. \end{aligned}$$

Let $P(\omega )=P_1(\omega )\cdot P_2(\omega )$ and P be a trigonometric polynomial not identically zero. Moreover,

$$\begin{aligned}&\int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF_k(\omega )=\int _{{\mathbb {R}}^m}|P_1(\omega )|^2\cdot |P_2(\omega )|^2 dF_k(\omega )\\&\quad \le \big (\displaystyle \sum _{j=1}^{n_k}|\lambda _j^{(k)}|^2\big )\int _{{\mathbb {R}}^m}|P_k(\omega )|^2\, dF_k(\omega )=0, \quad k=1,2. \end{aligned}$$

Hence,

$$\begin{aligned} \int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF_1(\omega )=\int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF_2(\omega )=0; \end{aligned}$$

then by taking the sum:

$$\begin{aligned} \int _{{\mathbb {R}}^m}|P(\omega )|^2\, dF(\omega )=0, \end{aligned}$$

thus, because of Lemma 2, C should not be a SPD covariance function.

For $n=3$, let us assume by contradiction that

$$\begin{aligned} C(\mathbf{h}) = \int _{{\mathbb {R}}^m} \exp ({i\mathbf{h}^T \omega }) dF(\omega ), \quad C_k(\mathbf{h}) = \int _{{\mathbb {R}}^m} \exp ({i\mathbf{h}^T \omega }) dF_k(\omega ), \quad k=1,2,3, \end{aligned}$$

with $C(\mathbf{h})=C_1(\mathbf{h})+C_2(\mathbf{h})+C_3(\mathbf{h}),$ and $F(\omega )=F_1(\omega )+F_2(\omega )+F_3(\omega ),$ where C is a SPD covariance function, while $C_k$, $k=1,2,3,$ are only positive definite covariance functions. Then, since $C_1(\mathbf{h})+C_2(\mathbf{h})$ is still positive definite, thus, again because of Lemma 2, C should not be a SPD covariance function. The proof can be completed by induction for any n.

Proof. Theorem 3

Part 1 (only if)

If the covariance function C in (19) is SPD, then for any $n\in N_+$ and any choice of $\mathbf{u}_1=({\mathbf{x}} _{_1},{\mathbf{y}} _{_1},{\mathbf{z}} _{_1}), \ldots , \mathbf{u}_n=({\mathbf{x}} _{_n},{\mathbf{y}} _{_n},{\mathbf{z}} _{_n})\in {\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3}$ and $\lambda _1, \ldots , \lambda _n\in {\mathbb {R}}$ not all zero,

$$\begin{aligned} \sum _{i=1}^n\sum _{j=1}^n \lambda _i \lambda _j C_1({\mathbf{x }}_i-{\mathbf{x }}_j,{\mathbf{y }}_i-{\mathbf{y }}_j) C_2({\mathbf{y }}_i-{\mathbf{y }}_j,{\mathbf{z }}_i-{\mathbf{z }}_j)> 0. \end{aligned}$$

(51)

This implies that for any $n\in N_+$ and any choice of $\mathbf{u}_1=({\mathbf{x}} _{_1},{\mathbf{y}} _{_1},{\mathbf{z}} _{_1}), \ldots , \mathbf{u}_n=({\mathbf{x}} _{_n},{\mathbf{y}} _{_n},{\mathbf{z}} _{_n})\in {\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3}$, where ${\mathbf{y}} _{_1}=\cdots ={\mathbf{y}} _{_n}$ and ${\mathbf{z}} _{_1}=\cdots ={\mathbf{z}} _{_n}$, and $\lambda _1, \ldots , \lambda _n\in {\mathbb {R}}$ not all zero,

$$\begin{aligned} \sum _{i=1}^n\sum _{j=1}^n \lambda _i \lambda _j C_1({\mathbf{x }}_i-{\mathbf{x }}_j,{\mathbf{0}} ) C_2({\mathbf{0}} ,{\mathbf{0}} )= C_2({\mathbf{0}} ,{\mathbf{0}} )\sum _{i=1}^n\sum _{j=1}^n \lambda _i \lambda _j C_1({\mathbf{x }}_i-{\mathbf{x }}_j,{\mathbf{0}} )> 0. \end{aligned}$$

(52)

Hence

$$\begin{aligned} \sum _{i=1}^n\sum _{j=1}^n \lambda _i \lambda _j C_1({\mathbf{x }}_i-{\mathbf{x }}_j,{\mathbf{0}} )> 0, \end{aligned}$$

(53)

thus $C_1$ is SPD on ${\mathbb {R}}^{m_1}$.

Similarly, for any $n\in N_+$ and any choice of $\mathbf{u}_1=({\mathbf{x}} _{_1},{\mathbf{y}} _{_1},{\mathbf{z}} _{_1}), \ldots , \mathbf{u}_n=({\mathbf{x}} _{_n},{\mathbf{y}} _{_n},{\mathbf{z}} _{_n})\in {\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3},$ where ${\mathbf{x}} _{_1}=\cdots ={\mathbf{x}} _{_n}$, ${\mathbf{y}} _{_1}=\cdots ={\mathbf{y}} _{_n}$ , and $\lambda _1, \ldots , \lambda _n\in {\mathbb {R}}$ not all zero,

$$\begin{aligned} \sum _{i=1}^n\sum _{j=1}^n \lambda _i \lambda _j C_1({\mathbf{0}} ,{\mathbf{0}} )C_2({\mathbf{0}} ,{\mathbf{z }}_i-{\mathbf{z }}_j) > 0. \end{aligned}$$

(54)

Thus $C_2$ is SPD on ${\mathbb {R}}^{m_3}$.

At the end, for any $n\in N_+$ and any choice of $\mathbf{u}_1=({\mathbf{x}} _{_1},{\mathbf{y}} _{_1},{\mathbf{z}} _{_1}), \ldots , \mathbf{u}_n=({\mathbf{x}} _{_n},{\mathbf{y}} _{_n},{\mathbf{z}} _{_n})\in {\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3},$ where ${\mathbf{x}} _{_1}=\cdots ={\mathbf{x}} _{_n}$, ${\mathbf{z}} _{_1}=\cdots ={\mathbf{z}} _{_n}$ , and $\lambda _1, \ldots , \lambda _n\in {\mathbb {R}}$ not all zero,

$$\begin{aligned} \sum _{i=1}^n\sum _{j=1}^n \lambda _i \lambda _j C_1({\mathbf{0}} ,{\mathbf{y }}_i-{\mathbf{y }}_j)C_2({\mathbf{y }}_i-{\mathbf{y }}_j,{\mathbf{0}} ) > 0. \end{aligned}$$

(55)

Thus, for Theorem 2, at least $C_1$ or $C_2$ must be SPD on ${\mathbb {R}}^{m_2}$; consequently, since the covariance function C in (19) is SPD on ${\mathbb {R}}^m$, if $C_2$ is SPD on ${\mathbb {R}}^{m_2}$ and ${\mathbb {R}}^{m_3}$, then it is SPD on ${\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3}$, or alternatively if $C_1$ is SPD on ${\mathbb {R}}^{m_1}$ and ${\mathbb {R}}^{m_2}$, then it is SPD on ${\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}$.

Part 2. (if).

Let $\mathbf{u}_1=({\mathbf{x}} _{_1},{\mathbf{y}} _{_1},{\mathbf{z}} _{_1}), \ldots , \mathbf{u}_n=({\mathbf{x}} _{_n},{\mathbf{y}} _{_n},{\mathbf{z}} _{_n})$ be distinct points in ${\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3}$, let ${\mathbf{x}} _{_1}, \ldots ,{\mathbf{x}} _{n_1}$, ${\mathbf{y}} _{_1}, \ldots ,{\mathbf{y}} _{n_2}$ and ${\mathbf{z}} _{_1}, \ldots ,{\mathbf{z}} _{n_3}$ be the corresponding distinct coordinates in ${\mathbb {R}}^{m_1}$, ${\mathbb {R}}^{m_2}$ and ${\mathbb {R}}^{m_3}$, respectively. Then $\{\mathbf{u }_{1}, \ldots , \mathbf{u }_{n}\}$ is a subset of $\{({\mathbf{x}} _i,{\mathbf{y}} _k,{\mathbf{z}} _g)\in {\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3},i=1,\ldots ,n_1, k=1,\ldots , n_2, g=1,\ldots , n_3\}.$ The latter set is a $(n_1\times n_2\times n_3)$ regular pattern.

Let $\Sigma$ be the $N\times N$ matrix, where $N=n_1\times n_2\times n_3$, whose entries are $C({\mathbf{x }}_i-{\mathbf{x }}_{j},{\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_g-{\mathbf{z }}_{h})$, $i,j= 1,2, \ldots , n_1$, $k,l= 1,2, \ldots , n_2$, $g,h= 1,2, \ldots , n_3$. Note that

$$\begin{aligned} \Sigma =\left[ \begin{array}{ccc} {\mathbf{C}} _{11} &{} \ldots &{} {\mathbf{C}} _{1n_2} \\ {\mathbf{C}} _{21} &{} \ldots &{} {\mathbf{C}} _{2n_2} \\ \vdots &{} \ddots &{} \vdots \\ {\mathbf{C}} _{n_21} &{} \ldots &{} {\mathbf{C}} _{n_2n_2} \\ \end{array} \right] , \end{aligned}$$

(56)

where

$$\begin{aligned} {\mathbf{C}} _{kl}=\, & {} {\mathbf{C}} _1({\mathbf{y }}_k-{\mathbf{y }}_{l})\otimes \,{\mathbf{C}} _2( {\mathbf{y }}_k-{\mathbf{y }}_{l}) \end{aligned}$$

(57)

$$\begin{aligned} {\mathbf{C}} _1({\mathbf{y }}_k-{\mathbf{y }}_{l})= & {} \left[ \begin{array}{ccc} C_1({\mathbf{x }}_1-{\mathbf{x }}_{1},{\mathbf{y }}_k-{\mathbf{y }}_{l}) &{} \ldots &{} C_1({\mathbf{x }}_1-{\mathbf{x }}_{n_1},{\mathbf{y }}_k-{\mathbf{y }}_{l}) \\ C_1({\mathbf{x }}_2-{\mathbf{x }}_{1},{\mathbf{y }}_k-{\mathbf{y }}_{l}) &{} \ldots &{} C_1({\mathbf{x }}_2-{\mathbf{x }}_{n_1},{\mathbf{y }}_k-{\mathbf{y }}_{l}) \\ \vdots &{} \ddots &{} \vdots \\ C_1({\mathbf{x }}_{n_1}-{\mathbf{x }}_{1},{\mathbf{y }}_k-{\mathbf{y }}_{l}) &{} \ldots &{} C_1({\mathbf{x }}_{n_1}-{\mathbf{x }}_{n_1},{\mathbf{y }}_k-{\mathbf{y }}_{l}) \\ \end{array} \right] , \end{aligned}$$

(58)

$$\begin{aligned} {\mathbf{C}} _2({\mathbf{y }}_k-{\mathbf{y }}_{l})= & {} \left[ \begin{array}{ccc} C_2({\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_1-{\mathbf{z }}_{1}) &{} \ldots &{} C_2({\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_1-{\mathbf{z }}_{n_3}) \\ C_2({\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_2-{\mathbf{z }}_{1}) &{} \ldots &{} C_2({\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_2-{\mathbf{z }}_{n_3}) \\ \vdots &{} \ddots &{} \vdots \\ C_2({\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_{n_3}-{\mathbf{z }}_{1}) &{} \ldots &{} C_2({\mathbf{y }}_k-{\mathbf{y }}_{l},{\mathbf{z }}_{n_3}-{\mathbf{z }}_{n_3}) \\ \\ \end{array} \right] . \end{aligned}$$

(59)

Then the generic element ${\mathbf{C}} _{kl}$ of the block covariance matrix $\Sigma$ can be written as the Kronecker product (denoted as $\otimes$) of two matrices ${\mathbf{C}} _{1(n_1\times n_1)}({\mathbf{y }}_k-{\mathbf{y }}_{l})$ and ${\mathbf{C}} _{2(n_3\times n_3)}({\mathbf{y }}_k-{\mathbf{y }}_{l})$.

For $n_2=1$, the covariance function C is defined just as the product of two covariance functions defined on disjoint domains, then the covariance matrix $\Sigma _1$ (where $\Sigma _1$ denotes the covariance matrix for C, when $n_2=1$) can be written just as the Kronecker product of two matrices ${\mathbf{C}} _{1(n_1\times n_1)}({\mathbf{0}} )$ and ${\mathbf{C}} _{2(n_3\times n_3)}({\mathbf{0}} )$. In this case, the strict positive definiteness of the covariance function C, for any choice of ${\mathbf{x}} _{_1}, \ldots ,{\mathbf{x}} _{n_1}$ and ${\mathbf{z}} _{_1}, \ldots ,{\mathbf{z}} _{n_3}$, is ensured if $C_1$ and $C_2$ are two SPD covariance functions on ${\mathbb {R}}^{m_1}$ and ${\mathbb {R}}^{m_3}$, respectively. Recall that a symmetric matrix is SPD if and only if the leading principal minors are positive, then it can be shown that the leading principal minors of the $(n\times n)$ matrix of a product covariance function, corresponding to the subset ${\mathbf{u}} _{_1}, {\mathbf{u}} _{_2}, \ldots ,{\mathbf{u}} _{n}$ in ${\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_3}$, are also positive.

For $n_2=2$, the block covariance matrix $\Sigma _{2}$ is the following:

$$\begin{aligned} \Sigma _2=\left[ \begin{array}{cc} {\mathbf{C}} _{11} &{} {\mathbf{C}} _{12} \\ {\mathbf{C}} _{21} &{} {\mathbf{C}} _{22} \\ \end{array} \right] . \end{aligned}$$

(60)

This matrix is positive definite if and only if ${\mathbf{C}} _{11}$ is SPD and the Schur complement ${\mathbf{C}} _{22}-{\mathbf{C}} _{21}{} {\mathbf{C}} _{11}^{-1}{} {\mathbf{C}} _{12}$ of ${\mathbf{C}} _{11}$ is SPD, furthermore this condition is equivalent to having $\rho ({\mathbf{C}} _{21}{} {\mathbf{C}} _{11}^{-1}{} {\mathbf{C}} _{12}{} {\mathbf{C}} _{22}^{-1})<1$, where $\rho (\cdot )$ is the spectral radius of a matrix, i.e. the maximum eigenvalue in modulus. In this case, it is worth pointing out that

• ${\mathbf{C}} _{11}^{-1}={\mathbf{C}} _{22}^{-1}={\mathbf{C}} _1({\mathbf{0}} )^{-1}\otimes \,{\mathbf{C}} _2({\mathbf{0}} )^{-1}$

• ${\mathbf{C}} _{12}={\mathbf{C}} _{12}^T={\mathbf{C}} _1({\mathbf{y }}_1-{\mathbf{y }}_{2})\otimes \,{\mathbf{C}} _2({\mathbf{y }}_1-{\mathbf{y }}_{2}).$

Thus

$$\begin{aligned}&\rho \left( {\mathbf{C}} _{21}{} {\mathbf{C}} _{11}^{-1}{} {\mathbf{C}} _{12} {\mathbf{C}} _{11}^{-1}\right) \nonumber \\&\quad =\rho \left\{ \left[ {\mathbf{C}} _1({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _1({\mathbf{0}} )^{^{^{-1}}}\!\!{\mathbf{C}} _1({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _1({\mathbf{0}} )^{^{^{-1}}}\!\right] \otimes \left[ {\mathbf{C}} _2({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _2({\mathbf{0}} )^{^{^{-1}}}\!\!{\mathbf{C}} _2({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _2({\mathbf{0}} )^{^{^{-1}}}\!\right] \right\} . \end{aligned}$$

(61)

If $C_2$ is a SPD covariance function on ${\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3}$ and $C_1$ is only a positive definite covariance function on ${\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}$, then $\rho [{\mathbf{C}} _2({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _2({\mathbf{0}} )^{^{-1}}{} {\mathbf{C}} _2({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _2({\mathbf{0}} )^{^{-1}}]<1$ and $\rho [{\mathbf{C}} _1({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _1({\mathbf{0}} )^{^{-1}}{} {\mathbf{C}} _1({\mathbf{y }}_1-{\mathbf{y }}_{2}){\mathbf{C}} _1({\mathbf{0}} )^{^{-1}}]\le 1$.

Taking into account that

• the eigenvalues of the Kronecker product of two matrices is the product of the eigenvalues of the two matrices (Horn and Johnson 1991, p. 245), and

• the spectral radius of the Kronecker product of two matrices is, according to its definition (Horn and Johnson 1996, p. 35), the maximum eigenvalue in modulus of the Kronecker product of two matrices,

it follows that the above spectral radius is less than 1, if $C_2$ is a SPD covariance function on ${\mathbb {R}}^{m_2}\times {\mathbb {R}}^{m_3}$ or alternatively $C_1$ is a SPD covariance function on ${\mathbb {R}}^{m_1}\times {\mathbb {R}}^{m_2}$.

For $n_2=3$, the block covariance matrix $\Sigma _3$ can be partitioned as follows

$$\begin{aligned} \Sigma _3=\left[ \begin{array}{ccc} {\mathbf{C}} _{11} &{} {\mathbf{C}} _{12} &{} {\mathbf{C}} _{13}\\ {\mathbf{C}} _{21} &{} {\mathbf{C}} _{22} &{} {\mathbf{C}} _{23}\\ {\mathbf{C}} _{31} &{} {\mathbf{C}} _{32} &{} {\mathbf{C}} _{33}\\ \end{array} \right] =\left[ \begin{array}{c} {\mathbf{A }}\\ {\mathbf{B}} \\ \end{array} \right] , \end{aligned}$$

(62)

where

$$\begin{aligned} A=\left[ \begin{array}{ccc} {\mathbf{C}} _{11} &{} {\mathbf{C}} _{12} &{} {\mathbf{C}} _{13}\\ {\mathbf{C}} _{21} &{} {\mathbf{C}} _{22} &{} {\mathbf{C}} _{23}\\ \end{array} \right] \end{aligned}$$

(63)

and ${\mathbf{B}} =[{\mathbf{C}} _{31} \;\; {\mathbf{C}} _{32} \;\; {\mathbf{C}} _{33}]$. If $C_2$ is SPD, then the row spaces $\textit{C}({\mathbf{A }})$ and $\textit{C}({\mathbf{B}} )$ are essentially disjoint. Thus, $rank (\Sigma _3)=rank({\mathbf{A }})+rank({\mathbf{B}} )=n_1\times 2\times n_3+n_1\times n_3=n_1\times 3\times n_3$, that is the covariance matrix $\Sigma _3$ is a full rank matrix, i.e. it is SPD. Similarly if $C_1$ is SPD.

For any other additional point in ${\mathbb {R}}^{m_2}$, the proof can be completed by induction. Recall that a symmetric matrix is SPD if and only if the leading principal minors are positive, then it can be shown that the leading principal minors of the $(n\times n)$ matrix of a product covariance function, corresponding to the subset ${\mathbf{u}} _{_1}, {\mathbf{u}} _{_2}, \ldots ,{\mathbf{u}} _{n}$, are also positive. This implies that the covariance matrix C is SPD for any choice of points. $\square$

Proof. Theorem 4

If $C_1$ and $C_2$ are, for all $x\in V\subseteq U$, two SPD covariance functions, defined on ${\mathbb {R}}^{n_1}\times {\mathbb {R}}$ and ${\mathbb {R}}^{n_2}\times {\mathbb {R}}$, respectively, then

$$\begin{aligned} \sum _{i=1}^n\sum _{i'=1}^n \lambda _i \lambda _{i'} C_1({\mathbf{x }}_i-{\mathbf{x }}_{i'},t_i-t_{i'};x) C_2({\mathbf{y }}_i-{\mathbf{y }}_{i'},t_i-t_{i'};x)> 0, \end{aligned}$$

(64)

for any $n\in \mathbb {N}_+$ and any choice of distinct points $\mathbf{u}_1=({\mathbf{x}} _{_1},{\mathbf{y}} _{_1},t_{_1}), \ldots ,$ $\ldots ,\mathbf{u}_n=({\mathbf{x}} _{_n},{\mathbf{y}} _{_n},t_{_n})\in {\mathbb {R}}^{n_1}\times {\mathbb {R}}^{n_2}\times {\mathbb {R}}$ and $\lambda _1, \ldots , \lambda _n\in {\mathbb {R}}$ not all zero. Consequently,

$$\begin{aligned}&\sum _{i=1}^n\sum _{i'=1}^n \lambda _i \lambda _{i'} \int _V C_1({\mathbf{x }}_i-{\mathbf{x }}_{i'},t_i-t_{i'};x) C_2({\mathbf{y }}_i-{\mathbf{y }}_{i'},t_i-t_{i'};x) d\mu (x)\nonumber \\&\quad = \int _V\biggl [\sum _{i=1}^n\sum _{i'=1}^n \lambda _i \lambda _{i'} C_1({\mathbf{x }}_i-{\mathbf{x }}_{i'},t_i-t_{i'};x) C_2({\mathbf{y }}_i-{\mathbf{y }}_{i'},t_i-t_{i'};x)\biggr ] d\mu (x)>0. \end{aligned}$$

(65)

This implies that the covariance function C in (36) is SPD. $\square$

Proof. Theorem 5

If $C\in \textit{L}_1({\mathbb {R}}^m\times {\mathbb {R}})$, then its Fourier transform

$$\begin{aligned} f({\omega },\tau )=\int _{{\mathbb {R}}^m\times {\mathbb {R}}} \exp \left( {-i\omega ^T{\mathbf{h }}-i\tau t}\right) C({\mathbf{h }},t)d{\mathbf{h }}dt \end{aligned}$$

(66)

is continuous. Indeed, for any $\varphi =(\omega ,\tau ) \in {\mathbb {R}}^m\times {\mathbb {R}}$ and $\epsilon \in {\mathbb {R}}^m\times {\mathbb {R}},$ we have $|f(\varphi + \epsilon )- f(\varphi )|= \left| \int _{{\mathbb {R}}^m\times {\mathbb {R}}} \exp [{-i\varphi ^T ({\mathbf{h}} ,t)}][\exp [{-i\epsilon ^T ({\mathbf{h}} ,t)}]-1]C({\mathbf{h}} ,t)d{\mathbf{h}} dt\right| \le$

$$\begin{aligned} \le \int _{{\mathbb {R}}^m\times {\mathbb {R}}}|\exp [{-i\epsilon ^T ({\mathbf{h}} ,t)}]-1| |C({\mathbf{h}} ,t)|d{\mathbf{h}} dt. \end{aligned}$$

(67)

Since

$$\begin{aligned} |\exp [{-i\epsilon ^T ({\mathbf{h}} ,t)}]-1| |C({\mathbf{h}} ,t)|\le 2|C({\mathbf{h}} ,t)|, \end{aligned}$$

(68)

and

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}|\exp [{-i\epsilon ^T ({\mathbf{h}} ,t)}]-1|=0, \end{aligned}$$

(69)

for any $({\mathbf{h}} ,t) \in {\mathbb {R}}^m\times {\mathbb {R}}$, then, by the dominated convergence theorem,

$$\begin{aligned} \lim _{\epsilon \rightarrow 0} \int _{{\mathbb {R}}^m\times {\mathbb {R}}}|\exp \left[ {-i\epsilon ^T ({\mathbf{h}} ,t)}\right] -1| |C({\mathbf{h}} ,t)|d{\mathbf{h}} dt=0. \end{aligned}$$

(70)

This proves the continuity of f. Moreover, if C is not identically zero, then its Fourier transform is not identically zero on ${\mathbb {R}}^m\times {\mathbb {R}}$ and there exists $(\omega _0,\tau _0)\in {\mathbb {R}}^m\times {\mathbb {R}}$ such that $f(\omega _0,\tau _0)>0$. Hence, because of the continuity of $f(\cdot ,\cdot )$, there exists an open set $I\subset {\mathbb {R}}^m\times {\mathbb {R}}$ such that $f(\omega ,\tau )>0, \, \forall (\omega ,\tau )\in I$. Thus, because a covariance function C is SPD if the support of the density f, associated with the measure F in (4), contains an open subset, C is SPD on ${\mathbb {R}}^m\times {\mathbb {R}}$. $\square$

Proof. Theorem 6

If $C_1 \in L_1({\mathbb {R}}^{m}\times {\mathbb {R}})$ is a continuous not identically zero covariance function, i.e. it is SPD, and $C_2$ is a not identically zero (i.e. $C(\mathbf{0})>0$) positive definite covariance function on ${\mathbb {R}}^m$, then C is integrable and not identically zero

$$\begin{aligned} \int _{{\mathbb {R}}^m\times {\mathbb {R}}} |C_1({\mathbf{h },t})C_2({\mathbf{h }})| d\mathbf{h }dt\le C_2 (\mathbf{0 })\int _{{\mathbb {R}}^m\times {\mathbb {R}}} |C_1({\mathbf{h },t})| d\mathbf{h }dt<+\infty . \end{aligned}$$

(71)

The Fourier transform

$$\begin{aligned} f({\omega },\tau )=\int _{{\mathbb {R}}^m\times {\mathbb {R}}}\exp ({-i\omega ^T{\mathbf{h }}-i\tau t})C_1({\mathbf{h }},t)C_2({\mathbf{h }})d{\mathbf{h }}dt \end{aligned}$$

(72)

is continuous and, because of theorem 5, it is SPD. Similarly for the result in 2., since it is easy to show that C is integrable, then because of theorem 5, it is SPD. $\square$