Bayesian local bandwidths in a flexible semiparametric kernel estimation for multivariate count data with diagnostics

Abstract

In this paper, we consider a flexible semiparametric approach for estimating multivariate probability mass functions. The corresponding estimator is governed by a parametric starter, for instance a multivariate Poisson distribution with nonnegative cross correlations which is basically estimated through an expectation–maximization algorithm, and a nonparametric part which is an unknown weight discrete function to be smoothed through multiple binomial kernels. Our central focus is upon the selection matrix of bandwidths by the local Bayesian method. We additionally discuss the diagnostic model to enact an appropriate choice between the parametric, semiparametric and nonparametric approaches. Retaining a pure nonparametric method implies losing parametric benefices in this modelling framework. Practical applications, including a tail probability estimation, on multivariate count datasets are analyzed under several scenarios of correlations and dispersions. This semiparametic approach demonstrates superior performances and better interpretations compared to parametric and nonparametric ones.

Appendix

Proof of Theorem 1

From (6), it is enough to calculate $\textrm{Bias}[{\widetilde{w}}_{n}({\textbf{x}})]$ and $\textrm{Var}[{\widetilde{w}}_{n}({\textbf{x}})]$, since one has $\textrm{Bias}[{\widehat{f}}_n({\textbf{x}})]=p_{d}({\textbf{x}};\widehat{\varvec{\theta }}_{n}){\mathbb {E}}[{\widetilde{w}}_n({\textbf{x}})]-f({\textbf{x}})$ and $\textrm{var}[{\widehat{f}}_n({\textbf{x}})]=[p_{d}({\textbf{x}};\widehat{\varvec{\theta }}_{n})]^2 \textrm{var} [{\widetilde{w}}_n({\textbf{x}})]$. Hence,

$$\begin{aligned} {\mathbb {E}}({\widetilde{w}}_n({\textbf{x}}))={\mathbb {E}}\left( \prod _{j=1}^{d} K_{x_{j} h_{j}}\left( X_{1 j}\right) /p_{d}({\textbf{X}}_1;\,\widehat{\varvec{\theta }}_{n})\right) ={\mathbb {E}}\left[ w\left( {\textsf{Z}}_{x_{1}, h_{1}}, {\textsf{Z}}_{x_{2}, h_{2}}, \ldots , {\textsf{Z}}_{x_{d}, h_{d}}\right) \right] , \end{aligned}$$

(13)

where the random variables ${\textsf{Z}}_{x_{j},h_{j}}$ are independent with mean $\mu _j$ and variance $\sigma _j$.

Then, using a second order Taylor exapansion and finite differences, we obtain

$$\begin{aligned} & w\left( {\textsf{Z}}_{x_{1}, h_{1}}, {\textsf{Z}}_{x_{2}, h_{2}}, \ldots , {\textsf{Z}}_{x_{d}, h_{d}}\right) =w\left( \mu _{1}, \mu _{2}, \ldots , \mu _{d}\right) \\ & \quad +\sum _{j=1}^{d}\left( {\textsf{Z}}_{x_{j} h_{j}}-\mu _{j}\right) w_{j}^{(1)}+\frac{1}{2} \sum _{j=1}^{d}\left( {\textsf{Z}}_{x_{j}, h_{j}}-\mu _{j}\right) ^{2} w_{j j}^{(2)} \\ & \quad +\sum _{l \ne j}^{d}\left( {\textsf{Z}}_{x_{l},h_{l}}-\mu _{l}\right) \left( {\textsf{Z}}_{x_{j}, h_{j}}-\mu _{j}\right) w_{l j}^{(2)}+o\left( \sum _{j=1}^{d} h_{j}^{2}\right) \end{aligned}$$

and (13) becomes

$$\begin{aligned} {\mathbb {E}}({\widetilde{w}}_n({\textbf{x}}))&= w\left( \mu _{1}, \mu _{2}, \ldots , \mu _{d}\right) +\frac{1}{2} \sum _{j=1}^{d} {\text {Var}}\left( {\textsf{Z}}_{x_{j}, h_{j}}\right) w_{j j}^{(2)}+o\left( \sum _{j=1}^{d} h_{j}^{2}\right) \\&= \left[ w\left( x_{1}, \ldots , x_{d}\right) +\sum _{j=1}^{d} h_{j} w_{j}^{(1)}+\frac{1}{2} \sum _{j=1}^{d} {\text {Var}}\left( {\textsf{Z}}_{x_{j} h_{j}}\right) w_{j j}^{(2)}\right] \{1+o(1)+n^{-2}\} , \end{aligned}$$

and the desired Bias is deduced.

The pointwise variance is successively obtained with

$$\begin{aligned} {\text {Var}}({\widehat{f}}({\textbf{x}})))&= \frac{1}{n}{\text {Var}}\left( \prod _{j=1}^{d} K_{x_{j}, h_{j}}\left( X_{1j}\right) \{p_{d}({\textbf{X}}_1;\,\widehat{\varvec{\theta }}_{n})\}^{-1}\right) \\&= \frac{1}{n}\left[ f({\textbf{x}}) \{p_{d}({\textbf{x}};\,\widehat{\varvec{\theta }}_{n})\}^{-2}\left( \prod _{j=1}^{d} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j}, h_{j}}=x_{j}\right) \right) ^{2}\right. \\ & \left. +\sum _{{\textbf{y}} \in {\mathbb {T}}_{d} \backslash {\textbf{x}}} f({\textbf{y}}) \{p_{d}({\textbf{y}};\,\widehat{\varvec{\theta }}_{n})\}^{-1}\left( \prod _{j=1}^{d} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j}, h_{j}}=y_{j}\right) \right) ^{2}\right] \\ & -\frac{1}{n}\left[ \left( \prod _{j=1}^{d} f({\textbf{x}}) \{p_{d}({\textbf{x}};\,\widehat{\varvec{\theta }}_{n})\}^{-1} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j} h_{j}}=x_{j}\right) \right) ^{2}\right. \\ & \left. +\left( \sum _{{\textbf{z}} \in {\mathbb {T}}_{d} \backslash {\textbf{x}}} f({\textbf{z}}) \{p_{d}({\textbf{z}};\,\widehat{\varvec{\theta }}_{n})\}^{-1} \prod _{j=1}^{d} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j}, h_{j}}=z_{j}\right) \right) ^{2}\right] \\&= \frac{1}{n} f(\textrm{x}) \{p_{d}({\textbf{x}};\,\widehat{\varvec{\theta }}_{n})\}^{-2}(1-f(\textrm{x}))\left( \prod _{j=1}^{d} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j}, h_{j}}=x_{j}\right) \right) ^{2}+R, \end{aligned}$$

where

$$\begin{aligned} R&= \frac{1}{n}\left[ \sum _{{\textbf{y}} \in {\mathbb {T}}_{d} \backslash {\textbf{x}}} f({\textbf{y}}) \{p_{d}({\textbf{y}};\,\widehat{\varvec{\theta }}_{n})\}^{-2}\left( \prod _{j=1}^{d} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j} h_{j}}=y_{j}\right) \right) ^{2}\right. \\ & \left. -\left( \sum _{{\textbf{z}} \in {\mathbb {T}}_{d} \backslash {\textbf{x}}} f({\textbf{z}}) \{p_{d}({\textbf{z}};\,\widehat{\varvec{\theta }}_{n})\}^{-1} \prod _{j=1}^{d} {\mathbb {P}}\left( {\textsf{Z}}_{x_{j} h_{j}}=z_{j}\right) \right) ^{2}\right] \rightarrow o\left( \frac{1}{n}\right) . \end{aligned}$$

Hence, this concludes the proof. $\square$

Proof of Theorem 2

Expanding $(x+h)^y=\sum \limits _{k=0}^{y}x^k h^{y-k}y![k!(y-k)!]^{-1}$, and denoting by ${\textbf{L}}:={\widehat{f}}_{n}({\textbf{x}})\pi ({\textbf{H}})$, we successively express ${\textbf{L}}$ as

$$\begin{aligned} {\textbf{L}}&= \frac{1}{n}\sum _{i=1}^n \frac{p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)}{p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)} \prod _{j=1}^d\frac{(x_j+1)!}{X_{ij}!(x_j+1-X_{ij})!}\left( \frac{x_j+h_j}{x_j+1}\right) ^{X_{ij}}\left( \frac{1-h_j}{x_j+1}\right) ^{x_j+1-X_{ij}} \\ & \times \frac{1}{{\textbf{B}}(\alpha ,\beta )}h_j^{\alpha -1}(1-h_j)^{\beta -1} \\&= \frac{1}{n{\textbf{B}}(\alpha ,\beta )^d}\sum _{i=1}^n \frac{p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)}{p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)} \prod _{j=1}^d\sum _{k=0}^{X_{ij}}\frac{(x_j+1)!~x_j^k~h_j^{X_{ij}-k+\alpha -1}~(1-h_j)^{x_j-X_{ij}+\beta }}{(x_j+1-X_{ij})!~k!~(X_{ij}-k)!~(x_j+1)^{x_j+1}} \\&= \frac{1}{n{\textbf{B}}(\alpha ,\beta )^d}\times {\textbf{N}}. \end{aligned}$$

(14)

By direct calculation, the second term $\int _{{\mathcal {M}}}{\widehat{f}}_{n}({\textbf{x}})\pi ({\textbf{H}})d{\textbf{H}}$ of (9) becomes

$$\begin{aligned} \int _{(0,1]^d}{\widehat{f}}_{n}({\textbf{x}})\pi ({\textbf{H}})d{\textbf{H}}=\frac{1}{n{\textbf{B}}(\alpha ,\beta )^d}\times {\textbf{D}}. \end{aligned}$$

(15)

Combining (14) and (15) as in (9), we easily get the closed expression of the posterior distribution ${\widehat{\pi }}({\textbf{H}}|{\textbf{x}},{\textbf{X}}_1,\dots ,{\textbf{X}}_n)$ provided in (11).

The diagonal elements of the matrix of bandwidths are obtained as:

$$\begin{aligned} \widehat{{\textbf{H}}}({\textbf{x}})&= \int _{(0,1]^d}(h_1,\ldots ,h_d) {\widehat{\pi }}({\textbf{H}}|{\textbf{x}},{\textbf{X}}_1,\ldots ,{\textbf{X}}_n)dh_1,\ldots , dh_d\\&= \int _{(0,1]^d}h_1{\widehat{\pi }}({\textbf{H}}|{\textbf{x}},{\textbf{X}}_1,\ldots ,{\textbf{X}}_n)dh_1,\ldots ,dh_d,\ldots ,\\ & \quad \int _{(0,1]^d}h_d{\widehat{\pi }}({\textbf{H}}|{\textbf{x}},{\textbf{X}}_1,\ldots ,{\textbf{X}}_n)dh_1\ldots , dh_d. \end{aligned}$$

Then

$$\begin{aligned} {\widehat{h}}_j(x_j)&= \sum _{i=1}^n \frac{p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)}{p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)} \left( \sum _{k=0}^{X_{ij}}\frac{(x_j+1)!x_j^{k}{\textbf{B}}(X_{ij}-k+\alpha +1,x_j-X_{ij}+\beta +1)}{(x_j+1-X_{ij})!k!(X_{ij}-k)!(x_j+1)^{x_j+1}}\right) \\ & \times \left( \prod \limits _{\begin{array}{c} m=1 \\ m \ne j \end{array}}^{d} \sum _{k=0}^{X_{im}}\frac{(x_m+1)!x_m^{k}{\textbf{B}}(X_{im}-k+\alpha ,x_m-X_{im}+\beta +1)}{(x_m+1-X_{im})!k!(X_{im}-k)!(x_m+1)^{x_m+1}}\right) \\ & \times \left( \sum _{i=1}^n \frac{p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)}{p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)}\prod _{s=1}^d \sum _{k=0}^{X_{is}}\frac{(x_s+1)!x_s^k {\textbf{B}}(X_{is}-k+\alpha ,x_s-X_{is}+\beta +1)}{(x_s+1-X_{is})!k!(X_{is}-k)!(x_s+1)^{x_s+1}}\right) ^{-1}\\&= {\textbf{D}}^{-1}\sum \limits _{i=1}^{n} \frac{p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)}{p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)} \left( \sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha +1,x_j-X_{ij}+\beta +1)\right) \\ & \times \left( \prod \limits _{\begin{array}{c} m=1 \\ m \ne j \end{array}}^{d}\sum _{k=0}^{X_{im}}{\textbf{A}}_{imk}{\textbf{B}}(X_{im}-k+\alpha ,x_m-X_{im}+\beta +1)\right) , \end{aligned}$$

which corresponds to Eq. (12). $\square$

Proof of Proposition 1

Using the property of the beta function and for fixed $j \in 1,\dots ,d$, ${\widehat{h}}_{j}(x_j)$ is written as

$$\begin{aligned} & {\widehat{h}}_j(x_j)=\sum \limits _{i=1}^{n} [p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\times \\ & \qquad \left[ \sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha +1,x_j-X_{ij}+\beta _{n}+1)\right] \\ & \qquad \times \left[ \prod \limits _{\begin{array}{c} m=1 \\ m \ne j \end{array}}^{d}\sum _{k=0}^{X_{im}}{\textbf{A}}_{imk}{\textbf{B}}(X_{im}-k+\alpha ,x_m-X_{im}+\beta _{n}+1)\right] \\ & \qquad \times \left\{ \sum \limits _{i=1}^{n} \left[ p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\right. \right. \\ & \qquad \left. \left. \widehat{\varvec{\theta }}_n)\right] \left[ \prod _{j=1}^{d}\sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right] \right\} ^{-1}\\ & \quad = \sum \limits _{i=1}^{n} [p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\\ & \qquad \times \left[ \sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right. \\ & \qquad \left. \times (X_{ij}-k+\alpha )/(x_{j}+\beta _{n}-k+\alpha +1)\right] \\ & \qquad \times \left[ \prod \limits _{\begin{array}{c} m=1 \\ m \ne j \end{array}}^{d}\sum _{k=0}^{X_{im}}{\textbf{A}}_{imk}{\textbf{B}}(X_{im}-k+\alpha ,x_m-X_{im}+\beta _{n}+1)\right] \\ & \qquad \times \left\{ \sum \limits _{i=1}^{n} [p_{d}({\textbf{x}}, \widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\right. \\ & \qquad \left. \left[ \prod _{j=1}^{d}\sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right] \right\} ^{-1}. \end{aligned}$$

Hence, ${\widehat{h}}_j(x_j)$ can be bounded to the left-hand side as follows:

$$\begin{aligned} & {\widehat{h}}_j(x_j)\ge \left( \frac{\alpha }{\beta _{n}+\alpha +x+1} \right) \\ & \qquad \times \left\{ \sum \limits _{i=1}^{n} [p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\right. \\ & \qquad \left. \left[ \prod _{j=1}^{d}\sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right] \right\} \\ & \qquad \times \left\{ \sum \limits _{i=1}^{n} [p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\right. \\ & \qquad \left. \left[ \prod _{j=1}^{d}\sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right] \right\} ^{-1}\\ & \quad \ge \left( \frac{\alpha }{\beta _{n}+\alpha +x+1} \right) . \end{aligned}$$

Since $X_{ij}\le x_{j}$, the bandwidth ${\widehat{h}}_j(x_j)$ is bounded to the right-hand side by:

$$\begin{aligned} & {\widehat{h}}_j(x_j)\le \left( \frac{x+1+\alpha }{\beta _{n}} \right) \\ & \qquad \times \left\{ \sum \limits _{i=1}^{n} [p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\right. \\ & \qquad \left. \left[ \prod _{j=1}^{d}\sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right] \right\} \\ & \qquad \times \left\{ \sum \limits _{i=1}^{n} [p_{d}({\textbf{x}},\widehat{\varvec{\theta }}_n)/p_{d}({\textbf{X}}_i,\widehat{\varvec{\theta }}_n)]\right. \\ & \qquad \left. \left[ \prod _{j=1}^{d}\sum _{k=0}^{X_{ij}}{\textbf{A}}_{ijk}{\textbf{B}}(X_{ij}-k+\alpha ,x_j-X_{ij}+\beta _{n}+1)\right] \right\} ^{-1}\\ & \quad \le \left( \frac{x+1+\alpha }{\beta _{n}}\right) . \end{aligned}$$

Then, one gets

$$\begin{aligned} \left( \frac{\alpha }{\beta _{n}+\alpha +x+1} \right) \le {\widehat{h}}_j(x_j)\le \left( \frac{x+1+\alpha }{\beta _{n}}\right) {,} \end{aligned}$$

which leads to the desired result. $\square$