Abstract
In the present paper, we consider the nonparametric regression model with random design based on \((\mathbf{X}_\mathrm{t},\mathbf{Y}_\mathrm{t})_{\mathrm{t}\ge 0}\) a \(\mathbb {R}^{d}\times \mathbb {R}^{q}\)-valued strictly stationary and ergodic continuous time process, where the regression function is given by \(m(\mathbf{x},\psi ) = \mathbb {E}(\psi (\mathbf{Y}) \mid \mathbf{X} = \mathbf{x}))\), for a measurable function \(\psi : \mathbb {R}^{q} \rightarrow \mathbb {R}\). We focus on the estimation of the location \({\varvec{\Theta }}\) (mode) of a unique maximum of \(m(\cdot , \psi )\) by the location \( \widehat{{\varvec{\Theta }}}_\mathrm{T}\) of a maximum of the Nadaraya–Watson kernel estimator \(\widehat{m}_\mathrm{T}(\cdot , \psi )\) for the curve \(m(\cdot , \psi )\). Within this context, we obtain the consistency with rate and the asymptotic normality results for \( \widehat{{\varvec{\Theta }}}_\mathrm{T}\) under mild local smoothness assumptions on \(m(\cdot , \psi )\) and the design density \(f(\cdot )\) of \(\mathbf{X}\). Beyond ergodicity, any other assumption is imposed on the data. This paper extends the scope of some previous results established under the mixing condition. The usefulness of our results will be illustrated in the construction of confidence regions.
Similar content being viewed by others
1 Introduction
Nonparametric estimation has been the subject of intense investigation for many years and this has led to the development of a large variety of methods. Because of numerous applications and their important role in mathematical statistics, the problem of estimating the density and regression function has been the subject of considerable interest during the last decades. One of the most commonly used classes of estimators is that formed by the so-called kernel-type estimators. For more theoretical aspects along with statistical applications the interested reader is referred to Tapia and Thompson [71], Wertz [74], Devroye and Györfi [23], Devroye [22], Nadaraya [57], Härdle [38], Wand and Jones [73], Eggermont and LaRiccia [30], Devroye and Lugosi [24] and the references therein. Recently, a number of statistical problems has found an unexpected solution being investigated by a “modal point of view”. This investigation includes classical processes such as clustering. This has led to a renewed interest in the estimation and the inference for the mode. The estimation of the conditional mode of an outcome variable given the regressors, is called modal regression. Modal regression is an alternative approach to the usual regression methods for exploring the relationship between a response variable \(\mathbf{Y}\) and a predictor variable \(\mathbf{X}\). Unlike conventional regression, which is based on the conditional mean of \(\mathbf{Y}\) given \(\mathbf{X} = \mathbf{x}\), modal regression estimates conditional modes of \(\mathbf{Y}\) given \(\mathbf{X} = \mathbf{x}\). Modal regression is a more reasonable modelling approach than the usual regression at least in two scenarios. Firstly when the conditional density function is skewed or has a heavy tail. When the conditional density function has skewness, the conditional mean may not provide a good representation for summarising the relations between the response and the covariate. The other scenario is when the conditional density function has multiple local modes. This occurs when the relation of \(\mathbf{X}\) and \(\mathbf{Y}\) contains multiple patterns. The conditional mean may not capture any of these patterns, so it can be a very bad summary; see Chen et al. [17] for an example. This situation has already been pointed out in Tarter and Lock [72]. Modal regression has a wide variety of applications including the analysis of traffic and forest fire data [31, 75], econometrics [45, 50, 51], and machine learning [33, 68]. For example, Kemp and Santos Silva [45] argue that the mode is the most intuitive measure of central tendency for positively skewed data found in many econometric applications such as wages, prices, and expenditures [45, p. 93]. For more recent reviews and further details on the subject the reader is referred to Chen [16] and Chacón [14].
We will start by providing some notation and definitions that are needed for the forthcoming sections. Let \((\mathbf{X}_\mathrm{t},\mathbf{Y}_\mathrm{t})_{\mathrm{t}\ge 0}\) be a \(\mathbb {R}^{d}\times \mathbb {R}^{q}\)-valued strictly stationary and ergodicFootnote 1 continuous time process defined on a probability space \((\Omega , \mathcal {F},\mathbb {P)}\). Let \(g(\cdot ,\cdot )\) be the density function of the random vector \((\mathbf{X}_\mathrm{t},\mathbf{Y}_\mathrm{t})\), \(f(\cdot )\) be the density of \(\mathbf{X}_\mathrm{t}\) and \(\rho (\cdot )\) the density of \(\mathbf{Y}\). For a given measurable function \(\psi (\cdot )\) and \(\mathbf{x}\in \mathbb {R}^{d}\) the regression function, whenever it exists, is defined to be
In this situation, we have the random design regression model and \(\mathbf{X}\) is called the design variable and \(\mathbf{Y}\) the response variable. The random design model is very important in clinical studies, where the design variable usually represents the age of a particular individual receiving treatment, and \(\mathbf{Y}\) is the quantity whose dependence on the age of the patient is investigated. A typical example (from forensic medicine) is given by Härdle and Marron [39], where \(\mathbf{Y}\) stands for the liver weight of female persons (depending on their age). Inequalities \(\mathbf{x} \le \mathbf{y}\) holds for all the components, i.e., \(x_{j}\le y_{j}\) for all \(j = 1,\ldots ,d\). The introduction of the function \(\psi (\cdot )\) allows us to include some important special cases:
-
\(\psi (\mathbf{Y}) = \mathbb {1}\{\mathbf{Y}\le \mathbf{y}\}\) gives the conditional distribution of \(\mathbf{Y}\) given \(\mathbf{X}=\mathbf{x}\).
-
\(\psi (\mathbf{Y}) =\mathbf{Y}^{k}\) gives the conditional moments of \(\mathbf{Y}\) given \(\mathbf{X}=\mathbf{x}\).
In the present paper, we focus on estimating the location \({\varvec{\Theta }}\) and the size \(m({\varvec{\Theta }},\psi )\) of a unique maximum (mode, peak) of the (unknown) function \(m(\cdot ,\psi )\). Our method is indirect in the sense that the estimators of \({\varvec{\Theta }}\) and \(m({\varvec{\Theta }},\psi )\) are based on a kernel estimator \(\widehat{m}_\mathrm{T}(\mathbf{x},\psi )\) of the regression curve \(m(\mathbf{x},\psi )\). We will use the Nadaraya–Watson estimator which is defined by
where \(K(\cdot )\) is a kernel, \(h_\mathrm{T}\) is a positive sequence of real numbers such that
The condition (i) is used to obtain the asymptotic unbiasedness of the kernel (density or regression) type estimators. We need more restrictive assumption on \(h_{T}\) for the consistency, this is given by the condition (ii), one can refer to Parzen [61]. In general, the strong consistency fails to hold when either (i) or (iii) is not satisfied. Now the location \({\varvec{\Theta }}\) (mode) and the size \(m({\varvec{\Theta }},\psi )\) are estimated by the respective functionals \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) and \(\widehat{m}_\mathrm{T}(\widehat{{{\varvec{\Theta }} }}_\mathrm{T},\psi )\) pertaining to \(\widehat{m}_\mathrm{T}(\cdot ,\psi )\), i.e., \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) is chosen through the equation
where the maximum is running over some compact set \(\mathfrak {C}\subset \mathbb {R}^{d}\). Note that \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) exists if \(K(\cdot )\) is continuous; however, it may not be unique. In fact, it is known that kernel estimators tend to produce some additional and superfluous modality. In this context, one can consider
where the infimum is taken with respect to the lexicographic order on \(\mathbb {R}^{d}\). However, this has no bearing on asymptotic theory; our results are valid for any choice of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) satisfying (1.2). To ensure both uniqueness and measurability of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\), one could use the so-called mode functional on \(C(\mathfrak {C})\) apparently introduced by Eddy [28] which considers the infimum of the maximised locations and whose measurability is also proved in a paper of Eddy [28]. Alternatively, Grund and Hall [35] have suggested to break ties at random if necessary. Anyway, the validity of our proofs will not be affected by potential non-measurability of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\), since we can always replace probabilities by outer probabilities when necessary with no further changes in the proofs, this issue is discussed in Ziegler [79], and also Ziegler [77, 78], Herrmann and Ziegler [41]. As it is mentioned in Ziegler [79], estimating the mode and size of a maximum of a nonparametric curve by the corresponding functionals of a kernel estimator of the curve is not new; it stems from the closely related problem of estimating the mode of a density. In continuation of Parzen [61] pioneering work on density estimation and estimation of the mode, Eddy [28, 29] and Romano [66] tackled optimality questions of the kernel density estimators of the mode. Romano [66] seems also to be the first to consider data-dependent bandwidths in this framework. In another paper, Romano [65] examined the limiting behaviour of bootstrap estimators of the location of the mode, an idea used later by Grund and Hall [35] in the context of bandwidth selection by minimising the bootstrapped \(L_{p}\)-error for the mode estimator. It is worth noticing that the conditional mode function estimate of the predictor is used for the first time by Collomb et al. [18]. The kernel type estimators were studied extensively in different setting of dependencies, we cite among many others Samanta and Thavaneswaran [67], Ould-Saïd [59], Quintela-Del-Río and Vieu [64], Berlinet et al. [5], Ferraty et al. [34], Ezzahrioui and Ould-Saïd [32], Benrabah et al. [3] and the references therein. Quintela-Del-Río and Vieu [64] motivated the use of the conditional mode by pointing out that the prediction of \(\mathbf{Y}\)-values given the \(\mathbf{X}\)-values is achieved through the regression function estimation. Finally, when the process is considered to be i.i.d., the almost sure convergence along with the mean convergence of the conditional density were obtained by Youndjé [76]. Ota et al. [60] proposed a new estimator of the conditional mode that is able to avoid the curse of dimensionality and at the same time is computationally scalable, thereby complementing the above existing methods.
Within the framework described above, our aim is to establish consistency and asymptotic normality results (which in turn can be exploited for the construction of confidence intervals) for the estimators \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) and \(\widehat{m}_\mathrm{T}(\widehat{{{\varvec{\Theta }} }}_\mathrm{T},\psi )\) for the location and the size of the peak under some mild local smoothness conditions on the regression function \(m(\cdot ,\psi )\) and the design density \(f(\cdot )\) (mostly imposed locally in a neighbourhood of \({{\varvec{\Theta }} }\)). Those results will be valid for a wide class of kernels not necessarily having compact support. This includes, in particular, the Gaussian kernel which is widely used in practice. Mixing is some kind of asymptotic independence assumption which is commonly used for seek of simplicity but which can be unrealistic in situations where there is strong dependence between the data. Extending non-parametric functional ideas to general dependence structure is a rather underdeveloped field. Note that the ergodic framework avoids the widely used strong mixing condition and its variants to measure the dependency and the involved probabilistic calculations that it implies (see, for instance, Masry [55]). It is worth noticing that the ergodicity is implied by all mixing conditions, being weaker than all of them. Further motivations to consider ergodic data are discussed in Laib and Louani [48, 49], Didi and Louani [27], Bouzebda et al. [12], Bouzebda et al. [8], Bouzebda and Didi [9,10,11] and Krebs [46], in some of these references the definitions of the ergodic property of continuous time processes are given. In the present work, we do not assume anything beyond ergodicity of the underlying process. It is worth noticing that strong mixing implies ergodicity; see e.g., Remark 2.6 on page 50 in combination with Proposition 2.8 on page 51 in Bradley [13]. Hence the present work extends the scope of applications compared to the existing works. On the other hand, we mention that there exist interesting processes which are ergodic but not mixing according to Andrews [1] and Bradley [13]. An example of an ergodic and non-mixing process was considered in Sect. 5.3 of Leucht and Neumann [52]. Indeed, assume that the process \(\{(T_{i},\lambda _{i}):i\in \mathbb {Z}\}\) is strictly stationary with \(T_{i}\mid \mathcal {T}_{i-1}\sim \text{ Poisson }(\lambda _{i})\), let \(\mathcal {T}_i\) be the \(\sigma \)-field generated by \((T_{i}, \lambda _{i},T_{i-1},\lambda _{i-1},\ldots )\). We assume that \(\lambda _{i}=\kappa (\lambda _{i-1}, T_{i-1})\), where \(\kappa :[0,\infty )\times \mathbb {N}\rightarrow (0,\infty )\). However, this process is not mixing in general; see Remark 3 of Neumann [58] for a counterexample. We refer to Leucht and Neumann [52] for further details and motivations for the use of the ergodicity assumption. One of their arguments, is that for certain classes of processes, it can be much easier to prove ergodicity rather than mixing assumption. It is known that any sequence \(\{\varepsilon _\mathrm{t}:t\in \mathbb {Z}\}\) of i.i.d. random variables is ergodic. Hence, it is immediately clear that \(\{\mathbf{Y}_\mathrm{t} :t \in \mathbb {Z}\}\) with
is also ergodic. Didi [25] has constructed an example of a non-mixing ergodic continuous time process. It is well known that the fractional Brownian motion \(\{W_\mathrm{t}^H:t\ge 0\}\) with parameter \(H\in (0,1)\) has strictly stationary increments. Otherwise, the fractional Gaussian noise, defined for every \(s>0\) by
is a strictly stationary centered long memory process when \(H\in (\frac{1}{2},1)\) (see for instance, Beran [4, p.55] and Lu [53, p.17]), hence the condition of strong mixing is not satisfied. Let \(\{G_\mathrm{t}:t\ge 0\}\) be a strictly stationary centered Gaussian process with correlation function
Relaying on the work of Maslowski and Pospíšil [54], Lemma 4.2, it follows that the process \(\{G_\mathrm{t}:t\ge 0\}\) is ergodic whenever
which is the case for the process \(\{G_\mathrm{t}^H: t\ge 0\}\). The ergodicity hypothesis seems to be the most naturally adapted and provides a better framework to study data series, for example, generated by noisy chaos.
To the best of our knowledge, the results presented here, respond to a problem that has not been studied systematically until recently, and it gives the main motivation to this paper. Indeed, we establish the exact rate of strong uniform consistency of the estimators \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) and we characterise the limiting law. To prove our results, we base our methodology upon the martingale approximation which allows to provide a unified nonparametric time series analysis framework enabling one to study systematically dependent data. This methodology is a quite different approach, in the i.i.d. context, compared to the existing ones.
The layout of the article is as follows. The assumptions and asymptotic properties of the estimators are given in Sect. 2, which includes the optimal convergence rates and the asymptotic normality of the estimators \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\). Some concluding remarks and possible future developments are mentioned in Sect. 3. To avoid interrupting the flow of the presentation, all mathematical proofs are presented in Sect. 4.
2 Main results
Let us introduce some notation and definitions. Let \(\alpha = (\alpha _{1},\ldots ,\alpha _{d})\) be a multi-index of the nonnegative integers \(\alpha _{ i}\), set \(|\alpha |=\sum _{ i=1}^{d}\alpha _{ i}\), and let
denote the partial differential operator of order \(\alpha \). For \(\alpha =0\) set \(D^{\alpha }=id\), for identity. For continuous real-valued functions \(\zeta _{1}(\cdot )\) and \(\zeta _{2}(\cdot )\) that are s-times continuously differentiable on \(\mathbb {R}^{d}\),
We will use the notation
Let us define the partial derivatives of order one of the regression estimator by
The derivatives of order \(\alpha =1,2\) of the estimators \(f_\mathrm{T}(\mathbf{x})\) and \(M_\mathrm{T}(\mathbf{x},\psi )\) are defined as follows
and
We denote by \(m^{(1)}(\cdot ,\psi )\) the gradient of the function \(m(\cdot ,\psi )\) : \(\mathbb {R}^{d} \rightarrow \mathbb { R}\), that is, \(m^{(1)}(\cdot ,\psi )\) is the \(d \times 1\)-vector of the partial derivatives of \(m(\cdot ,\psi )\)
Using the definition of the conditional mode function, i.e. the mode of \(m(\cdot ,\psi )\), we have
Similarly, it follows from the statement (2.1) that
We denote by \(m^{(2)}(\cdot ,\psi )\) the Hessian of the function \(m(\cdot ,\psi )\), that is, \(m^{(2)}(\cdot ,\psi )\) is the \(d \times d\)-matrix of the second partial derivatives of \(m(\cdot ,\psi )\) . Furthermore, assumption (A.7) implies that
By the definition of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\), we have \(\widehat{m}^{(1)}_\mathrm{T}(\widehat{{{\varvec{\Theta }} }}_\mathrm{T},\psi ) = 0\) so that
For each \(i \in \{1,\ldots , d\}\), Taylor’s expansion applied to the real-valued application \(\frac{\partial }{\partial _{x_{i}}} \widehat{m}^{(1)}_\mathrm{T}(\cdot ,\psi )\) implies the existence of \({\varvec{\Theta }}_\mathrm{T}^\star (i)=(\Theta _{\mathrm{T},1}^\star (i),\ldots ,\Theta _{\mathrm{T},d}^\star (i))^{\top }\)
Define the \(d \times d\) matrix \(H_\mathrm{T} = (H_{\mathrm{T},i,j} )1\le i,j\le d\) by setting
Equation (2.2) can then be rewritten as
The last relation will play an important role in our proofs, in particular for the asymptotic normality. To formulate our assumptions, some additional notation is required, for some constant \(\delta >0\) small enough, let \(n\in \mathbb {N}\) be such that \(T=\delta n\), and \(T_j=j\delta \), for \(j=1,\ldots ,n\). Let \(\mathcal {F}_\mathrm{t}\) be the \(\sigma -\)field defined by
Set \(\mathcal {F}_{j}\) to be the \(\sigma \)-field defined by
Let \( \mathcal {S}_{\mathrm{t},\delta }\) be the \(\sigma \)-field defined by
Let \(\mathcal {G}_t:=\sigma \{(\mathbf{X}_s,\mathbf{Y}_s): 0\le s\le t\}\), and for \(\delta >0\) small enough, let \(g^{\mathcal {G}_{t-\delta }}(\cdot )\) and \(\rho ^{\mathcal {G}_{t-\delta }}(\cdot )\) be the conditional densities of \((\mathbf{X},\mathbf{Y})\) and \(\mathbf{Y}\) respectively, given the \(\sigma -\)field \(\mathcal {G}_{t-\delta }\). Finally, if \(\zeta (\cdot )\) is a real-valued random function which satisfies \(\zeta (u) / u \rightarrow 0\) a.s. as \(u \rightarrow 0,\) we write \(\zeta (u)=o_{\text{ a.s. } }(u)\). In the same way, we say that \(\zeta (u)\) is \(O_{\text{ a.s. } }(u)\) if \(\zeta (u) / u\) is a.s. bounded as \(u \rightarrow 0 .\)
2.1 Assumptions
In our analysis, the following assumptions are needed.
-
(A.1)
The kernel \(K(\cdot )\), is a probability density function compactly supported,
-
(i)
Kernel K is assumed to be Lipschitz with ratio \(C_K<\infty \) and order \(\gamma \), i.e.,
$$\begin{aligned} |K(\mathbf{x})-K(\mathbf{x}^{'}) | \le C_K\Vert \mathbf{x}-\mathbf{x}^{'}\Vert ^{\gamma }, \quad (\mathbf{x},\mathbf{x}^{'})\in \mathbb {R}^{2d}; \end{aligned}$$ -
(ii)
\(\int _{\mathbb {R}^d} \Vert \mathbf{x}\Vert K(\mathbf{x}) d\mathbf{x} <\infty ;\)
-
(i)
-
(A.2)
There exists \(\Gamma < \infty \), such that, for all \(\mathbf{x} \in \mathfrak {C}\),
$$\begin{aligned} \underset{\mathbf{x} \in \mathfrak {C}}{\sup } |m(\mathbf{x},\psi )|< \Gamma ; \end{aligned}$$ -
(A.3)
-
(i)
Recall that \(\mathfrak {C}\) is a compact set of \(\mathbb {R}^d\). Assume, for all \(\mathbf{x} \in \mathfrak {C}\), that there exists \(\lambda >0\) and finite constant \(0<\eta \) such that
$$\begin{aligned} \lambda \le f(\mathbf{x}) \le \eta ; \end{aligned}$$ -
(ii)
the density \(f(\cdot )\) is an element of \(\mathcal {C}^2(\mathbb {R}^{d})\);
-
(i)
-
(A.4)
For every \(t\in \mathbb {R}_+\), for every \(\mathbf{x}\in \mathbb {R}^{d},\)
-
(i)
The conditional density \(f_{\mathbf{X}_\mathrm{t}}^{\mathcal {F}_{\mathrm{t}- \delta }}(\cdot )\) of \(\mathbf{X}_\mathrm{t}\) given the \(\sigma \)-field \(\mathcal {F}_{\mathrm{t}- \delta }\) exists a.s. and is an element of \(\mathcal {C}^2(\mathbb {R}^{d})\);
-
(ii)
For any \(\delta >0\) small enough
$$\begin{aligned} \underset{T\rightarrow \infty }{\lim } \frac{1}{T} \int _0^T f_{\mathbf{X}_\mathrm{t}}^{\mathcal {F}_{\mathrm{t}-\delta }}(\mathbf{x}) dt = f(\mathbf{x}) , \quad \text{ a.s. }; \end{aligned}$$
-
(i)
-
(A.5)
For every \(t\in \mathbb {R}_+\), for every \(\mathbf{x}\in \mathbb {R}^{d},\)
$$\begin{aligned} \underset{T\rightarrow \infty }{\lim } \underset{\mathbf{x} \in \mathbb {R}^d}{\sup } \left| \frac{1}{T} \int _0^T f_{\mathbf{X}_\mathrm{t}}^{\mathcal {F}_{\mathrm{t}-\delta }}(\mathbf{x}) dt - f(\mathbf{x}) \right| = 0, \quad \text{ a.s. }, \end{aligned}$$for any \(\delta >0\) small enough;
-
(A.6)
For any t and r such that \(t\in [0,T]\) and \(t\le r\le t+\delta \) we have
-
(i)
$$\begin{aligned} \mathbb {E}(|\psi (\mathbf{Y}_r)|\vert \mathcal {S}_{\mathrm{t},\delta })=\mathbb {E}(|\psi (\mathbf{Y}_r)|\vert \mathbf{X}_r)=m(\mathbf{X}_r); \end{aligned}$$
-
(ii)
there exist constants \(C_\psi >0\) and \(\beta >0\) such that, for any couple \((\mathbf{x},\mathbf{x}^\prime )\in \mathbb {R}^{2d}\),
$$\begin{aligned} \left| m(\mathbf{x},\psi )-m(\mathbf{x}^\prime ,\psi )\right| \le C_\psi \left\| \mathbf{x}-\mathbf{x}^\prime \right\| ^\beta ; \end{aligned}$$ -
(iii)
For any \(k\ge 2\) and any \(\delta >0\),
$$\begin{aligned} \mathbb {E}\left( \left| \psi ^k(\mathbf{Y}_r)\right| \vert \mathcal {S}_{\mathrm{t},\delta }\right) =\mathbb {E}\left( \left| \psi ^k(\mathbf{Y}_r)\right| \vert \mathbf{X}_r\right) , \end{aligned}$$and the function \(\Phi _k(\mathbf{x},\psi )=\mathbb {E}\left( \left| \psi ^k(\mathbf{Y})\right| \vert \mathbf{X}=\mathbf{x}\right) \) is continuous in the neighbourhood of \(\mathbf{x}\);
-
(i)
-
(A.7)
For any fixed \(\mathbf{x} \in \mathbb {R}^d\),
-
(i)
\(m(\mathbf{x},\psi )\) is twice differentiable on \(\mathbb {R}^{d}\), the matrix \(m^{(2)}(\mathbf{x},\psi )\) is continuous in a neighbourhood of \({\varvec{\Theta }}\), and \(m^{(2)}({\varvec{\Theta }},\psi )\) is nonsingular;
-
(ii)
\(m^{(2)}({\varvec{\Theta }},\psi )\) is bounded on \(\mathbb {R}^{d}\).
-
(i)
2.2 Comments on hypotheses
Conditions (A.1) are very common in the nonparametric function estimation literature. Notice that the condition (A.1) is classical in the nonparametric estimation procedures. In particular, by imposing the condition (A.1)(i), the kernel function exploits the smoothness of the density function or the regression function. If we loose the condition that the kernel function \(K(\cdot )\) must be a density, the convergence rate could be faster. Indeed, the convergence rate can be made arbitrarily close to the parametric \(n^{-1}\) as the order increases. In fact, Chacón et al. [15] showed that the parametric rate \(n^{-1}\) can be attained by the use of super-kernels, and that super-kernel density estimators automatically adapt to the unknown degree of smoothness of the density. The main drawback of higher-order kernels in this situation is the negative impact of the kernel may make the estimated density not a density itself. The interested reader is referred to, e.g., Jones et al. [44], Jones and Signorini [43] and Jones [42]. They set some kind of regularity upon the kernels used in our estimates. Notice that the transform of the stationary ergodic process \((\mathbf{X}_\mathrm{t},\mathbf{Y}_\mathrm{t})_{\mathrm{t}\ge 0}\) into the process \((\psi ^2(\mathbf{Y}_\mathrm{t}))_{\mathrm{t}\ge 0}\) is a measurable function. Therefore, making use of Proposition 4.3 of Krengel [47] and then the ergodic Theorem, we obtain
Condition (A.3)(ii), is a technical condition that simplifies our proofs, precisely, we assume that the density function \(f(\cdot )\) is bounded away from zero and infinity on the compact set \(\mathfrak C\) in a similar way as in Ziegler [79], Stute [70], Harel and Puri [40], Debbarh [20]. For any set \(B\subset \mathbb {R}^d\) and \(\epsilon >0\), denote by \(B^\epsilon \) the set of all \(\mathbf{x}\in \mathbb {R}^d\) such that there exists \(\mathbf{y}\in B\) with \(\Vert \mathbf{x}-\mathbf{y}\Vert <\epsilon \). One can use that \(f(\cdot )\) is continuous and strictly positive on \(\mathfrak C^\epsilon \), but this will add much extra complexity to the proofs. Condition (A.4) involves the ergodic nature of the data as given, for instance, in Györfi et al. [36]. Assuming that \(\rho ^{\mathcal G_{\mathrm{t}-\delta }}(\cdot )\) and \(g^{\mathcal G_{\mathrm{t}-\delta }}(\cdot )\) belong to the space \(\mathfrak {C}^0\), at least, of continuous functions, which is a separable Banach space. Moreover, approximating the integral \(\displaystyle {\int _0^T \rho ^{\mathcal G_{\mathrm{t}-\delta }}(\mathbf{y}) dt}\) and \(\displaystyle {\int _0^T g^{\mathcal G_{\mathrm{t}-\delta }}(\mathbf{x},\mathbf{y}) dt}\) by their Riemann’s sums, it follows that
and
Since the processes \((\mathbf{X}_{\mathrm{T}_j}, \mathbf{Y}_{\mathrm{T}_j})_{j\ge 1}\) and \((\mathbf{Y}_{\mathrm{T}_j})_{j\ge 1}\) are stationary and ergodic (see Proposition 4.3 of Krengel [47]) following Delecroix [21] (see, Lemma 4 and Corollary 1 along with with their proofs), one may prove that the sequences \((\rho ^{\mathcal G_{(j-1)\delta }}(\mathbf{y}))_{j\ge 1}\) and \((g^{\mathcal G_{(j-1)\delta }}(\mathbf{x},\mathbf{y}))_{j\ge 1}\) of conditional densities are stationary and ergodic. Moreover, making use of Beck [2]’s theorem (see, for instance, Györfi et al. [36], Theorem 2.1.1), it follows that
and
It is then clear that both the conditions (A.4) and (A.5) are satisfied. Condition (A.6)(i) is usual in the literature dealing with the study of ergodic processes. The hypothesis (A.6)(ii) is a regularity condition upon the regression function. For the condition (A.6)(iii), we can refer to the following examples.
Example 2.1
Consider the regression model \(Y_t= m(X_t) + \epsilon _t,\) where the random variables \(\epsilon _t\)’s stand as martingale differences with respect to the \(\sigma \)-field \(\mathcal {S}_{r,\delta }, r\le t\le r+\delta \), generated by \(\big \{(X_s,\epsilon _s), (X_t) : 0\le s< r, r\le t\le r + \delta \big \}\). Clearly, we have
almost surely.
Example 2.2
Consider the regression model \(Y_t= m(X_t) + \sigma (X_t)\epsilon _t\), where the random variables \(\epsilon _t\) are centered and independent of the process \((X_t)_{t\ge 0}\). Taking \(\mathcal {S}_{r,\delta }\) as the \(\sigma \)-field generated by \(\big \{(X_s): 0\le s\le r\big \}\), it follows, for \(t\le r\), that
almost surely.
Remark 2.3
For notational convenience, we have chosen the same bandwidth sequence for all margins. This assumption can be dropped easily. If one wants to make use of the vector bandwidths (see, in particular, Chapter 12 of Devroye and Lugosi [24]). With obvious changes in the notation, our results and their proofs remain true when \(h_\mathrm{T}\) is replaced by a vector bandwidth \(\mathbf{h}_\mathrm{T} = (h^{(1)}_\mathrm{T}, \ldots , h^{(1)}_\mathrm{T})\), where \(\min h^{(i)}_\mathrm{T} > 0\). In this situation we set \(h_\mathrm{T}=\prod _{i=1}^{d} h_\mathrm{T}^{(i)}\), and for any vector \(\mathbf{v} = (v_{1} ,\ldots ,v_{d})\) we replace \(\mathbf{v}/h\) by \((v_{1}/h_\mathrm{T}^{(1)},\ldots ,v_{d}/h_\mathrm{T}^{(d)})\). For a better understanding we will use real-valued bandwidths throughout the text.
2.3 Theoretical properties
Below, we write \(Z {\mathop {=}\limits ^{\mathcal {D}}} \mathcal {N}(\mu , \sigma ^{2} )\) whenever the random variable Z follows a normal law with expectation \(\mu \) and variance matrix \(\sigma ^{2}\), \({{\mathop {\rightarrow }\limits ^{\mathcal {D}}}}\) denotes the convergence in distribution and \({{\mathop {\rightarrow }\limits ^{\mathbb {P}}}}\) the convergence in probability.
2.3.1 Consistency
The following theorem gives the almost sure consistency result.
Theorem 2.4
Under the hypotheses (A.1)–(A.4) and (A.6), for any n large enough, we have
The proof of Theorem 2.4 is postponed to the Sect. 4.
2.3.2 Asymptotic normality
To establish the asymptotic normality of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\), observing the statement (2.4), we have to prove that the numerator suitably normalised is asymptotically normally distributed and that the denominator converges in probability to \(m_\mathrm{T}^{(2)}(\widehat{{{\varvec{\Theta }} }}_\mathrm{T},\psi )\). Let G be the \(d \times d\) matrix defined by, for \(i,j=1,\ldots ,d\),
Let us introduce the matrix \(V({\varvec{\Theta }},\psi )\), the \(d \times d\) matrix defined by, for \(i,j=1,\ldots ,d\)
The main result to be proved here may now be stated precisely as follows.
Theorem 2.5
-
1.
Under the assumptions (A.1), (A.3)(i)–(ii), (A.4) and (A.6), for any n large enough, we have
$$\begin{aligned} \sqrt{Th_\mathrm{T}^{d+1}} \widehat{m}_\mathrm{T}^{(1)}({\varvec{\Theta }},\psi )\overset{D}{\rightarrow }N(0,{V({\varvec{\Theta }},\psi )}). \end{aligned}$$ -
2.
If the assumptions (A.1)–(A.5), (A.6)(i) and (A.7) are fulfilled, we have, as \(T\rightarrow \infty \), \(\widehat{m}_\mathrm{T}^{(2)}(\cdot ,\psi )\) converges uniformly to \(m^{(2)}(\cdot ,\psi )\) on the compact set \(\mathfrak {C}\). Then, for any n large enough, we have
$$\begin{aligned} \sqrt{Th_\mathrm{T}^{d+1}} \left( \widehat{{{\varvec{\Theta }} }}_\mathrm{T}-{\varvec{\Theta }}\right) \overset{D}{\rightarrow }N(0,[m^{(2)}({\varvec{\Theta }},\psi )]^{-1}V({\varvec{\Theta }},\psi )[m^{(2)}({\varvec{\Theta }},\psi )]^{-1}). \end{aligned}$$(2.6)
The proof of Theorem 2.5 is postponed to the Sect. 4.
2.4 Confidence set
The asymptotic variance in the central limit theorem depends on the unknown functions, which should be estimated in practice. Let us introduce the matrix \(\widehat{V}(\widehat{{{\varvec{\Theta }} }}_\mathrm{T},\psi )\) an estimate of \(V({\varvec{\Theta }},\psi )\), that is a \(d \times d\) matrix defined for \(i,j=1,\ldots ,n\), by
The asymptotic variance is estimated by
Furthermore, from (2.6), the approximate confidence region of \({\varvec{\Theta }}\) can be obtained as
where \(c_\alpha \), denotes the \((1-\alpha )-\)quantile of the multivariate normal distribution. Note that \(c_\alpha \) is not unique since \({\varvec{\Theta }}\) is assumed to be a vector. Sinotina and Vogel [69] used a different approach to construct confidence sets derived as suitable neighbourhoods for maximum points of a regression estimator. The approach relies on the concentration-of-measure inequalities for the regression estimators.
Remark 2.6
It can be observed that our proofs constitute a generalisation of those used in the kernel density mode. Hence, one can obtain easily the corresponding results for the mode density estimators as a particular case of our setting. More precisely, one can consider the kernel estimator of the conditional density of \(\mathbf{Y}\) given \(\mathbf{X}=\mathbf{x}\), defined by
where \(\mathbf{K}(\cdot )\) is a kernel, \(\breve{h}_\mathrm{T}\) is a positive sequence of real numbers tending to 0 at a specific rate. We refer to Bouzebda et al. [8] for more details about the framework of functional ergodic discrete time processes.
Remark 2.7
Chen et al. [17] considered that the conditional (or local) mode set at x is defined as
where \(p(y\mid x) = p(x,y)/f(x)\) is the conditional density of Y given \(X=x\). As a simplification, the set M(x) can be expressed in terms of the joint density as:
At each x, the local mode set M(x) may consist of several points, and so M(x) is in general a multivalued function. Under appropriate conditions, as we will show, these modes change smoothly as x changes. Thus, local modes behave like a collection of surfaces called modal manifolds in Chen et al. [17]. In our setting, we have considered the extension of the work of Ziegler [79] to the multivariate ergodic setting. The approaches are different and the extension of Chen et al. [17] to the ergodic setting is of interest. The proof of such a statement, however, should require a different methodology than that used in the present paper, and we leave this problem open for future research.
Remark 2.8
In continuous time, data are often collected by using a sampling scheme. Several discretisation schemes have been proposed throughout the literature including deterministic and randomised sampling. The interested reader is referred to Masry [56], Prakasa Rao [62, 63], Bosq [7] and Blanke and Pumo [6]. To simplify the idea, we consider the density estimator of \(f(\cdot )\) based on \(\{\mathbf{X}_{t}: t\in [0,T]\}\) and let \(\{\mathbf{X}(t_k): k=1,\ldots ,n\}\) be its sampled discrete sequence. The sampled estimator of the density \(f(\cdot )\) is then
As in Masry [56], we only recall two cases of designs: irregular sampling and random sampling.
- Deterministic sampling.:
-
Consider the case where the instants \((t_{k})_{1\le k\le n}\) are deterministic irregularly spaced with
$$\begin{aligned} \inf _{1\le k\le n}|t_{j+1}-t_{j}|=\frac{1}{\tau }, \end{aligned}$$for some \(\tau >0\). For \(1\le k \le n\), consider \(\mathcal {G}_{k} :=\sigma (X(t_{k}))\) the \(\sigma \)-field generated by \(\{\mathbf{X}_{s} :0\le s\le t_{k}\}\). Obviously, \((\mathcal {G}_{k})_{1\le k \le n}\) in an increasing family of \(\sigma \)-fields.
- Random sampling.:
-
Assume that the instants \((t_{k})_{1\le k\le n}\) form a sequence of uniform random variables in the interval [0, T] independent of the process \(\{\mathbf{X}_{t}: t\in [0,T]\}\). Define
$$\begin{aligned} 0 \le \tau _{1}< \cdots < \tau _{n} \le T \end{aligned}$$as the associated order statistics. Notice that \((\tau _{k})_{1\le k \le n}\) are the process observation points. Obviously, the spacings between these points are all positive. As a consequence, taking \(\mathcal {G}_{k} :=\sigma (X(t_{k}))\) the \(\sigma \)-field generated by \(\{\mathbf{X}_{s} :0\le s\le \tau _{k}\}\), it follows that \((\mathcal {G}_{k})_{1\le k \le n}\) is a sequence of increasing \(\sigma \)-fields.
We would like here to mention that the penalisation procedure for the choice of the mesh \(\delta \) of the observations gives an optimal rate of convergence as demonstrated in Comte and Merlevède [19], we leave this problem open for future research in the framework of ergodic processes.
3 Concluding remarks
In the present paper, we are mainly concerned with the nonparametric regression model, where the regression function \(m(\cdot , \psi )\) is given by \(m(\mathbf{x},\psi ) = \mathbb {E}(\psi (\mathbf{Y}) \mid \mathbf{X} = \mathbf{x}))\). For a measurable function \(\psi : \mathbb {R}^{q} \rightarrow \mathbb {R}\), estimation of the location \({\varvec{\Theta }}\) (mode) of a unique maximum of \(m(\cdot , \psi )\) by the location \( \widehat{{\varvec{\Theta }}}_\mathrm{T}\) of a maximum of the Nadaraya–Watson kernel estimator \(\widehat{m}_\mathrm{T}(\cdot ,\psi )\) for the curve \(m(\cdot , \psi )\) is considered. Within this context, we obtain consistency and asymptotic normality results for \( \widehat{{\varvec{\Theta }}}_\mathrm{T}\) under mild local smoothness assumptions on \(m(\cdot , \psi )\) and the design density of \(\mathbf{X}\). It is worth noticing that the ergodic framework covers and completes various situations compared to the mixing case and is more convenient to use in practice, in this sense our work extends the already existing research in the literature. We have illustrated how to use our results to construct the confidence set for the mode \({\varvec{\Theta }}\). In a future research one could consider the same estimation problem for stationary and ergodic discrete time processes in the case of censored data. It will be of interest to relax the stationarity to the local stationarity and establish similar results to those presented in this work, which requires a different mathematical methodology than the one used in this document. We leave this problem open for further investigation.
4 Proofs
This section is devoted to the proofs of our results. The previously defined notation continues to be used in what follows.
From the definition of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) in (1.2) and \({\varvec{\Theta }}\), we have
Consider the following decomposition
where
and
The following simple lemmas will play an instrumental role in the sequel.
Lemma 4.1
Let \((Z_n)_{n\ge 1} \) be a sequence of real martingale differences with respect to the sequence of \(\sigma -\)fields \((\mathcal {F}_n= \sigma (Z_1,\ldots ,Z_n))_{n\ge 1}\), where is the \(\sigma \)-field generated by the random variables \(Z_1,\ldots ,Z_n\). Set
For any \(p \ge 2\) and any \(n\ge 1\), assume that there exist some nonnegative constants C and \(d_n\) such that
Then, for any \(\epsilon >0\), we have
where
Lemma 4.2
Let \(\Lambda \times \Lambda ^{'}\) be an index set and for each \((\eta ,\eta ^{'}) \in \Lambda \times \Lambda ^{'}\), let \(\{ Z_i(\eta ,\eta ^{'}), i \ge 1\}\), be a sequence of a martingale difference such that \(\left| Z_i(\eta ,\eta ^{'})\right| \le B\) a.s. then, for all \(\epsilon >0\) and all sufficiently large n, we have
The following proposition describes the almost sure consistency of \(\widehat{m}_\mathrm{T}(\mathbf{x},\psi )\) with rate.
Proposition 4.3
Under assumptions (A.1)–(A.4) and (A.6), we have
4.1 Proof of Proposition 4.3.
Making use of conditions (A.2) and (A.3), we infer readily that
Lemma 4.4
Didi and Louani [26] Let \((\mathbf{X}_\mathrm{t})_{\mathrm{t}\ge 0}\) be a strictly stationary and ergodic process, under (A.1) and (A.4), we have then
4.2 Proof of Lemma 4.4
Notice that we have the following decomposition
where
Two terms to be investigated, we first take a closer look to the the second term \( F_{2,T}(\mathbf{x})\). We have
Taylor expansion of \(f^{ \mathcal {F}_{\mathrm{t}-\delta }}(\mathbf{x} - h_\mathrm{T}{} \mathbf{r})\) in neighbourhood of \(\mathbf{x}\) with assumption (A.4)(i), yields
where \(\mathbf{x}^*\) is between \(\mathbf{x}\) and \(\mathbf{x}-a_\mathrm{T}{} \mathbf{r}\). It follows from assumption (A.4)(i) that
Making use of assumptions (A.1)(iii) and (A.4)(ii), it follows that
Now, we will focus on the first term of decomposition (4.9), \(G_{1,T}(\mathbf{x})\), it is clear that
where
and
We observe that the sequence \(\{Z_{\mathrm{T},k}(\mathbf{x})\}, k=1,\ldots ,n,\) is a sequence of martingale differences with respect to the \(\sigma -\)field
Under assumption (A.1), the kernel \(K(\cdot )\) is a compactly supported probability function, then we obtain
where
Now, for any \(\epsilon>\), making use of Lemma 4.2 we obtain
The right-hand side of the last inequality is the general term of a convergent series, hence, for sufficiently large T we conclude by Borel-Cantelli lemma that
which means that
The proof is achieved by combining the statements (4.10) and (4.11). \(\square \)
The following lemma gives the rate of convergence of \(f_\mathrm{T}(\mathbf{x})\) over a compact set \(\mathfrak {C} \).
Lemma 4.5
Didi and Louani [26] Under assumption (A.1), we have
4.3 Proof of Lemma 4.5
We refer to the Theorem 1 of Didi and Louani [26]. As in the proof of (4.11) in Lemma 4.4 we obtain the result by using Lemma 4.1 instead of Lemma 4.2. \(\square \)
In order to complete the proof of Proposition 4.3, we will will show Lemma 4.6 and Lemma 4.7 given hereafter.
Lemma 4.6
If hypothesis (A.1)(i), (A.3), (A.4)(i), (A.6)(ii)-(iii) are fulfilled, we have
4.4 Proof of Lemma 4.6
For \(k=1,\ldots ,l\), let \(\mathbf{x}_k, \in \mathfrak {C}\). Consider a covering of the compact set \(\mathfrak {C}\) by a finite number l of spheres \(\mathcal {S}_k\) centered upon by \(\mathbf{x}_k\), with radius
we have that
Then we have
Making use of the Cauchy-Schwarz inequality together with assumption (A.1)(i), (A.3), (A.6)(iv) and Lemma 4.4, we readily obtain
Considering the right hand side of statement (4.14) together with the fact that \(\mathbb {E}\left[ \psi ^2\left( \mathbf{Y}_0\right) \right] < \infty \), we obtain for
that
Making use of similar arguments as those used for \(\Psi _{1,T}(\mathbf{x}, \mathbf{x}_k)\), we infer that
Using the fact in (2.5), we get
This implies that we have
Now we deal with \(\Psi _{2,T}(\mathbf{x}, y, s_k)\). Observe that
where
where
We observe that the sequence \(\big \{ R_{\mathrm{T},j}(\mathbf{x}_k)\big \}_{0\le j \le n}\) is a sequence of martingale differences adapted to the filtration
For \(p\ge 2\), making use of Jensen and Minkowski’s inequalities, we get
Furthermore, by assumption (A.6)(iii), we get
where \(\eta (\mathbf{x})\) is a constant. We infer from condition (A.4)(i) that
Notice that the Eq. (4.17) can be rewritten using Eq. (4.18) as follows
where \(C=2\left\| K \right\| \) and
Let
An application of Lemma 4.1 and keeping in mind that \(\epsilon _\mathrm{T}=\epsilon _0\left( \log T/Th_\mathrm{T}^{d}\right) ^{1/2} \), we get, for any \(\epsilon _0>0\),
where \(C_1\) is a positive constant. The right-hand side of the previous inequality, is the general term of convergent series, hence for T large enough, we obtain the following statement via the Borel-Cantelli lemma
This, in turn, implies that
By combining (4.15), (4.16) and (4.20) with Lemma 4.4, we obtain that,
Therefore the proof is complete. \(\square \)
We next evaluate the term \(B_\mathrm{T}(\mathbf{x},\psi )\) defined in (4.5).
Lemma 4.7
Under assumptions (A.1) and (A.6)(i)–(ii), we have
4.5 Proof of Lemma 4.7
First, we will use the notation
We let
Observe that assumption (A.6)(i) implies that
Under assumption (A.6)(ii), we have
We obtain that
The proof of the lemma is therefore completed. \(\square \)
Recalling (4.21), the proof of Theorem 4.3 is completed by combining Lemmas 4.4, 4.5 and 4.7. \(\square \)
In the following lemma, we give the almost sure convergence of \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\).
Lemma 4.8
Under the hypotheses of Theorem (2.4), we have, as \(T\rightarrow \infty \),
4.6 Proof of Lemma 4.8
The uniqueness hypothesis of the conditional mode of the regression gives
Combining conditions (4.24) and (4.1), we obtain, for any fixed \(\mathbf{x} \in \mathfrak {C}\) all \(\epsilon >0\), that there exists a \(\xi >0\) such that
Which gives the desired result provided that the right-hand side of Eq. (4.25) converges almost surely to zero. The proof is therefore completed by using Proposition 4.3. \(\square \)
The following lemma gives the uniform convergence of \(\widehat{m}^{(2)}_\mathrm{T}(\mathbf{x},\psi )\) over the compact set \(\mathfrak {C}\). To simplify our reasoning, from now on, all our will be given in the univariate setting. The extension to the multivariate setting follows easily.
Lemma 4.9
If assumptions (A.1)(ii), (A.3), (A.4)(i), (A.5), (A.6)(i) and (A.7)(i) are fulfilled, we have, as \(T\rightarrow \infty \),
4.7 Proof of Lemma 4.9
We first observe that we have
Let us define
Consider the following decomposition
To achieve the asymptotic uniform convergence over the compact set \(\mathfrak {C}\) of the term \(A_{\mathrm{T},1}(\mathbf{x},\psi )\) in the decomposition (4.27), we have to prove that
Using a simple integration par parts and Lemma 4.6, we obtain proof of (4.28)–(4.30), and combining Assumptions (A.1), (A.3), (A.4), (A.6)(i)–(ii) and statements (4.28)–(4.30) we obtain
Remark that
We now treat the second term \(A_{\mathrm{T},2}(\mathbf{x},\psi )\) in (4.27). We have
To achieve the asymptotic uniform convergence over the compact set \(\mathfrak {C}\) of the term \(A_{\mathrm{T},2}(\mathbf{x},\psi )\), we have to show the following statements
Observe that statement (4.33) may be rewritten as follows
The desired result can be obtained in similar way to the statement (4.23). We have
On the other hand, under assumption (A.6)(i), we have
By integrating by parts we infer that
By integrating by parts and the change of variable \(\mathbf{y} =\frac{\mathbf{x}-\mathbf{u}}{h_\mathrm{T}}\) combined with Taylor expansions of order one, under assumptions (A.4)(i), (A.5) and (A.7)(i), we readily obtain
where
is a stationary and ergodic process. Therefore, one have (see Krengel [47], Theorem 4.4),
where
By combining the statements (4.37) and (4.38) we conclude proof of (4.34). Moreover, statement (4.35) may be proved in the same way as in statement (4.34), keeping in mind that
By applying integration by parts twice, we obtain (4.34). Combining statement (4.33), (4.34) and (4.35), yields to
Statements (4.31) and (4.39) complete the proof of Lemma 4.9. \(\square \)
4.8 Proof of Theorem 2.4
Under assumption (A.3)(ii) and using Taylor expansion of \(m({\varvec{\Theta }}_\mathrm{T},\psi )\) around \(\Theta \) we obtain
where \({\varvec{\Theta }}_\mathrm{T}^\star \) is between \({\varvec{\Theta }}_\mathrm{T}\) and \({\varvec{\Theta }}\), it follows from equations (4.1) and (4.40) that
Using Lemma 4.8 and condition (A.7)(ii), one obtains
Therefore,
which is enough, while considering Proposition 4.3, to complete the proof. \(\square \)
4.9 Proof of Theorem 2.5
By using formula (2.4), we readily obtain
where \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}^\star \) is a random variable taking his values between \({\varvec{\Theta }}\) and \( \widehat{{{\varvec{\Theta }} }}_\mathrm{T}\). From the hypothesis made on \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}\) it results that \(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}^\star \) also converges a.s. towards \({\varvec{\Theta }}\). The continuity of function \(m^{(2)}(\cdot ,\psi )\) leads
For T large enough, we have almost surely
The uniform convergence in probability \( \widehat{m}_\mathrm{T}^{(2)}(\cdot ,\psi )\) to \(m^{(2)}(\cdot ,\psi )\) over \(\mathfrak {C}\) implies the convergence of the sequence \( \widehat{m}_\mathrm{T}^{(2)}(\widehat{{{\varvec{\Theta }} }}_\mathrm{T}^\star ,\psi )\) in probability to the non-null real \(m^{(2)}({\varvec{\Theta }},\psi )\). The conclusion results from the asymptotic normality of \(m_\mathrm{T}^{(1)}({\varvec{\Theta }},\psi )\). Since
almost surely and uniformly on the set \(\mathfrak {C}\), refer for details [26]. Notice that we have
where
and
We will make use of the following additional notation
where
Observe that
Lemma 4.10, stated below, will play an instrumental role in the proof of Theorem 2.5.
Lemma 4.10
Under assumptions (A.1), (A.3)(i)–(ii), (A.4) and (A.6), as \(n \rightarrow \infty \), we have
4.10 Proof of Lemma 4.10
It is easily seen that \((W_{i}(\mathbf{x},\psi ))_{1\le i\le n}\) is a sequence of martingale differences with respect to the sequence of \(\sigma -\)fields \((\mathcal {F}_{i-1})_{1\le i\le n}\). Therefore, we have to check the following two conditions.
-
(a)
$$\begin{aligned} \sum _{i=1}^n \mathbb E\left[ \Delta _{i}^2(\mathbf{x},\psi )| \mathcal {F}_{ i-2}\right] \overset{\mathbb {P}}{\rightarrow } \sigma ^2( \mathbf{x},\psi ); \end{aligned}$$
-
(b)
$$\begin{aligned} n \mathbb E\left[ \Delta _{i}^2(\mathbf{x},\psi ) \mathbb {1}_{\{|\Delta _{i}(\mathbf{x},\psi )|> \epsilon \}}\right] = o(1)~~\text{ holds, } \text{ for } \text{ any } ~~\epsilon >0. \end{aligned}$$
These conditions are necessary to establish the asymptotic normality related to discrete time martingale difference sequences (see, for instance, Hall and Heyde [37]).
4.11 Proof of (a)
First, observe that
Note that, for \(T_{i-1}\le t\le T_i\), we have \(\mathcal {S}_{\mathrm{T}-\delta ,\delta }\subset \mathcal {F}_{i-2}\). Therefore, making use of condition (A.6)(i), we obtain
Using Taylor’s formula combined with assumption (A.4), we obtain
where
is a stationary and ergodic process. So the sum \( \frac{1}{n} \sum _{i=1}^n g_i^{\mathcal {F}_{i-2} } (\mathbf{x})\) has a finite limit, (see Krengel [47, Theorem 4.4]), which is
Moreover, observe that by assumptions (A.3), we have
Using Jensen inequality and Assumption (A.6)(iii), we obtain
According to Assumption (A.6)(iii), the function \(\Phi _2(\cdot ,\psi )\) is continuous in the neighbourhood of \(\mathbf{x}\), we have
By a first order Taylor expansion of the function \(f_{\mathrm{T},T{i-2}}\), for \(\mathbf{x}^*\) in \([\mathbf{x}-h_\mathrm{T}{} \mathbf{v},\mathbf{x}]\), we obtain
It is clear, whenever \(\delta \) is small enough, that the quantities
may be approximated by
Consequently, using the ergodic and stationarity properties of the process \((\mathbf{X}_\mathrm{t})_{\mathrm{T} \ge 0}\), it follows that
It follows that
This implies that
4.12 Proof of (b)
Using the inequalities of Holder, Markov, Jensen and Minkowski’s inequalities, together with Assumption (A.6)(iii), we obtain for all \(\epsilon >0\) and all p and q, such that
that
By a first order Taylor’s expansion, we have
Combining statements (4.45) and (4.46), we obtain
Lemmas 4.9 and 4.10 combined with Theorem 2.4 complete the proof of Theorem 2.5.\(\square \)
Notes
The definition is provided in the “Appendix”.
References
Andrews, D.W.K.: Nonstrong mixing autoregressive processes. J. Appl. Probab. 21(4), 930–934 (1984)
Beck, A.: On the strong law of large numbers. In: Ergodic Theory (Proc. Internat. Sympos., Tulane Univ., New Orleans, La., 1961), pp. 21–53. Academic Press, New York (1963)
Benrabah, O., Ould Saïd, E., Tatachak, A.: A kernel mode estimate under random left truncation and time series model: asymptotic normality. Stat. Pap. 56(3), 887–910 (2015)
Beran, J.: Statistics for Long-Memory Processes. Volume 61 of Monographs on Statistics and Applied Probability. Chapman and Hall, New York (1994)
Berlinet, A., Gannoun, A., Matzner-Løber, E.: Normalité asymptotique d’estimateurs convergents du mode conditionnel. Can. J. Stat. 26(2), 365–380 (1998)
Blanke, D., Pumo, B.: Optimal sampling for density estimation in continuous time. J. Time Ser. Anal. 24(1), 1–23 (2003)
Bosq, D.: Nonparametric Statistics for Stochastic Processes. Estimation and Prediction. Volume 110 of Lecture Notes in Statistics. Springer, New York (1998)
Bouzebda, S., Chaouch, M., Laïb, N.: Limiting law results for a class of conditional mode estimates for functional stationary ergodic data. Math. Methods Stat. 25(3), 168–195 (2016)
Bouzebda, S., Didi, S.: Multivariate wavelet density and regression estimators for stationary and ergodic discrete time processes: asymptotic results. Commun. Stat. Theory Methods 46(3), 1367–1406 (2017a)
Bouzebda, S., Didi, S.: Additive regression model for stationary and ergodic continuous time processes. Commun. Stat. Theory Methods 46(5), 2454–2493 (2017b)
Bouzebda, S., Didi, S.: Some results about kernel estimators for function derivatives based on stationary and ergodic continuous time processes with applications. Commun. Stat. Theory Methods (2020). (to appear)
Bouzebda, S., Didi, S., El Hajj, L.: Multivariate wavelet density and regression estimators for stationary and ergodic continuous time processes: asymptotic results. Math. Methods Stat. 24(3), 163–199 (2015)
Bradley, R.C.: Introduction to Strong Mixing Conditions, vol. 1. Kendrick Press, Heber City (2007)
Chacón, J.E.: The modal age of statistics (2018). arXiv:1807.02789
Chacón, J.E., Montanero, J., Nogales, A.G.: A note on kernel density estimation at a parametric rate. J. Nonparametric Stat. 19(1), 13–21 (2007)
Chen, Y.-C.: Modal regression using kernel density estimation: a review. Wiley Interdiscip. Rev. Comput. Stat. 10(4), e1431 (2018)
Chen, Y.-C., Genovese, C.R., Tibshirani, R.J., Wasserman, L.: Nonparametric modal regression. Ann. Stat. 44(2), 489–514 (2016)
Collomb, G., Härdle, W., Hassani, S.: A note on prediction via estimation of the conditional mode function. J. Stat. Plan. Inference 15(2), 227–236 (1987)
Comte, F., Merlevède, F.: Adaptive estimation of the stationary density of discrete and continuous time mixing processes. ESAIM Probab. Stat. 6, 211–238 (2002). (New directions in time series analysis (Luminy, 2001))
Debbarh, M.: Normalité asymptotique de l’estimateur par ondelettes des composantes d’un modèle additif de régression. C. R. Math. Acad. Sci. Paris 343(9), 601–606 (2006)
Delecroix, M.: Sur l’estimation et la prévision non-paramétrique des processus ergodiques. Université des sciences de Lille, Flandre-Artois, Doctorat d’État (1987)
Devroye, L.: A Course in Density Estimation. Volume 14 of Progress in Probability and Statistics. Birkhäuser Boston Inc., Boston (1987)
Devroye, L., Györfi, L.: Nonparametric Density Estimation. Wiley Series in Probability and Mathematical Statistics: Tracts on Probability and Statistics. Wiley, New York (1985). (The\({{\mathbf{L}}_{1}}\)view)
Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer Series in Statistics. Springer, New York (2001)
Didi, S.: Quelques propriétés asymptotiques en estimation non paramétrique de fonctionnelles de processus stationnaires en temps continu. Ph.D. thesis. Thèse de doctorat de Statistique, Paris 6 2014 (2014)
Didi, S., Louani, D.: Consistency results for the kernel density estimate on continuous time stationary and dependent data. Stat. Probab. Lett. 83(4), 1262–1270 (2013)
Didi, S., Louani, D.: Asymptotic results for the regression function estimate on continuous time stationary and ergodic data. Stat. Risk Model. 31(2), 129–150 (2014)
Eddy, W.F.: Optimum kernel estimators of the mode. Ann. Stat. 8(4), 870–882 (1980)
Eddy, W.F.: The asymptotic distributions of kernel estimators of the mode. Z. Wahrsch. Verw. Gebiete 59(3), 279–290 (1982)
Eggermont, P.P.B., LaRiccia, V.N.: Maximum Penalized Likelihood Estimation. Density Estimation. Springer Series in Statistics, vol. I. Springer, New York (2001)
Einbeck, J., Tutz, G.: Modelling beyond regression functions: an application of multimodal regression to speed-flow data. J. R. Stat. Soc. Ser. C (Appl. Stat.) 55(4), 461–475 (2006)
Ezzahrioui, M., Ould-Saïd, E.: Asymptotic normality of a nonparametric estimator of the conditional mode function for functional data. J. Nonparametric Stat. 20(1), 3–18 (2008)
Feng, Y., Fan, J., Suykens, J.A.K.: A statistical learning approach to modal regression (2017). arXiv:1702.05960
Ferraty, F., Laksaci, A., Vieu, P.: Functional time series prediction via conditional mode estimation. C. R. Math. Acad. Sci. Paris 340(5), 389–392 (2005)
Grund, B., Hall, P.: On the minimisation of \(L^p\) error in mode estimation. Ann. Stat. 23(6), 2264–2284 (1995)
Györfi, L., Härdle, W., Sarda, P., Vieu, P.: Nonparametric Curve Estimation from Time Series. Lecture Notes in Statistics, vol. 60. Springer, Berlin (1989)
Hall, P., Heyde, C.C.: Martingale Limit Theory and Its Application. Probability and Mathematical Statistics. Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New York-London (1980)
Härdle, W.: Applied Nonparametric Regression. Volume 19 of Econometric Society Monographs. Cambridge University Press, Cambridge (1990)
Härdle, W., Marron, J.S.: Optimal bandwidth selection in nonparametric regression function estimation. Ann. Stat. 13(4), 1465–1481 (1985)
Harel, M., Puri, M.L.: Conditional \(U\)-statistics for dependent random variables. J. Multivariate Anal. 57(1), 84–100 (1996)
Herrmann, E., Ziegler, K.: Rates on consistency for nonparametric estimation of the mode in absence of smoothness assumptions. Stat. Probab. Lett. 68(4), 359–368 (2004)
Jones, M.C.: On higher order kernels. J. Nonparametric Stat. 5(2), 215–221 (1995)
Jones, M.C., Signorini, D.F.: A comparison of higher-order bias kernel density estimators. J. Am. Stat. Assoc. 92(439), 1063–1073 (1997)
Jones, M.C., Linton, O., Nielsen, J.P.: A simple bias reduction method for density estimation. Biometrika 82(2), 327–338 (1995)
Kemp, G.C.R., Santos Silva, J.M.C.: Regression towards the mode. J. Econom. 170(1), 92–101 (2012)
Krebs, J.T.N.: The bootstrap in kernel regression for stationary ergodic data when both response and predictor are functions. J. Multivar. Anal. 173, 620–639 (2019)
Krengel, U.: Ergodic Theorems. Volume 6 of de Gruyter Studies in Mathematics. Walter de Gruyter & Co., Berlin (1985). (With a supplement by Antoine Brunel)
Laib, N., Louani, D.: Nonparametric kernel regression estimation for functional stationary ergodic data: asymptotic properties. J. Multivar. Anal. 101(10), 2266–2281 (2010)
Laib, N., Louani, D.: Rates of strong consistencies of the regression function estimator for functional stationary ergodic data. J. Stat. Plan. Inference 141(1), 359–372 (2011)
Lee, M.-J.: Mode regression. J. Econom. 42(3), 337–349 (1989)
Lee, M.-J.: Quadratic mode regression. J. Econom. 57(1–3), 1–19 (1993)
Leucht, A., Neumann, M.H.: Degenerate \(U\)- and \(V\)-statistics under ergodicity: asymptotics, bootstrap and applications in statistics. Ann. Inst. Stat. Math. 65, 349–386 (2013)
Lu, Z.: Analyse des processus longue mémoire stationnaires et non-stationnaires : estimations, applications et prévisions. Ph.D. thesis. Thèse de doctorat de Mathématiques, Mathématiques financières et statistiques appliquées, Cachan, Ecole normale supérieure 2009 (2009)
Maslowski, B., Pospíšil, J.: Ergodicity and parameter estimates for infinite-dimensional fractional Ornstein-Uhlenbeck process. Appl. Math. Optim. 57(3), 401–429 (2008)
Masry, E.: Nonparametric regression estimation for dependent functional data: asymptotic normality. Stoch. Process. Appl. 115(1), 155–177 (2005)
Masry, E.: Probability density estimation from sampled data. IEEE Trans. Inf. Theory 29(5), 696–709 (1983)
Nadaraya, È.A.: Nonparametric Estimation of Probability Densities and Regression Curves. Volume 20 of Mathematics and its Applications (Soviet Series). Kluwer Academic Publishers Group, Dordrecht (1989). (Translated from the Russian by Samuel Kotz)
Neumann, M.H.: Absolute regularity and ergodicity of Poisson count processes. Bernoulli 17(4), 1268–1284 (2011)
Ould-Saïd, E.: A note on ergodic processes prediction via estimation of the conditional mode function. Scand. J. Stat. 24(2), 231–239 (1997)
Ota, H., Kato, K., Hara, S.: Quantile regression approach to conditional mode estimation. Electron. J. Stat. 13(2), 3120–3160 (2019)
Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)
Prakasa Rao, B.L.S.: Statistical Inference for Diffusion Type Processes, Volume 8 of Kendall’s Library of Statistics. Edward Arnold, London; Oxford University Press, New York (1999)
Prakasa Rao, B.L.S.: Nonparametric density estimation for stochastic processes from sampled data. Publ. Inst. Statist. Univ. Paris, 35(3), 51–83 (1991) (1990)
Quintela-Del-Río, A., Vieu, P.: A nonparametric conditional mode estimate. J. Nonparametric Stat. 8(3), 253–266 (1997)
Romano, J.P.: Bootstrapping the mode. Ann. Inst. Stat. Math. 40(3), 565–586 (1988a)
Romano, J.P.: On weak convergence and optimality of kernel density estimates of the mode. Ann. Stat. 16(2), 629–647 (1988b)
Samanta, M., Thavaneswaran, A.: Nonparametric estimation of the conditional mode. Commun. Stat. Theory Methods 19(12), 4515–4524 (1990)
Sasaki, H., Ono, Y., Sugiyama, M.: Modal regression via direct log-density derivative estimation. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) Neural Information Processing, pp. 108–116. Springer, Cham (2016)
Sinotina, T., Vogel, S.: Universal confidence sets for the mode of a regression function. IMA J. Manag. Math. 23(4), 309–323 (2012)
Stute, W.: Conditional \(U\)-statistics. Ann. Probab. 19(2), 812–825 (1991)
Tapia, R.A., Thompson, J.R.: Nonparametric Probability Density Estimation. Volume 1 of Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore (1978)
Tarter, M.E., Lock, M.D.: Model-Free Curve Estimation. Volume 56 of Monographs on Statistics and Applied Probability. Chapman & Hall, New York (1993)
Wand, M.P., Jones, M.C.: Kernel Smoothing. Volume 60 of Monographs on Statistics and Applied Probability. Chapman and Hall Ltd., London (1995)
Wertz, W.: Statistical Density Estimation: A Survey, Volume 13 of Angewandte Statistik und Ökonometrie [Applied Statistics and Econometrics]. Vandenhoeck & Ruprecht, Göttingen. With German and French Summaries (1978)
Yao, W., Li, L.: A new regression model: modal linear regression. Scand. J. Stat. 41(3), 656–671 (2014)
Youndjé, E.: Estimation non paramétrique de la densité conditionnelle par la méthode du noyau. Ph.D. thesis. Thèse de doctorat de Sciences et techniques communes Rouen 1993 (1993)
Ziegler, K.: On bootstrapping the mode in the nonparametric regression model with random design. Metrika 53(2), 141–170 (2001)
Ziegler, K.: On nonparametric kernel estimation of the mode of the regression function in the random design model. J. Nonparametric Stat. 14(6), 749–774 (2002)
Ziegler, K.: On the asymptotic normality of kernel regression estimators of the mode in the nonparametric random design model. J. Stat. Plan. Inference 115(1), 123–144 (2003)
Acknowledgements
The authors would like to thank the two referees for their constructive suggestions that have led to improve the presentation of this paper. The authors would like to express their appreciation for everybody fighting against the COVID-19 pandemic.
Author information
Authors and Affiliations
Contributions
SD, SB: conceptualization, methodology, investigation, writing—original draft, writing—review and Editing. Both authors contributed equally to this work.
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
For the sake of clarity, introduce some details defining the ergodic property of continuous time processes. Let \(\mathbf{Z}=\{\mathbf{Z}_t\}_{t\ge 0}\) be a continuous time process taking values in some measurable space \((E,\mathcal {\mathbf{Z}})\) on which is defined a probability measure \(\mu \). For \(\delta \ge 0\), let \(\Upsilon ^\delta \) be a \(\delta \)-shift transformation, i.e., \((\Upsilon ^\delta (x))_s=x_{s+\delta }\). A measurable set \(\mathcal {A}\) is \(\delta \)-invariant if it does not change under \(\delta \)-shift transformation , i.e., \(\Upsilon ^\delta (\mathcal {A})= \mathcal {A}\).
Definition 5.1
(\(\delta \)-ergodicity) A continuous time process \(\mathbf{Z}=\{\mathbf{Z}_t\}_{t\ge 0}\) is \(\delta \)-ergodic if every measurable \(\delta \)-invariant set related to the process \(\mathbf{Z}\) has probability either 1 or 0, in other words, for any \(\delta \)-invariant set \(\mathcal {A}\), \(\mu (\mathcal {A})=(\mu (\mathcal {A}))^2\).
This definition means that if we take the process X and slice it into time blocks of length \(\delta \) then the new discrete time process \((\mathbf{Z}_0^\delta ,\mathbf{Z}_\delta ^{2\delta },\mathbf{Z}_{2\delta }^{3\delta },\mathbf{Z}_{3\delta }^{4\delta },\ldots )\) is ergodic. For discrete time processes, we refer, for instance, to Krengel [47] for the definition and details on the ergodic property.
Definition 5.2
(Ergodicity) A continuous time process \(\mathbf{Z}=\{\mathbf{Z}_t\}_{t\ge 0}\) is ergodic if it is \(\delta \)-ergodic for every \(\delta >0\).
It is well known from the ergodic theorem that, for a measurable function g and a stationary ergodic process \(\mathbf{Z}=\{\mathbf{Z}_t\}_{t\ge 0}\), we have
We refer to the book of Krengel [47] for an account of details and results on the ergodic theory.
Rights and permissions
About this article
Cite this article
Bouzebda, S., Didi, S. Some asymptotic properties of kernel regression estimators of the mode for stationary and ergodic continuous time processes. Rev Mat Complut 34, 811–852 (2021). https://doi.org/10.1007/s13163-020-00368-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13163-020-00368-6
Keywords
- Conditional density
- Conditional mode
- Kernel estimate
- Nadaraya–Watson estimators
- Ergodic processes
- Martingale difference arrays
- Prediction
- Continuous time processes
- Confidence regions
- Rate of convergence
- Strong consistency