Abstract
Important information concerning a multivariate data set, such as modal regions, is contained in the derivatives of the probability density or regression functions. Despite this importance, nonparametric estimation of higher order derivatives of the density or regression functions have received only relatively scant attention. The main purpose of the present work is to investigate general recursive kernel type estimators of function derivatives. We establish the central limit theorem for the proposed estimators. We discuss the optimal choice of the bandwidth by using the plug in methods. We obtain also the pointwise MDP of these estimators. Finally, we investigate the performance of the methodology for small samples through a short simulation study.
Similar content being viewed by others
References
Allaoui, S., Bouzebda, S., Chesneau, C. and Liu, J. (2021). Uniform almost sure convergence and asymptotic distribution of the wavelet-based estimators of partial derivatives of multivariate density function under weak dependence. J. Nonparametr. Stat. 33, 170–196.
Altman, N. and Leger, C. (1995). Bandwidth selection for kernel distribution function estimation. J. Statist. Plann. Inference 46, 195–214.
Bojanic, R. and Seneta, E. (1973). A unified theory of regularly varying sequences. Math. Z. 134, 91–106.
Bouzebda, S. (2012). On the strong approximation of bootstrapped empirical copula processes with applications. Math. Methods Statist. 21, 153–188.
Bouzebda, S. and Didi, S. (2020). Some results about kernel estimators for function derivatives based on stationary and ergodic continuous time processes with applications. Communications in Statistics - Theory and Methods, 1–48.
Bouzebda, S. and El-hadjali, T. (2020). Uniform convergence rate of the kernel regression estimator adaptive to intrinsic dimension in presence of censored data. J. Nonparametr. Stat. 32, 864–914.
Bouzebda, S. and Elhattab, I. (2011). Uniform-in-bandwidth consistency for kernel-type estimators of Shannon’s entropy. Electron. J. Stat. 5, 440–459.
Bouzebda, S., Elhattab, I. and Seck, C. T. (2018). Uniform in bandwidth consistency of nonparametric regression based on copula representation. Statist. Probab. Lett. 137, 173–182.
Bouzebda, S., Elhattab, I. and Nemouchi, B. (2021). On the uniform-in-bandwidth consistency of the general conditional U-statistics based on the copula representation. J. Nonparametr. Stat. 33, 321–358.
Bouzebda, S. and Nemouchi, B. (2020). Uniform consistency and uniform in bandwidth consistency for nonparametric regression estimates and conditional U-statistics involving functional data. J. Nonparametr. Stat. 32, 452–509.
Charnigo, R., Hall, B. and Srinivasan, C. (2011). A generalized cp criterion for derivative estimation. Technometrics 53, 238–253.
Deheuvels, P. (1973). Sur l’estimation séquentielle de la densité. C.R. Acad. Sci. Paris Ser. A-B 276, 1119–1121.
Deheuvels, P. (2011). One bootstrap suffices to generate sharp uniform bounds in functional estimation. Kybernetika (Prague) 47, 855–865.
Delaigle, A. and Gijbels, I. (2004). Practical bandwidth selection in deconvolution kernel density estimation. Comput. Statist. Data Anal. 45, 249–267.
Devroye, L. (1987). A course in density estimation, volume 14 of Progr. Probab. Statist. Birkhäuser Boston Inc., Boston.
Devroye, L. and Györfi, L. (1985). Nonparametric density estimation. Wiley Series in Probability and Mathematical Statistics: Tracts on Probability and Statistics. Wiley, New York. The L1 view.
Devroye, L. and Lugosi, G. (2001). Combinatorial methods in density estimation Springer Series in Statistics. Springer, New York.
Dobrovidov, A. V. and Rud’ ko, I. M. (2010). Choice of bandwidth in the kernel function in the nonparametric estimation of the derivative of density by the smoothed cross-validation method. Avtomat. i Telemekh., (2), 42–58.
Duflo, M. (1997). Random iterative models. Collection Applications of Mathematics. Springer, Berlin.
Eggermont, P. P. B. and LaRiccia, V. N. (2001). Maximum penalized likelihood estimation. Vol. I Springer Series in Statistics. Springer, New York. Density estimation.
Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab.13, 1–37.
Einmahl, U. and Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Ann. Statist. 94, 236–238.
Eubank, R. L. and Speckman, P. L. (1993). Confidence bands in nonparametric regression. J. Amer. Statist. Assoc. 88, 1287–1301.
Galambos, J. and Seneta, E. (1973). Regularly varying sequences. J. Amer. Math. Soc. 41, 110–116.
Genovese, C. R., Perone-Pacifico, M., Verdinelli, I. and Wasserman, L. A. (2013). Nonparametric inference for density modes. CoRR, arXiv:1312.7567.
Härdle, W. (1990). Applied nonparametric regression volume 19 of Econometric Society Monographs. Cambridge University Press, Cambridge.
Mokkadem, A., Pelletier, M. and Thiam, B. (2006). Large and moderate deviations principles for recursive kernel estimator of a multivariate density and its partial derivatives. Serdica Math. J. 32, 323–354.
Mokkadem, A. and Pelletier, M. (2007). A companion for the Kiefer-Wolfowitz-Blum stochastic approximation algorithm. Ann. Statist. 35, 1749–1772.
Mokkadem, A., Pelletier, M. and Slaoui, Y (2009a). The stochastic approximation method for the estimation of a multivariate probability density. J. Statist. Plann. Inference 139, 2459–2478.
Mokkadem, A., Pelletier, M. and Slaoui, Y (2009b). Revisiting Révész’s stochastic approximation method for the estimation of a regression function. ALEA Lat. Am. J. Probab. Math. Stat. 6, 63–114.
Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and its Applications 10, 186–190.
Nadaraya, È. A. (1989). Nonparametric estimation of probability densities and regression curves volume 20 of Mathematics and its Applications (Soviet Series). Kluwer Academic Publishers Group, Dordrecht. Translated from the Russian by Samuel Kotz.
Noh, Y. -K., Sugiyama, M., Liu, S., Plessis, M. C. d., Park, F. C. and Lee, D. D. (2018). Bias reduction and metric learning for nearest-neighbor estimation of kullback-leibler divergence. Neural Computation, 30(7), 1930–1960. PMID 29902113.
Park, C. and Kang, K. -H. (2008). SiZer analysis for the comparison of regression curves. Comput. Statist. Data Anal. 52, 3954–3970.
Puhalskii, A. (1994). The method of stochastic exponentials for large deviations. Stochastic Process. Appl. 54, 45–70.
Ramsay, J. O. and Silverman, B. W. (2002). Applied functional data analysis. Springer Series in Statistics. Springer, New York. Methods and case studies.
Ramsay, J. O. and Silverman, B. W. (2005). Functional data analysis, 2nd edition. Springer Series in Statistics. Springer, New York.
Révész, P. (1973). Robbins-monro procedure in a Hilbert space and its application in the theory of learning processes. I. Studia Sci. Math. Hungar. 8, 391–398.
Révész, P. (1977). How to apply the method of stochastic approximation in the non-parametric estimation of a regression function. Math. Operationsforsch. Statist. Ser. Statist 8, 119–126.
Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90, 1257–1270.
Scott, D. W. (1992). Multivariate density estimation. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Wiley, New York. Theory, practice, and visualization: A Wiley-Interscience Publication.
Silverman, B. W. (1986). Density estimation for statistics and data analysis. Chapman and Hall, London.
Singh, R. S. (1977). Applications of estimators of a density and its derivatives to certain statistical problems. J. Roy. Statist. Soc. Ser. B 39, 357–363.
Slaoui, Y. (2013). Large and moderate principles for recursive kernel density estimators defined by stochastic approximation method. Serdica Math. J. 39, 53–82.
Slaoui, Y. (2014a). Bandwidth selection for recursive kernel density estimators defined by stochastic approximation method. J. Probab. Stat., 2014, ID 739640. https://doi.org/10.1155/2014/739640.
Slaoui, Y. (2014b). The stochastic approximation method for the estimation of a distribution function. Math. Methods Statist. 23, 306–325.
Slaoui, Y. (2015a). Large and moderate deviation principles for averaged stochastic approximation method for the estimation of a regression function. Serdica Math. J. 41, 307–328.
Slaoui, Y. (2015b). Plug-in Bandwidth selector for recursive kernel regression estimators defined by stochastic approximation method. Stat. Neerl. 69, 483–509.
Slaoui, Y. (2019). Wild bootstrap bandwidth selection of recursive nonparametric relative regression for independent functional data. J. Multivariate Anal.173, 494–511.
Slaoui, Y. (2020). Recursive nonparametric regression estimation for independent functional data. Statist. Sinica. 30. https://doi.org/10.5705/ss.202018.0069.
Tapia, R. A. and Thompson, J. R. (1978). Nonparametric probability density estimation, volume 1 of Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore.
Tsybakov, A. B. (1990). Recurrent estimation of the mode of a multidimensional distribution. Probl. Inf. Transm. 8, 119–126.
Vasil’ev, V. A., Dobrovidov, A. V. and Koshkin, G.M.s (2004). Neparametricheskoe otsenivanie funktsionalov ot raspredeleniĭ statsionarnykh posledovatel’ nosteĭ. “Nauka”, Moscow.
Wand, M. P. and Jones, M. C. (1995). Kernel smoothing volume 60 of Monographs on Statistics and Applied Probability. Chapman and Hall Ltd., London.
Watson, G. S. (1964). Smooth regression analysis. Sankhya A 26, 359–372.
Wegman, E. J. and Davies, H. I. (1979). Remarks on some recursive estimators of a probability density. Ann. Statist. 7, 316–327.
Wertz, W. (1978). Statistical density estimation: a survey volume 13 of Angewandte Statistik und Ökonometrie [Applied Statistics and Econometrics]. Göttingen, Vandenhoeck & Ruprecht. With German and French summaries.
Wolverton, C. and Wagner, T. (1969). Asymptotically optimal discriminant functions for pattern classification. IEEE Trans. Inform. Theory 15, 258–265.
Yamato, H. (1971). Sequential estimation of a continuous probability density function and mode. IBull Math. Statist. 14, 1–12.
Acknowledgments
The authors are indebted to the Editor-in-Chief, Associate Editor and the referee for their very valuable comments, suggestions careful reading of the article which led to a considerable improvement of the manuscript. The authors thank Mr. Issam Elhattab for his important help with the simulation section.
Funding
This work benefited from the financial support of the GDR 3477 GeoSto.
Author information
Authors and Affiliations
Corresponding author
Additional information
Disclosure statement
No potential conflict of interest was reported by the authors.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bouzebda, S., Slaoui, Y. Nonparametric Recursive Estimation for Multivariate Derivative Functions by Stochastic Approximation Method. Sankhya A 85, 658–690 (2023). https://doi.org/10.1007/s13171-021-00272-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13171-021-00272-1