Automation and Remote Control

, Volume 62, Issue 9, pp 1453–1476

# Nonparametric Estimation of the Logarithmic Density Derivative of Sequences with Strong Mixing

• A. V. Dobrovidov
• G. M. Koshkin
Article

## Abstract

In nonparametric signal estimation, there is a need for estimating the logarithmic probability density derivatives. This problem is complex, because the logarithmic density derivative is a function with singularity—a ratio containing density in the denominator. Since the density estimate can take values close or even equal to zero, the estimate of the logarithmic derivative becomes unstable. This difficulty is surmounted by constructing a new nonparametric estimate for the logarithmic derivative, i.e., an estimate that is stable to observation and based piecewise-smooth approximation. Its properties for dependent observations generated by stationary processes satisfying the strong mixing condition are studied. The rate of convergence of the nonparametric estimate and the principal part of the expansion of the mean-square estimate error are determined.

## Keywords

Probability Density Estimate Error Stationary Process System Theory Density Estimate
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## REFERENCES

1. 1.
Roussas, G.G., On Some Properties of Nonparametric Estimates of Probability Density Functions, Bull. Soc. Math., 1968, no. 1, pp. 29-43.Google Scholar
2. 2.
Roussas, G.G., Nonparametric Estimation in Markov Processes, Ann. Inst. Statist. Math., 1969, vol. 21, no. 1, pp. 73-87.Google Scholar
3. 3.
Rosenblatt, M., Density Estimates and Markov Sequences. Nonparametric Techniques in Statistical Inference, Cambridge: Cambridge Univ. Press, 1970.Google Scholar
4. 4.
Rosenblatt, M., Curve Estimates, Ann. Math. Statist., 1971, vol. 42, no. 6, pp. 1815-1842.Google Scholar
5. 5.
Rosenblatt, M., Markov Processes: Structure and Asymptotic Behavior, Berlin: Springer-Verlag, 1971.Google Scholar
6. 6.
Koshkin, G.M. and Tarasenko, F.P., A Consistency Condition for Weakly Dependent Samples, Mat. Statist. Prilozh., 1976, issue 4, pp. 29-41.Google Scholar
7. 7.
Koshkin, G.M. and Tarasenko, F.P., Recurrent Probability Density Estimation and Regression Lines of Dependent Samples, Mat. Statist. Prilozh., 1976, issue 4, pp. 122-138.Google Scholar
8. 8.
Koshkin, G.M., An Approach to Estimation of Transition Distribution Functions and Moments for Certain Markov Processes, Mat. Statist. Prilozh., 1976, issue 4, pp. 116-121.Google Scholar
9. 9.
Delecroix, M., Central Limit Theorems for Density Estimators of a ϕ-Mixing Process, in Recent Developments in Statistics, Amsterdam: North Holland, 1977, pp. 409-414.Google Scholar
10. 10.
Masry, E., Probability Density Estimation from Sampled Data, IEEE Trans. Inf. Theory, 1983, vol. 29, no. 5, pp. 696-709.Google Scholar
11. 11.
Masry, E., Recursive Probability Density Estimation for Weakly Dependent Stationary Processes, IEEE Trans. Inf. Theory, 1986, vol. 32, no. 2, pp. 254-267.Google Scholar
12. 12.
Koshkin, G.M. and Tarasenko, F.P., Nonparametric Algorithms for Identifying and Control of Continuous-Discrete Stochastic Objects, 8th IFAC-IFORS Symp. Identif. Syst. Parameter Estimation, Beijing: Pergamon Press, 1988, vol. 2, pp. 882-887.Google Scholar
13. 13.
Castellana, J.V. and Leadbetter, M.R., On Smoothed Probability Density Estimation for Stationary Processes, Stoch. Proces. Appl., 1986, vol. 21, no. 2, pp. 179-193.Google Scholar
14. 14.
Györfi, L., Strong Consistent Density Estimate from Ergodic Samples, J. Mult. Anal., 1981, vol. 11, no. 1, pp. 81-84.Google Scholar
15. 15.
Györfi, L., Recent Results on Nonparametric Regression Estimate and Multiple Classification, Prob. Control Inf. Theory, 1981, vol. 10, no. 1, pp. 43-52.Google Scholar
16. 16.
17. 17.
Györfi, L., Hardle, W., Sarda, P., and Vieu, P., Nonparametric Curve Estimation from Time Series, Berlin: Springer, 1988.Google Scholar
18. 18.
Masry, E., Nonparametric Estimation of Conditional Probability Densities and Expectations of Stationary Processes: Strong Consistency and Rates, Stoch. Proces. Appl., 1989, vol. 32, no. 1, pp. 109-127.Google Scholar
19. 19.
Masry, E., Almost Sure Convergence of Recursive Density Estimators for Stationary Mixing Processes, Statist. Prob. Lett., 1987, vol. 5, no. 2, pp. 249-254.Google Scholar
20. 20.
Masry, E. and Györfi, L., Strong Consistency and Rates for Recursive Probability Density Estimators of Stationary Processes, J. Mult. Anal., 1987, vol. 22, no. 1, pp. 79-93.Google Scholar
21. 21.
Doukhan, P. and Ghindes, M., Estimation de la transition de probabilite d'une chain de Markov Doeblin-recurrente. Etude du eas processus autoregressif general d'ordre 1, Stoch. Proces. Appl., 1983, vol. 15, no. 3, pp. 271-293.Google Scholar
22. 22.
Robinson, P.M., Nonparametric Estimation from Time Series Residuals, Cah. Cent. Etud. Rech. Oper., 1986, vol. 28, no. 1-3, pp. 197-202.Google Scholar
23. 23.
Vasil'ev, V.A., Estimation of the Distribution of Disturbances in an Autoregression Process, Mat. Stat. Pril., 1986, issue 10, pp. 9-24.Google Scholar
24. 24.
Masry, E., Multivariate Probability Density Deconvolution for Stationary Random Processes, IEEE Trans. Inf. Theory, 1991, vol. 37, no. 4, pp. 1105-1115.Google Scholar
25. 25.
Tran, L.T., Kernel Density Estimation of Random Fields, J. Mult. Anal., 1990, vol. 34, no. 1, pp. 37-53.Google Scholar
26. 26.
Vasil'ev, V.A. and Koshkin, G.M., Estimation of Limiting Distribution Density and Its Derivative from Weakly Dependent Observations, Probl. Peredachi Inf., 1997, vol. 33, issue 2, pp. 66-80.Google Scholar
27. 27.
Koshkin, G. and Vasil'iev, V., Estimation of Multivariate Probability Density and Its Derivatives from Weakly Dependent Observations, in: Proc. Steklov Math. Inst. Seminar “Statistics and Control of Stochastic Processes,” The Liptser Festschrift, Steklov Math. Inst., Yu.M. Kabanov, B.L. Rozovskii, and A.N. Shiryaev, Eds, Singapore: World Scientific, 1997, pp. 229-241.Google Scholar
28. 28.
Vail'ev, V.A. and Koshkin, G.M., Estimation of Distribution Density Functions from Dependent Observations, Probl. Peredachi Inf., 1997, vol. 33, issue 4, pp. 45-60.Google Scholar
29. 29.
Vail'ev, V.A. and Koshkin, G.M., Nonparametric Identification of Autoregressions, Teor. Veroyatn. Primenen., 1998, vol. 43, issue 3, pp. 577-588.Google Scholar
30. 30.
Koshkin, G.M. and Vasil'iev, V.A., Nonparametric Estimation of Multivariate Density Derivatives from Dependent Observations, Math. Methods Statist., 1998, vol. 7, no. 4, pp. 361-400.Google Scholar
31. 31.
Nemirovskii, A.S. and Tsypkin, Ya.Z., Optimal Adaptive Control Algorithms, Avtom. Telemekh., 1984, no. 12, pp. 64-77.Google Scholar
32. 32.
Nadaraya, E.A., Nonparametric Estimates of Probability Density and Regression, Teor. Veroyatn. Primenen., 1965, vol. 10, issue 1, pp. 199-203.Google Scholar
33. 33.
Kushnir, A.F., Asymptotically Optimal Criteria for Hypothesis Testing Regression Problems, Teor. Veroyatn. Primenen., 1968, vol. 13, issue 4, pp. 682-700.Google Scholar
34. 34.
Dobrovidov, A.V., Asymptotically ε-Optimal Nonparametric Filtering of Stationary Sequences with Unknown Statistical Characteristics, Avtom. Telemekh., 1984, no. 12, pp. 40-49.Google Scholar
35. 35.
Dobrovidov, A.V. and Koshkin, G.M., Neparametricheskoe otsenivanie signalov (Nonparametric Estimation of Signals), Moscow: Nauka, Fizmatlit, 1997.Google Scholar
36. 36.
Cramer, H., Mathematical Methods of Statistics, Princeton: Princeton Univ. Press, 1937. Translated under the title Matematicheskie metody statistiki, Moscow: Mir, 1975.Google Scholar
37. 37.
Penskaya, M.Ya., Stable Estimation of a Function of a Parameter, in Statisticheskie metody otsenivaniya i proverki gipotez (Statistical Methods of Estimation and Testing of Hypotheses), Perm: Perm. Gos. Univ., 1990, pp. 44-55.Google Scholar
38. 38.
Koshkin, G.M., Moments of Deviations of the Estimates of a Substitution and Its Piecewise-Smooth Approximations, Sib. Mat. Zh., 1999, vol. 40, no. 3, pp. 601-614.Google Scholar
39. 39.
Vasil'ev, V.A. and Koshkin, G.M., Nonparametric Estimation of the Relations of Derivatives of a Multivariate Distribution Density from Dependent Observations, Sib. Mat. Zh., 2000, vol. 41, no. 2, pp. 312-330.Google Scholar
40. 40.
Rosenblatt, M., Remarks on Some Nonparametric Estimates of Density functions, Ann. Math. Statist., 1956, vol. 27, no. 3, pp. 832-837.Google Scholar
41. 41.
Parzen, E., On Estimation of a Probability Density Function and Mode, Ann. Math. Statist., 1962, vol. 33, no. 3, pp. 1065-1076.Google Scholar
42. 42.
Gasser, T. and Müller, H.-G., Kernel Estimation of Regression Functions, Lect. Notes Math., 1979, vol. 757, pp. 23-68.Google Scholar
43. 43.
Müller, H.-G. and Gasser, T., Optimal Convergence Properties of Kernel Estimates of Derivatives of a Density Function, Lect. Notes Math., 1979, vol. 757, pp. 144-154.Google Scholar
44. 44.
Koshkin, G.M., An Improved Nonnegative Kernel Density Estimate, Teor. Veroyatn. Primenen., 1988, vol. 33, issue 4, pp. 817-822.Google Scholar
45. 45.
Ibragimov, I.A. and Linnik, Yu.V., Nezavisimye i statsionarno svyazannye velichiny (Independent and Stationarily Connected Variables), Moscow: Nauka, 1965.Google Scholar
46. 46.
Bradley, R. and Bryc, W., Multilinear Forms and Measures of Dependence between Random Variables, J. Mult. Anal., 1985, vol. 16, no. 3, pp. 335-367.Google Scholar
47. 47.
Davydov, Yu.A., Convergence of Distributions generated by Stationary Random Processes, Teor. Veroyatn. Primenen., 1968, vol. 13, issue. 4, pp. 730-737.Google Scholar
48. 48.
Koshkin, G.M., Asymptotic Properties of Functions of Statistics and Their Application in Nonparametric Estimation, Avtom. Telemekh., 1990, no. 3, pp. 82-97.Google Scholar
49. 49.
Natanson, I.P., Teoriya funktsii veshchestvennoi peremennoi (Theory of Functions of a Real Variable), Moscow: Nauka, 1974.Google Scholar
50. 50.
Schuster, E.F., Estimation of a Probability Density Function and its derivatives, Ann. Math. Statist., 1969, vol. 40, no. 4, pp. 1187-1195.Google Scholar
51. 51.
Schuster, E.F., Joint Asymptotic Distribution of the Estimated Regression Function at a Finite Number of Distinct Points, Ann. Math. Statist., 1972, vol. 43, no. 1, pp. 84-88.Google Scholar
52. 52.
Billingsley, P., Convergence of Probability Measures, New York: Wiley, 1968. Translated under the title Skhodimost' veroyatnostnykh mer, Moscow: Nauka, 1977.Google Scholar