Automation and Remote Control

, Volume 62, Issue 9, pp 1453–1476 | Cite as

Nonparametric Estimation of the Logarithmic Density Derivative of Sequences with Strong Mixing

  • A. V. Dobrovidov
  • G. M. Koshkin


In nonparametric signal estimation, there is a need for estimating the logarithmic probability density derivatives. This problem is complex, because the logarithmic density derivative is a function with singularity—a ratio containing density in the denominator. Since the density estimate can take values close or even equal to zero, the estimate of the logarithmic derivative becomes unstable. This difficulty is surmounted by constructing a new nonparametric estimate for the logarithmic derivative, i.e., an estimate that is stable to observation and based piecewise-smooth approximation. Its properties for dependent observations generated by stationary processes satisfying the strong mixing condition are studied. The rate of convergence of the nonparametric estimate and the principal part of the expansion of the mean-square estimate error are determined.


Probability Density Estimate Error Stationary Process System Theory Density Estimate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Roussas, G.G., On Some Properties of Nonparametric Estimates of Probability Density Functions, Bull. Soc. Math., 1968, no. 1, pp. 29-43.Google Scholar
  2. 2.
    Roussas, G.G., Nonparametric Estimation in Markov Processes, Ann. Inst. Statist. Math., 1969, vol. 21, no. 1, pp. 73-87.Google Scholar
  3. 3.
    Rosenblatt, M., Density Estimates and Markov Sequences. Nonparametric Techniques in Statistical Inference, Cambridge: Cambridge Univ. Press, 1970.Google Scholar
  4. 4.
    Rosenblatt, M., Curve Estimates, Ann. Math. Statist., 1971, vol. 42, no. 6, pp. 1815-1842.Google Scholar
  5. 5.
    Rosenblatt, M., Markov Processes: Structure and Asymptotic Behavior, Berlin: Springer-Verlag, 1971.Google Scholar
  6. 6.
    Koshkin, G.M. and Tarasenko, F.P., A Consistency Condition for Weakly Dependent Samples, Mat. Statist. Prilozh., 1976, issue 4, pp. 29-41.Google Scholar
  7. 7.
    Koshkin, G.M. and Tarasenko, F.P., Recurrent Probability Density Estimation and Regression Lines of Dependent Samples, Mat. Statist. Prilozh., 1976, issue 4, pp. 122-138.Google Scholar
  8. 8.
    Koshkin, G.M., An Approach to Estimation of Transition Distribution Functions and Moments for Certain Markov Processes, Mat. Statist. Prilozh., 1976, issue 4, pp. 116-121.Google Scholar
  9. 9.
    Delecroix, M., Central Limit Theorems for Density Estimators of a ϕ-Mixing Process, in Recent Developments in Statistics, Amsterdam: North Holland, 1977, pp. 409-414.Google Scholar
  10. 10.
    Masry, E., Probability Density Estimation from Sampled Data, IEEE Trans. Inf. Theory, 1983, vol. 29, no. 5, pp. 696-709.Google Scholar
  11. 11.
    Masry, E., Recursive Probability Density Estimation for Weakly Dependent Stationary Processes, IEEE Trans. Inf. Theory, 1986, vol. 32, no. 2, pp. 254-267.Google Scholar
  12. 12.
    Koshkin, G.M. and Tarasenko, F.P., Nonparametric Algorithms for Identifying and Control of Continuous-Discrete Stochastic Objects, 8th IFAC-IFORS Symp. Identif. Syst. Parameter Estimation, Beijing: Pergamon Press, 1988, vol. 2, pp. 882-887.Google Scholar
  13. 13.
    Castellana, J.V. and Leadbetter, M.R., On Smoothed Probability Density Estimation for Stationary Processes, Stoch. Proces. Appl., 1986, vol. 21, no. 2, pp. 179-193.Google Scholar
  14. 14.
    Györfi, L., Strong Consistent Density Estimate from Ergodic Samples, J. Mult. Anal., 1981, vol. 11, no. 1, pp. 81-84.Google Scholar
  15. 15.
    Györfi, L., Recent Results on Nonparametric Regression Estimate and Multiple Classification, Prob. Control Inf. Theory, 1981, vol. 10, no. 1, pp. 43-52.Google Scholar
  16. 16.
    Pracasa Rao, B.L.S., Nonparametric Functional Estimation, Orlando: Academic, 1983.Google Scholar
  17. 17.
    Györfi, L., Hardle, W., Sarda, P., and Vieu, P., Nonparametric Curve Estimation from Time Series, Berlin: Springer, 1988.Google Scholar
  18. 18.
    Masry, E., Nonparametric Estimation of Conditional Probability Densities and Expectations of Stationary Processes: Strong Consistency and Rates, Stoch. Proces. Appl., 1989, vol. 32, no. 1, pp. 109-127.Google Scholar
  19. 19.
    Masry, E., Almost Sure Convergence of Recursive Density Estimators for Stationary Mixing Processes, Statist. Prob. Lett., 1987, vol. 5, no. 2, pp. 249-254.Google Scholar
  20. 20.
    Masry, E. and Györfi, L., Strong Consistency and Rates for Recursive Probability Density Estimators of Stationary Processes, J. Mult. Anal., 1987, vol. 22, no. 1, pp. 79-93.Google Scholar
  21. 21.
    Doukhan, P. and Ghindes, M., Estimation de la transition de probabilite d'une chain de Markov Doeblin-recurrente. Etude du eas processus autoregressif general d'ordre 1, Stoch. Proces. Appl., 1983, vol. 15, no. 3, pp. 271-293.Google Scholar
  22. 22.
    Robinson, P.M., Nonparametric Estimation from Time Series Residuals, Cah. Cent. Etud. Rech. Oper., 1986, vol. 28, no. 1-3, pp. 197-202.Google Scholar
  23. 23.
    Vasil'ev, V.A., Estimation of the Distribution of Disturbances in an Autoregression Process, Mat. Stat. Pril., 1986, issue 10, pp. 9-24.Google Scholar
  24. 24.
    Masry, E., Multivariate Probability Density Deconvolution for Stationary Random Processes, IEEE Trans. Inf. Theory, 1991, vol. 37, no. 4, pp. 1105-1115.Google Scholar
  25. 25.
    Tran, L.T., Kernel Density Estimation of Random Fields, J. Mult. Anal., 1990, vol. 34, no. 1, pp. 37-53.Google Scholar
  26. 26.
    Vasil'ev, V.A. and Koshkin, G.M., Estimation of Limiting Distribution Density and Its Derivative from Weakly Dependent Observations, Probl. Peredachi Inf., 1997, vol. 33, issue 2, pp. 66-80.Google Scholar
  27. 27.
    Koshkin, G. and Vasil'iev, V., Estimation of Multivariate Probability Density and Its Derivatives from Weakly Dependent Observations, in: Proc. Steklov Math. Inst. Seminar “Statistics and Control of Stochastic Processes,” The Liptser Festschrift, Steklov Math. Inst., Yu.M. Kabanov, B.L. Rozovskii, and A.N. Shiryaev, Eds, Singapore: World Scientific, 1997, pp. 229-241.Google Scholar
  28. 28.
    Vail'ev, V.A. and Koshkin, G.M., Estimation of Distribution Density Functions from Dependent Observations, Probl. Peredachi Inf., 1997, vol. 33, issue 4, pp. 45-60.Google Scholar
  29. 29.
    Vail'ev, V.A. and Koshkin, G.M., Nonparametric Identification of Autoregressions, Teor. Veroyatn. Primenen., 1998, vol. 43, issue 3, pp. 577-588.Google Scholar
  30. 30.
    Koshkin, G.M. and Vasil'iev, V.A., Nonparametric Estimation of Multivariate Density Derivatives from Dependent Observations, Math. Methods Statist., 1998, vol. 7, no. 4, pp. 361-400.Google Scholar
  31. 31.
    Nemirovskii, A.S. and Tsypkin, Ya.Z., Optimal Adaptive Control Algorithms, Avtom. Telemekh., 1984, no. 12, pp. 64-77.Google Scholar
  32. 32.
    Nadaraya, E.A., Nonparametric Estimates of Probability Density and Regression, Teor. Veroyatn. Primenen., 1965, vol. 10, issue 1, pp. 199-203.Google Scholar
  33. 33.
    Kushnir, A.F., Asymptotically Optimal Criteria for Hypothesis Testing Regression Problems, Teor. Veroyatn. Primenen., 1968, vol. 13, issue 4, pp. 682-700.Google Scholar
  34. 34.
    Dobrovidov, A.V., Asymptotically ε-Optimal Nonparametric Filtering of Stationary Sequences with Unknown Statistical Characteristics, Avtom. Telemekh., 1984, no. 12, pp. 40-49.Google Scholar
  35. 35.
    Dobrovidov, A.V. and Koshkin, G.M., Neparametricheskoe otsenivanie signalov (Nonparametric Estimation of Signals), Moscow: Nauka, Fizmatlit, 1997.Google Scholar
  36. 36.
    Cramer, H., Mathematical Methods of Statistics, Princeton: Princeton Univ. Press, 1937. Translated under the title Matematicheskie metody statistiki, Moscow: Mir, 1975.Google Scholar
  37. 37.
    Penskaya, M.Ya., Stable Estimation of a Function of a Parameter, in Statisticheskie metody otsenivaniya i proverki gipotez (Statistical Methods of Estimation and Testing of Hypotheses), Perm: Perm. Gos. Univ., 1990, pp. 44-55.Google Scholar
  38. 38.
    Koshkin, G.M., Moments of Deviations of the Estimates of a Substitution and Its Piecewise-Smooth Approximations, Sib. Mat. Zh., 1999, vol. 40, no. 3, pp. 601-614.Google Scholar
  39. 39.
    Vasil'ev, V.A. and Koshkin, G.M., Nonparametric Estimation of the Relations of Derivatives of a Multivariate Distribution Density from Dependent Observations, Sib. Mat. Zh., 2000, vol. 41, no. 2, pp. 312-330.Google Scholar
  40. 40.
    Rosenblatt, M., Remarks on Some Nonparametric Estimates of Density functions, Ann. Math. Statist., 1956, vol. 27, no. 3, pp. 832-837.Google Scholar
  41. 41.
    Parzen, E., On Estimation of a Probability Density Function and Mode, Ann. Math. Statist., 1962, vol. 33, no. 3, pp. 1065-1076.Google Scholar
  42. 42.
    Gasser, T. and Müller, H.-G., Kernel Estimation of Regression Functions, Lect. Notes Math., 1979, vol. 757, pp. 23-68.Google Scholar
  43. 43.
    Müller, H.-G. and Gasser, T., Optimal Convergence Properties of Kernel Estimates of Derivatives of a Density Function, Lect. Notes Math., 1979, vol. 757, pp. 144-154.Google Scholar
  44. 44.
    Koshkin, G.M., An Improved Nonnegative Kernel Density Estimate, Teor. Veroyatn. Primenen., 1988, vol. 33, issue 4, pp. 817-822.Google Scholar
  45. 45.
    Ibragimov, I.A. and Linnik, Yu.V., Nezavisimye i statsionarno svyazannye velichiny (Independent and Stationarily Connected Variables), Moscow: Nauka, 1965.Google Scholar
  46. 46.
    Bradley, R. and Bryc, W., Multilinear Forms and Measures of Dependence between Random Variables, J. Mult. Anal., 1985, vol. 16, no. 3, pp. 335-367.Google Scholar
  47. 47.
    Davydov, Yu.A., Convergence of Distributions generated by Stationary Random Processes, Teor. Veroyatn. Primenen., 1968, vol. 13, issue. 4, pp. 730-737.Google Scholar
  48. 48.
    Koshkin, G.M., Asymptotic Properties of Functions of Statistics and Their Application in Nonparametric Estimation, Avtom. Telemekh., 1990, no. 3, pp. 82-97.Google Scholar
  49. 49.
    Natanson, I.P., Teoriya funktsii veshchestvennoi peremennoi (Theory of Functions of a Real Variable), Moscow: Nauka, 1974.Google Scholar
  50. 50.
    Schuster, E.F., Estimation of a Probability Density Function and its derivatives, Ann. Math. Statist., 1969, vol. 40, no. 4, pp. 1187-1195.Google Scholar
  51. 51.
    Schuster, E.F., Joint Asymptotic Distribution of the Estimated Regression Function at a Finite Number of Distinct Points, Ann. Math. Statist., 1972, vol. 43, no. 1, pp. 84-88.Google Scholar
  52. 52.
    Billingsley, P., Convergence of Probability Measures, New York: Wiley, 1968. Translated under the title Skhodimost' veroyatnostnykh mer, Moscow: Nauka, 1977.Google Scholar

Copyright information

© MAIK “Nauka/Interperiodica” 2001

Authors and Affiliations

  • A. V. Dobrovidov
    • 1
  • G. M. Koshkin
    • 1
  1. 1.Trapeznikov Institute of Control SciencesRussian Academy of SciencesMoscowRussia

Personalised recommendations