Statistical Estimation of Parameters for Binary Conditionally Nonlinear Autoregressive Time Series


The problem of statistical parameter estimation is considered for binary GLM-based autoregression with the link function of general form and the base functions (regressors) nonlinear w.r.t. the lagged variables. A new consistent asymptotically normal frequencies-based estimator (FBE) is constructed and compared with the classical MLE. It is shown that the FBE has less restrictive sufficient conditions of uniqueness than the MLE (does not need log-concavity of the inverse link) and can be computed recursively under the model extension. The sparse version of the FBE is proposed and the optimal model-dependent weight matrices (parameterizing the FBE) are found for the FBE and for the sparse FBE. The proposed empirical choice of the subset of s-tuples for the sparse FBE is examined by numerical and analytical examples. Computer experiments for comparison of the FBE versus the MLE are performed on simulated and real (genetic) data.

This is a preview of subscription content, access via your institution.


  1. 1.

    NCBI Nucleotide database, 07.04.2017.

  2. 2.

    S. Amari and H. Nagaoka,Methods of Information Geometry, Oxford Univ. Press, 2000.

    MATH  Google Scholar 

  3. 3.

    M. Bagnoli and T. Bergstrom,Log-Concave Probability and Its Applications, Univ. of Michigan, 1989.

    MATH  Google Scholar 

  4. 4.

    G. Bernardi, “Isochores and the Evolutionary Genomics of Vertebrates”, Gene 241 (1), 3–17 (2000).

    Article  Google Scholar 

  5. 5.

    P. Billingsley, “Statistical Methods in Markov Chains”, Ann. Math. Statist. 32 (1), 12–40 (1961).

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    J. L. Doob,Stochastic Processes, Wiley, New York, 1953.

    MATH  Google Scholar 

  7. 7.

    L. Fahrmeir and H. Kaufmann, “Consistency and Asymptotic Normality of the Maximum Likelihood Estimator in Generalized Linear Models”, Ann. Statist. 13 (1), 342–368 (1985).

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    J. Fan and Q. Yao,Nonlinear Time Series: Nonparametric and Parametric Methods, Springer, New York, 2003.

    Book  MATH  Google Scholar 

  9. 9.

    K. Fokianos and R. Fried, “Interventions in INGARCHProcesses”, J. Time Series Analysis 31 (3), 210–225 (2010).

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    R. Fried S. Kuhls and I. Molina, “Analyzing Associations in Multivariate Binary Time Series”, in,COMPSTAT: Proc. in Comput. Statist., Ed. by A. Rizzi and M. Vichi Physica-Verlag, 2006, pp. 985–992.

    Google Scholar 

  11. 11.

    P. E. Gill W. Murray, and M. H. Wright,,Practical Optimization, Academic Press, London, 1981.

    MATH  Google Scholar 

  12. 12.

    S. Gouveia M. G. Scotto C. H. Weiss and P. J. S. G. Ferreira, “Binary Auto-Regressive Geometric Modelling in a DNA Context”, J. Roy. Statist. Soc., Ser. C 66 (2), 253–271 (2017).

    MathSciNet  Article  Google Scholar 

  13. 13.

    S. J. Haberman, “Maximum Likelihood Estimates in Exponential Response Models”, Ann. Statist. 5 (5), 815–841 (1977).

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    M. Hayashi and S. Watanabe, “Information Geometry Approach to Parameter Estimation in Markov Chains”, Ann. Statist. 44 (4), 1495–1535 (2016).

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    P. A. Jacobs and P. A. W. Lewis, “Stationary Discrete Autoregressive-Moving Average Time Series Generated by Mixtures”, J. Time Series Analysis 4 (1), 19–36 (1983).

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    R. I. Jennrich and P. F. Sampson, “Newton–Raphson and Related Algorithms for Maximum Likelihood Variance Component Estimation”, Technometrics 18 (1), 11–17 (1976).

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    C. Jordan, “Essai sur la géométrie à n dimensions”, Bulletin de la SociétéMathématique de France 3, 103–174 (1875).

    Article  MATH  Google Scholar 

  18. 18.

    M. Kanter, “Autoregression for Discrete Processesmod 2”, J. Appl. Probab. 12, 371–375 (1975).

    Article  MATH  Google Scholar 

  19. 19.

    B. Kedem and K. Fokianos,Regression Models for Time Series Analysis, Wiley, Hoboken, 2002.

    Book  MATH  Google Scholar 

  20. 20.

    A. Yu. Kharin,“Robust Bayesian Prediction under Distortions of Prior and Conditional Distributions”, J. Math. Sci. 126 (1), 992–997 (2005).

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    A. Kharin, “Performance and Robustness Evaluation in Sequential Hypotheses Testing”, Commun. in Statist.–Theory Methods 45 (6), 1693–1709 (2016).

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    A. Yu. Kharin and D. V. Kishylau, “Robust Sequential Test for Hypotheses about Discrete Distributions in the Presence of Outliers”, J. Math. Sci. 205 (1), 68–73 (2015).

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Y. Kharin, “Robustness of Clustering under Outliers”, Lecture Notes in Computer Science, 1280, 501–511 (1997).

    Article  Google Scholar 

  24. 24.

    Y. Kharin, “Robustness of the Mean Square Risk in Forecasting of Regression Time Series”, Commun. in Statist.–Theory Methods 40 (16), 2893–2903 (2011).

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Yu. Kharin,,Robustness in Statistical Forecasting, Springer, New York, 2013.

  26. 26.

    Yu. Kharin and A. Piatlitski, “Markov Chain of Order s with r Partial Connections and Statistical Inference on its Parameters”, DiscreteMath. and Appl. 17 (3), 295–317 (2007).

    MathSciNet  MATH  Google Scholar 

  27. 27.

    Yu. S. Kharin and E. V. Vecherko, “Statistical Estimation of Parameters for Binary Markov Chain Models with Embeddings”, DiscreteMath. and Appl. 23 (2), 153–169 (2013).

    MATH  Google Scholar 

  28. 28.

    Yu. S. Kharin and V. A. Voloshko, “Robust Estimation of AR Coefficients under Simultaneously Influencing Outliers and Missing Values”, J. Statist. Planning and Inference 141 (9), 3276–3288 (2011).

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Yu. S. Kharin and V. A. Voloshko, “On Asymptotic Properties of the Plug-in Cepstrum Estimator for Gaussian Time Series”, Math. Methods Statist. 21 (1), 43–60 (2012).

    MathSciNet  Article  MATH  Google Scholar 

  30. 30.

    Y. Kharin and E. Zhuk, “Filtering of Multivariate Samples Containing, “Outliers” for Clustering”, Pattern Recognition Lett. 19 (12), 1077–1085 (1998).

    Article  MATH  Google Scholar 

  31. 31.

    R. Koenker and J. Yoon, “Parametric Links for Binary Choice Models: a Fisherian–Bayesian Colloquy”, J. Econometrics 152 (2), 120–130 (2009).

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    P. McCullagh and J. A. Nelder,Generalized Linear Models. Second Ed., Chapman and Hall: London, 1989.

    Book  MATH  Google Scholar 

  33. 33.

    J. Nelder and R. Wedderburn,“Generalized Linear Models”, J. Royal Statist. Soc., Ser. A 135 (3), 370–384 (1972).

    Article  Google Scholar 

  34. 34.

    B. Noble and J.W. Daniel,Applied Linear Algebra, Prentice-Hall, Englewood Cliffs, NJ, 1988, 3rd. ed.

    MATH  Google Scholar 

  35. 35.

    A. Raftery and S. Tavare, “Estimation and Modelling Repeated Patterns in High Order Markov Chains with the Mixture Transition DistributionModel”, J. Appl. Statist. 43 (1), 179–199 (1994).

    Article  MATH  Google Scholar 

  36. 36.

    A. N. Shiryaev,Probability, Springer, New York, 1995.

    MATH  Google Scholar 

  37. 37.

    E. M. Stein and R. Shakarchi Complex Analysis, Princeton Univ. Press, 2003.

    MATH  Google Scholar 

  38. 38.

    V. A. Voloshko, “Steganographic Capacity for One-Dimensional Markov Cover”, Discrete Math. Appl. 27 (4), 247–268 (2017).

    MathSciNet  Article  MATH  Google Scholar 

  39. 39.

    R.W.M. Wedderburn,“On the Existence and Uniqueness of the Maximum Likelihood Estimates for Certain Generalized Linear Models”, Biometrika 63 (1), 27–32 (1976).

    Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Yu. S. Kharin.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kharin, Y.S., Voloshko, V.A. & Medved, E.A. Statistical Estimation of Parameters for Binary Conditionally Nonlinear Autoregressive Time Series. Math. Meth. Stat. 27, 103–118 (2018).

Download citation


  • time series
  • binary data
  • parsimonious model
  • estimation
  • asymptotic properties

2010 Mathematics Subject Classification

  • 62M05
  • 62F12
  • 60J10