Abstract
Let {X n } ∞ n=0 be a stationary real-valued time series with unknown distribution. Our goal is to estimate the conditional expectation ofX n+1 based on the observations,X i , 0≤i≤n in a strongly consistent way. Bailey and Ryabko proved that this is not possible even for ergodic binary time series if one estimates at all values ofn. We propose a very simple algorithm which will make prediction infinitely often at carefully selected stopping times chosen by our rule. We show that under certain conditions our procedure is strongly (pointwise) consistent, andL 2 consistent without any condition. An upper bound on the growth of the stopping times is also presented in this paper.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Algoet, P. (1992). Universal schemes for prediction, gambling and portfolio selection.Annals of Probability, 20:901–941.
Algoet, P. (1994). The strong law of large numbers for sequential decisions under uncertainity.IEEE Transactions on Information Theory, 40:609–634.
Algoet, P. (1999). Universal schemes for learning the best nonlinear predictor given the infinite past and side information.IEEE Transactions on Information Theory, 45:1165–1185.
Ash, R. (1972).Real Analysis and Propbability Academic Press, New York.
Bailey, D. (1976).Sequential Schemes for Classifying and Predicting Ergodic Processes. Ph.D thesis, Stanford University.
Cover, T. andThomas, J. (1991).Elements of Information Theory. Wiley, New York.
Cover, T. M. (1975). Open problems in Information Theory, In1975 IEEE Joint workshop on Information Theory, pp. 35–36. IEEE Press, New York.
Csiszár, I. (2002). Large-scale typicality of Markov sample paths and consistency of MDL order estimators.IEEE Transactons on Information Theory, 48:1616–1628.
Csiszár, I. andShields, P. (2000). The consistency of the BIC Markov order estimator.Annals of Statistics, 28:1601–1619.
Gray, R. (1988).Probability, Random Processes, and Ergodic Properties. Springer-Verlag, New York.
Györfi, L., Kohler, M., Krzyzak, A., andWalk.,H. (2002).A Distribution Free Theory of Nonparametric Regression. Springer-Verlag, New York.
Györfi, L. andLugosi, G. (2002). Strategies for sequential prediction of stationary time series. In M. Dror, P. Ecuyer, and F. Szidarovszky eds.,Modeling Uncertainity An Examination of Stochastic Theory, Methods, and Applications, pp. 225–248. Kluwer Academic Publishers, Dordrecht.
Györfi, L., Lugosi, G., andMorvai, G. (1999). A simple randomized algorithm for consistent sequential prediction of ergodic time series.IEEE Transactions on Information Theory, 45:2642–2650.
Györfi, L., Morvai, G., andYakowitz, S. (1998). Limits to consistent on-line forecasting for ergodic time series.IEEE Transactions on Information Theory, 44:886–892.
Kalikow, S. (1990). Random Markov processes and uniform martingales.Israel Journal of Mathematics, 71:33–54.
Keane, M. (1972). Strongly mixing g-measures.Invent. Math., 16:309–324.
Morvai, G. (2003). Guessing the output of a stationary, binary time series. In Y. Haitovsky, H. Lerche and Y. Ritov, eds.Foundations of Statistical Inference, pp. 205–213. Physika Verlag, Heidelberg New York.
Morvai, G. andWeiss, B. (2003). Forecasting for stationary binary time series.Acta Applicandae Mathematicae, 79:25–34.
Morvai, G., Yakowitz, S., andAlgoet, P. (1997). Weakly convergent nonparametric forecasting of stationary time series.IEEE Transactions on Information Theory, 43:483–498.
Morvai, G., Yakowitz, S., andGyörfi, L. (1996). Nonparametric inferences for ergodic, stationary time series.Annals of Statistics, 24:370–379.
Ornstein, D. (1974).Ergodic Theory, Randomness, and Dynamical Systems. Yale University Press, New Haven.
Ornstein, D. (1978). Guessing the next output of a stationary process.Israel Journal of Mathematics, 30:292–296.
Ornstein, D. andWeiss, B. (1993). Entropy and data compression schemes.IEEE Transactions on Information Theory, 39:78–83.
Révész, P. (1968).The Law of Large Numbers. Academic Press, New York.
Ryabko, B. Y. (1988). Prediction of random sequences and universal coding.Problems of Inform. Trans. 24:87–96.
Schäfer, D. (2002). Strongly consistent online forecasting of centered gaussian processes.IEEE Transactions on Information Theory, 48:791–799.
Shields, P. (1991). Cutting and stacking: a method for constructing stationary processes.IEEE Transactions on Information Theory, 37:1605–1614.
Weiss, B. (2000).Single Orbit Dynamics. American Mathematical Society, Providence, RI.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Morvai, G., Weiss, B. Intermittent estimation of stationary time series. Test 13, 525–542 (2004). https://doi.org/10.1007/BF02595785
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02595785