Abstract
This paper is about the one-step ahead prediction of the future of observations drawn from an infinite-order autoregressive AR(\(\infty \)) process. It aims to design penalties (fully data driven) ensuring that the selected model verifies the efficiency property but in the non-asymptotic framework. We show that the excess risk of the selected estimator enjoys the best bias-variance trade-off over the considered collection. To achieve these results, we needed to overcome the dependence difficulties by following a classical approach which consists in restricting to a set where the empirical covariance matrix is equivalent to the theoretical one. We show that this event happens with probability larger than \(1-c_0/n^2\) with \(c_0>0\). The proposed data-driven criteria are based on the minimization of the penalized criterion akin to the Mallows’s \(C_p\).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baraud, Y., Comte, F., Viennet, G. (2001a). Model selection for (auto-) regression with dependent data. ESAIM: Probability and Statistics, 5, 33–49.
Baraud, Y., Comte, F., Viennet, G., et al. (2001b). Adaptive estimation in autoregression or-mixing regression via model selection. The Annals of Statistics, 29(3), 839–875.
Bardet, J.-M., Wintenberger, O. (2009). Asymptotic normality of the quasi-maximum likelihood estimator for multidimensional causal processes. The Annals of Statistics, 37(5B), 2730–2759.
Birgé, L., Massart, P. (2001). Gaussian model selection. Journal of the European Mathematical Society, 3(3), 203–268.
Birgé, L., Massart, P. (2007). Minimal penalties for gaussian model selection. Probability theory and related fields, 138(1–2), 33–73.
Comte, F., Dedecker, J., Taupin, M.-L. (2008). Adaptive density deconvolution with dependent inputs. Mathematical methods of Statistics, 17(2), 87.
Comte, F., Genon-Catalot, V. (2020). Regression function estimation as a partly inverse problem. Annals of the Institute of Statistical Mathematics, 72(4), 1023–1054.
Dedecker, J., Prieur, C. (2005). New dependence coefficients. examples and applications to statistics. Probability Theory and Related Fields, 132(2), 203–236.
Doukhan, P., Wintenberger, O. (2008). Weakly dependent chains with infinite memory. Stochastic Processes and their Applications, 118(11), 1997–2013.
Goldenshluger, A., Zeevi, A. (2001). Nonasymptotic bounds for autoregressive time series modeling. Annals of Statistics, 29, 417–444.
Hsu, D., Kakade, S. M., Zhang, T. ( 2011). An analysis of random design linear regression. arXiv preprint arXiv:1106.2363 .
Ing, C.-K., Wei, C.-Z. (2003). On same-realization prediction in an infinite-order autoregressive process. Journal of Multivariate Analysis, 85(1), 130–155.
Ing, C.-K., Wei, C.-Z., et al. (2005). Order selection for same-realization predictions in autoregressive processes. The Annals of Statistics, 33(5), 2423–2474.
Klein, T., Rio, E., et al. (2005). Concentration around the mean for maxima of empirical processes. The Annals of Probability, 33(3), 1060–1077.
Lebarbier, E., Mary-Huard, T. (2004). Le critère BIC: fondements théoriques et interprétation, PhD thesis, INRIA.
Lerasle, M., et al. (2011). Optimal model selection for density estimation of stationary data under various mixing conditions. The Annals of Statistics, 39(4), 1852–1877.
Shibata, R. (1980). Asymptotically efficient selection of the order of the model for estimating parameters of a linear process. The Annals of Statistics, 8(1), 147–164.
Van de Geer, S. A. (2002). On hoeffding’s inequality for dependent random variables. In Empirical process techniques for dependent data (pp. 161–169), Springer.
Acknowledgements
The author thanks William KENGNE and Jean-Marc BARDET for proofreads and helpful discussions. Kare Kamila has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 754362.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Kare Kamila has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 754362.
About this article
Cite this article
Kamila, K. Data-driven model selection for same-realization predictions in autoregressive processes. Ann Inst Stat Math 75, 567–592 (2023). https://doi.org/10.1007/s10463-022-00855-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-022-00855-1