Skip to main content
SpringerLink
Log in

High-dimensional VAR with low-rank transition

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We propose a vector auto-regressive model with a low-rank constraint on the transition matrix. This model is well suited to predict high-dimensional series that are highly correlated, or that are driven by a small number of hidden factors. While our model has formal similarities with factor models, its structure is more a way to reduce the dimension in order to improve the predictions, rather than a way to define interpretable factors. We provide an estimator for the transition matrix in a very general setting and study its performances in terms of prediction and adaptation to the unknown rank. Our method obtains good result on simulated data, in particular when the rank of the underlying process is small. On macroeconomic data from Giannone et al. (Rev Econ Stat 97(2):436–451, 2015), our method is competitive with state-of-the-art methods in small dimension and even improves on them in high dimension.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Alquier, P., Guedj, B.: Simpler PAC-Bayesian bounds for hostile data. Mach. Learn. 107(5), 887–902 (2018)

    MathSciNet  MATH  Google Scholar 

  • Alquier, P., Li, X.: Prediction of quantiles by statistical learning and application to GDP forecasting. In: International Conference on Discovery Science, pp. 22–36. Springer, Berlin (2012)

  • Alquier, P., Marie, N.: Matrix factorization for multivariate time series analysis. Electron. J. Stat. 13(2), 4346–4366 (2019)

    MathSciNet  MATH  Google Scholar 

  • Alquier, P., Wintenberger, O.: Model selection for weakly dependent time series forecasting. Bernoulli 18(3), 883–913 (2012)

    MathSciNet  MATH  Google Scholar 

  • Alquier, P., Li, X., Wintenberger, O.: Prediction of time series by statistical learning: general losses and fast rates. Depend. Model. 1, 65–93 (2013)

    MATH  Google Scholar 

  • Alquier, P., Cottet, V., Lecué, G.: Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions. Ann. Stat. 47(4), 2117–2144 (2019a)

    MathSciNet  MATH  Google Scholar 

  • Alquier, P., Doukhan, P., Fan, X.: Exponential inequalities for nonstationary Markov chains. Depend. Model. 7, 150–168 (2019b)

    MathSciNet  MATH  Google Scholar 

  • Anderson, T.W.: Estimating linear restrictions on regression coefficients for multivariate normal distributions. Ann. Math. Stat. 22(3), 327–351 (1951)

    MathSciNet  MATH  Google Scholar 

  • Arlot, S.: Minimal penalties and the slope heuristics: a survey. arXiv preprint arXiv:1901.07277 (2019)

  • Arlot, S., Massart, P.: Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res. 10(Feb), 245–279 (2009)

    Google Scholar 

  • Basu, S., Meckesheimer, M.: Automatic outlier detection for time series: an application to sensor data. Knowl. Inf. Syst. 11(2), 137–154 (2007)

    Google Scholar 

  • Basu, S., Li, X., Michailidis, G.: Low rank and structured modeling of high-dimensional vector autoregressions. IEEE Trans. Signal Process. 67(5), 1207–1222 (2019)

    MathSciNet  MATH  Google Scholar 

  • Baudry, J.-P., Maugis, C., Michel, B.: Slope heuristics: overview and implementation. Stat. Comput. 22(2), 455–470 (2012)

    MathSciNet  MATH  Google Scholar 

  • Bauwens, L., Laurent, S., Rombouts, J.V.K.: Multivariate GARCH models: a survey. J. Appl. Econom. 21, 79–109 (2006)

    MathSciNet  Google Scholar 

  • Bing, X., Wegkamp, M.H.: Adaptive estimation of the rank of the coefficient matrix in high-dimensional multivariate response regression models. Ann. Stat. 47(6), 3157–3184 (2019)

    MathSciNet  MATH  Google Scholar 

  • Birgé, L., Massart, P.: Minimal penalties for Gaussian model selection. Probab. Theory Relat. Fields 138(1–2), 33–73 (2007)

    MathSciNet  MATH  Google Scholar 

  • Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1), 1–122 (2011)

    MATH  Google Scholar 

  • Buja, A., Hastie, T., Tibshirani, R.: Linear smoothers and additive models. Ann. Stat. 17(2), 453–510 (1989)

    MathSciNet  MATH  Google Scholar 

  • Bunea, F., She, Y., Wegkamp, M.H.: Optimal selection of reduced rank estimators of high-dimensional matrices. Ann. Stat. 39(2), 1282–1309 (2011)

    MathSciNet  MATH  Google Scholar 

  • Candès, E.J., Plan, Y.: Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements. IEEE Trans. Inf. Theory 57(4), 2342–2359 (2011)

    MathSciNet  MATH  Google Scholar 

  • Carel, L.: Big data analysis in the field of transportation. Ph.D. thesis, Université Paris-Saclay (2019)

  • Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)

    MATH  Google Scholar 

  • Chan, J., Leon-Gonzalez, R., Strachan, R.W.: Invariant inference and efficient computation in the static factor model. J. Am. Stat. Assoc. 113(522), 819–828 (2018)

    MathSciNet  MATH  Google Scholar 

  • Chen, L., Wu, W.B.: Testing for trends in high-dimensional time series. J. Am. Stat. Assoc. 114, 869–881 (2018)

    MathSciNet  MATH  Google Scholar 

  • Cornec, M.: Constructing a conditional GDP fan chart with an application to French business survey data. OECD J. J. Bus. Cycle Meas. Anal. 2013(2), 109–127 (2014)

    Google Scholar 

  • Davis, R.A., Zang, P., Zheng, T.: Sparse vector autoregressive modeling. J. Comput. Graph. Stat. 25(4), 1077–1096 (2016)

    MathSciNet  Google Scholar 

  • De Castro, Y., Goude, Y., Hébrail, G., Mei, J.: Recovering multiple nonnegative time series from a few temporal aggregates. In: ICML 2017-34th International Conference on Machine Learning, pp. 1–9 (2017)

  • Dedecker, J., Doukhan, P., Lang, G., León, JR., Louhichi, S., Prieur, C.: Weak Dependence: With Examples and Applications. Lecture Notes in Statistics, vol. 190, p. xiv+318. Springer, New York (2007)

  • Dedecker, J., Fan, X.: Deviation inequalities for separately Lipschitz functionals of iterated random functions. Stoch. Process. Appl. 125(1), 60–90 (2015)

    MathSciNet  MATH  Google Scholar 

  • Dedecker, J., Doukhan, P., Fan, X.: Deviation inequalities for separately Lipschitz functionals of composition of random functions. J. Math. Anal. Appl. 479(2), 1549–1568 (2019)

    MathSciNet  MATH  Google Scholar 

  • Doukhan, P.: Stochastic Models for Time Series. Series Mathématiques et Applications, vol. 80. Springer, Berlin (2018)

    MATH  Google Scholar 

  • Engle, R.: ARCH: Selected Readings. Oxford University Press, Oxford (1995)

    Google Scholar 

  • Francq, C., Zakoian, J.-M.: GARCH Models: Structure, Statistical Inference and Financial Applications. Wiley, New York (2019)

    MATH  Google Scholar 

  • Gaillard, P., Goude, Y., Nedellec, R.: Additive models and robust aggregation for GEFCom2014 probabilistic electric load and electricity price forecasting. Int. J. Forecast. 32(3), 1038–1050 (2016)

    Google Scholar 

  • Garnier, R.: Simulations for penalized estimation of VAR with law rank. https://github.com/garnier94/Simulation_low_rank_VAR1 (2019)

  • Ge, R., Jin, C., Zheng, Y.: No spurious local minima in nonconvex low rank problems: a unified geometric analysis. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1233–1242. JMLR. org (2017)

  • Giannone, D., Lenza, M., Primiceri, G.E.: Prior selection for vector autoregressions. Rev. Econ. Stat. 97(2), 436–451 (2015)

    MATH  Google Scholar 

  • Giordani, P., Pitt, M., Kohn, R.: Bayesian inference for time series state space models. In: Geweke, J., Koop, G., Van Dijk, H. (eds.) The Oxford Handbook of Bayesian Econometrics. Oxford University Press, Oxford (2011)

    Google Scholar 

  • Giraud, C., Roueff, F., Sanchez-Perez, A.: Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes. Ann. Stat. 43(6), 2412–2450 (2015)

    MathSciNet  MATH  Google Scholar 

  • Hallin, M., Lippi, M.: Factor models in high-dimensional time series—a time-domain approach. Stoch. Process. Appl. 123(7), 2678–2695 (2013)

    MathSciNet  MATH  Google Scholar 

  • Hang, H., Steinwart, I.: Fast learning from \(\alpha \)-mixing observations. J. Multivar. Anal. 127, 184–199 (2014)

    MathSciNet  MATH  Google Scholar 

  • Izenman, A.J.: Reduced-rank regression for the multivariate linear model. J. Multivar. Anal. 5(2), 248–264 (1975)

    MathSciNet  MATH  Google Scholar 

  • Ji, S., Ye, J.: An accelerated gradient method for trace norm minimization. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 457–464. ACM (2009)

  • Jochmann, M., Koop, G., Leon-Gonzalez, R., Strachan, R.W.: Stochastic search variable selection in vector error correction models with an application to a model of the UK macroeconomy. J. Appl. Econom. 28(1), 62–81 (2013)

    MathSciNet  Google Scholar 

  • Klopp, O., Lounici, K., Tsybakov, A.B.: Robust matrix completion. Probab. Theory Relat. Fields 169(1–2), 523–564 (2017)

    MathSciNet  MATH  Google Scholar 

  • Klopp, O., Lu, Y., Tsybakov, A.B., Zhou, H.H.: Structured matrix estimation and completion. Bernoulli 25(4B), 3883–3911 (2019)

    MathSciNet  MATH  Google Scholar 

  • Koltchinskii, V., Lounici, K., Tsybakov, A.B.: Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Stat. 39(5), 2302–2329 (2011)

    MathSciNet  MATH  Google Scholar 

  • Koop, G., Potter, S.: Forecasting in dynamic factor models using Bayesian model averaging. Econom. J. 7(2), 550–565 (2004)

    MathSciNet  MATH  Google Scholar 

  • Kumar, M., Patel, N.R.: Using clustering to improve sales forecasts in retail merchandising. Ann. Oper. Res. 174(1), 33–46 (2010)

    MathSciNet  MATH  Google Scholar 

  • Kuznetsov, V., Mohri, M.: Learning theory and algorithms for forecasting non-stationary time series. In: Advances in Neural Information Processing Systems, pp. 541–549 (2015)

  • Lam, C., Yao, Q.: Factor modeling for high-dimensional time series: inference for the number of factors. Ann. Stat. 40(2), 694–726 (2012)

    MathSciNet  MATH  Google Scholar 

  • Lam, C., Yao, Q., Bathia, N.: Estimation of latent factors for high-dimensional time series. Biometrika 98(4), 901–918 (2011)

    MathSciNet  MATH  Google Scholar 

  • Lippi, M., Bertini, M., Frasconi, P.: Short-term traffic flow forecasting: an experimental comparison of time-series analysis and supervised learning. IEEE Trans. Intell. Transp. Syst. 14(2), 871–882 (2013)

    Google Scholar 

  • Lipton, Z.C., Kale, D.C., Wetzel, R.: Modeling missing data in clinical time series with RNNs. In: Machine Learning for Healthcare, vol. 56 (2016)

  • London, B., Huang, B., Taskar, B., Getoor, L.: PAC-Bayesian collective stability. In: Artificial Intelligence and Statistics, pp. 585–594 (2014)

  • McDonald, D.J., Shalizi, C.R., Schervish, M.: Nonparametric risk bounds for time-series forecasting. J. Mach. Learn. Res. 18(32), 1–40 (2017)

    MathSciNet  MATH  Google Scholar 

  • Meir, R.: Nonparametric time series prediction through adaptive model selection. Mach. Learn. 39(1), 5–34 (2000)

    MathSciNet  MATH  Google Scholar 

  • Moridomi, K., Hatano, K., Takimoto, E.: Tighter generalization bounds for matrix completion via factorization into constrained matrices. IEICE Trans. Inf. Syst. 101(8), 1997–2004 (2018)

    Google Scholar 

  • Negahban, S., Wainwright, M.J.: Estimation of (near) low-rank matrices with noise and high-dimensional scaling. Ann. Stat. 39(2), 1069–1097 (2011)

    MathSciNet  MATH  Google Scholar 

  • Poignard, B.: Sparse multivariate ARCH models: finite sample properties. https://arxiv.org/pdf/1808.05352v1.pdf (2018)

  • Purser, A., Bergmann, M., Lundälv, T., Ontrup, J., Nattkemper, T.W.: Use of machine-learning algorithms for the automated detection of cold-water coral habitats: a pilot study. Mar. Ecol. Prog. Ser. 397, 241–251 (2009)

    Google Scholar 

  • Saha, A., Sindhwani, V.: Learning evolving and emerging topics in social media: a dynamic NMF approach with temporal regularization. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, pp. 693–702. ACM (2012)

  • Shalizi, C., Kontorovich, A.: Predictive PAC learning and process decompositions. In: Advances in Neural Information Processing Systems, pp. 1619–1627 (2013)

  • Steinwart, I., Christmann, A.: Fast learning from non-iid observations. In: Advances in Neural Information Processing Systems, pp. 1768–1776 (2009)

  • Steinwart, I., Hush, D., Scovel, C.: Learning from dependent observations. J. Multivar. Anal. 100(1), 175–194 (2009)

    MathSciNet  MATH  Google Scholar 

  • Suzuki, T.: Convergence rate of Bayesian tensor estimator and its minimax optimality. In: International Conference on Machine Learning, pp. 1273–1282 (2015)

  • Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Advanced Intelligence Project, RIKEN, 1-4-1 Nihonbashi Chuo-ku, Tokyo, Japan

    Pierre Alquier

  2. CIMFAV, Universidad de Valparaíso, General Cruz 222, Valparaiso, Chile

    Karine Bertin & Paul Doukhan

  3. Laboratoire AGM UMR 8088, Université Paris Seine, 2 Bd. Adolphe Chauvin, 95000, Cergy-Pontoise, France

    Paul Doukhan & Rémy Garnier

  4. CDiscount, 120 Quai de Bacalan, 33000, Bordeaux, France

    Rémy Garnier

Authors
  1. Pierre Alquier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karine Bertin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Doukhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rémy Garnier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Doukhan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

P. Alquier: This author gratefully acknowledges financial support from the research programme New Challenges for New Data from LCL and GENES, hosted by the Fondation du Risque and from Labex ECODEC (ANR-11-LABEX-0047). P. Doukhan: The work of the second and the third authors has been developed within the MME-DII center of excellence (ANR-11-LABEX-0023-01) and with the help of PAI-CONICYT MEC \(\hbox {N}^\circ \) 80170072. The authors have been supported by Fondecyt Project 1171335 and Mathamsud 18-MATH-07.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alquier, P., Bertin, K., Doukhan, P. et al. High-dimensional VAR with low-rank transition. Stat Comput 30, 1139–1153 (2020). https://doi.org/10.1007/s11222-020-09929-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-020-09929-7

Keywords

Search

Navigation