Advertisement

Non-asymptotic Confidence Regions for Regularized Linear Regression Estimates

  • Balázs Csanád CsájiEmail author
Conference paper
Part of the Mathematics in Industry book series (MATHINDUSTRY, volume 30)

Abstract

Building confidence regions for regression models is of high importance, for example, they can be used for uncertainty quantification and are also fundamental for robust optimization. In practice, these regions are often computed from the asymptotic distributions, which however only lead to heuristic confidence sets. Sign-Perturbed Sums (SPS) is a resampling method which can construct exact, non-asymptotic, distribution-free confidence regions under very mild statistical assumptions. In its standard form, the SPS regions are built around the least-squares estimate of linear regression problems, and have favorable properties, such as they are star convex, strongly consistent, and have efficient ellipsoidal outer-approximations. In this paper, we extend the SPS method to regularized estimates, particularly, we present variants of SPS for ridge regression, LASSO and elastic net regularization.

Notes

Acknowledgements

This research was partially supported by the National Research, Development and Innovation Office (NKFIH), grant numbers ED_18-2-2018-0006 and KH_17 125698, and by the János Bolyai Research Fellowship of the Hung. Academy of Sciences, BO/00217/16/6.

References

  1. 1.
    Csáji, B.Cs.: Score permutation based finite sample inference for generalized autoregressive conditional heteroskedasticity (GARCH) models. In: 19th International Conference on Artificial Intelligence and Statistics (AISTATS), Cadiz, Spain, pp. 296–304 (2016)Google Scholar
  2. 2.
    Csáji, B.Cs., Weyer, E.: Closed-loop applicability of the sign-perturbed sums method. In: 54th IEEE Conference on Decision and Control, Osaka, Japan, pp. 1441–1446 (2015)Google Scholar
  3. 3.
    Csáji, B.Cs., Campi, M.C., Weyer, E.: Sign-perturbed sums: a new system identification approach for constructing exact non-asymptotic confidence regions in linear regression models. IEEE Trans. Signal Process. 63(1), 169–181 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Knight, K., Fu, W.: Asymptotics for LASSO-type estimators. Ann. Stat. 5, 1356–1378 (2000)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Kolumbán, S.: System identification in highly non-informative environment. PhD thesis, Budapest University of Technology and Economics, Hungary, and Vrije Universiteit Brussels, Belgium (2016)Google Scholar
  6. 6.
    Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice-Hall, Upper Saddle River (1999)zbMATHGoogle Scholar
  7. 7.
    Weyer, E., Campi, M.C., Csáji, B.Cs.: Asymptotic properties of SPS confidence regions. Automatica 82, 287–294 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.EPIC Centre of ExcellenceMTA SZTAKI: Institute for Computer Science and Control, Hungarian Academy of SciencesBudapestHungary

Personalised recommendations