Non-asymptotic Confidence Regions for Regularized Linear Regression Estimates
Building confidence regions for regression models is of high importance, for example, they can be used for uncertainty quantification and are also fundamental for robust optimization. In practice, these regions are often computed from the asymptotic distributions, which however only lead to heuristic confidence sets. Sign-Perturbed Sums (SPS) is a resampling method which can construct exact, non-asymptotic, distribution-free confidence regions under very mild statistical assumptions. In its standard form, the SPS regions are built around the least-squares estimate of linear regression problems, and have favorable properties, such as they are star convex, strongly consistent, and have efficient ellipsoidal outer-approximations. In this paper, we extend the SPS method to regularized estimates, particularly, we present variants of SPS for ridge regression, LASSO and elastic net regularization.
This research was partially supported by the National Research, Development and Innovation Office (NKFIH), grant numbers ED_18-2-2018-0006 and KH_17 125698, and by the János Bolyai Research Fellowship of the Hung. Academy of Sciences, BO/00217/16/6.
- 1.Csáji, B.Cs.: Score permutation based finite sample inference for generalized autoregressive conditional heteroskedasticity (GARCH) models. In: 19th International Conference on Artificial Intelligence and Statistics (AISTATS), Cadiz, Spain, pp. 296–304 (2016)Google Scholar
- 2.Csáji, B.Cs., Weyer, E.: Closed-loop applicability of the sign-perturbed sums method. In: 54th IEEE Conference on Decision and Control, Osaka, Japan, pp. 1441–1446 (2015)Google Scholar
- 5.Kolumbán, S.: System identification in highly non-informative environment. PhD thesis, Budapest University of Technology and Economics, Hungary, and Vrije Universiteit Brussels, Belgium (2016)Google Scholar