Abstract
Establishing the convergence of splines can be cast as a variational problem which is amenable to a \(\Gamma \)-convergence approach. We consider the case in which the regularization coefficient scales with the number of observations, n, as \(\lambda _n=n^{-p}\). Using standard theorems from the \(\Gamma \)-convergence literature, we prove that the general spline model is consistent in that estimators converge in a sense slightly weaker than weak convergence in probability for \(p\le \frac{1}{2}\). Without further assumptions, we show this rate is sharp. This differs from rates for strong convergence using Hilbert scales where one can often choose \(p>\frac{1}{2}\).
Similar content being viewed by others
References
Aerts, M., Claeskens, G., Wand, M. P. (2002). Some theory for penalized spline generalized additive models. Journal of Statistical Planning and Inference, 103(1–2), 455–470.
Agapiou, S., Larsson, S., Stuart, A. M. (2013). Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems. Stochastic Processes and their Applications, 123(10), 3828–3860.
Bissantz, N., Hohage, T., Munk, A. (2004). Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise. Inverse Problems, 20(6), 1773–1789.
Bissantz, N., Hohage, T., Munk, A., Ruymgaart, F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM Journal on Numerical Analysis, 45(6), 2610–2636.
Bogachev, V. I. (1998). Gaussian measures. Providence: The American Mathematical Society.
Braides, A. (2002). \(\Gamma \) -convergence for beginners. Oxford: Oxford University Press.
Brown, L. D., Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. The Annals of Statistics, 24(6), 2384–2398.
Carroll, R. J., Van Rooij, A. C. M., Ruymgaart, F. H. (1991). Theoretical aspects of ill-posed problems in statistics. Acta Applicandae Mathematica, 24(2), 113–140.
Claeskens, G., Krivobokova, T., Opsomer, J. D. (2009). Asymptotic properties of penalized spline estimators. Biometrika, 96(3), 529–544.
Cox, D. D. (1983). Asymptotics for \(M\)-type smoothing splines. The Annals of Statistics, 11(2), 530–551.
Cox, D. D. (1988). Approximation of method of regularization estimators. The Annals of Statistics, 16(2), 694–712.
Craven, P., Wahba, G. (1979). Smoothing noisy data with spline functions. Numerische Mathematik, 31(4), 377–403.
Dudley, R. M. (2002). Real analysis and probability. Cambridge: Cambridge University Press.
Eilers, P. H. C., Marx, B. D. (1996). Flexible smoothing with B-splines and penalties. Statistical Science, 11(2), 89–121.
Feinberg, E. A., Kasyanov, P. O., Zadoianchuk, N. V. (2014). Fatou’s lemma for weakly converging probabilities. Theory of Probability & Its Applications, 58(4), 683–689.
Goldenshluger, A., Pereverzev, S. V. (2000). Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probability Theory and Related Fields, 118(2), 169–186.
Hall, P., Opsomer, J. D. (2005). Theory for penalised spline regression. Biometrika, 92(1), 105–118.
Hastie, T. J., Tibshirani, R. J. (1990). Generalized additive models. Boca Raton: Chapman and Hall.
Hurvich, C. M., Simonoff, J. S., Tsai, C.-L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 60(2), 271–293.
Kauermann, G., Krivobokova, T., Fahrmeir, L. (2009). Some asymptotic results on generalized penalized spline smoothing. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2), 487–503.
Kimeldorf, G. S., Wahba, G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2), 495–502.
Kimeldorf, G. S., Wahba, G. (1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications, 33(1), 82–95.
Kou, S. C., Efron, B. (2002). Smoothers and the \(C_p\), generalized maximum likelihood, and extended exponential criteria: A geometric approach. Journal of the American Statistical Association, 97(459), 766–782.
Lai, M.-J., Wang, L. (2013). Bivariate penalized splines for regression. Statistica Sinica, 23, 1399–1417.
Li, K.-C. (1987). Asymptotic optimality for \(C_p\), \(C_L\), cross-validation and generalized cross-validation: Discrete index set. The Annals of Statistics, 15(3), 958–975.
Li, Y., Ruppert, D. (2008). On the asymptotics of penalized splines. Biometrika, 95(2), 415–436.
Lukas, M. A. (2006). Robust generalized cross-validation for choosing the regularization parameter. Inverse Problems, 22(5), 1883–1902.
Mair, B. A., Ruymgaart, F. H. (1996). Statistical inverse estimation in Hilbert scales. SIAM Journal on Applied Mathematics, 56(5), 1424–1444.
Mallows, C. L. (1973). Some comments on \(C_P\). Technometrics, 15(4), 661–675.
Nychka, D. W., Cox, D. D. (1989). Convergence rates for regularized solutions of integral equations from discrete noisy data. The Annals of Statistics, 17(2), 556–572.
Ragozin, D. L. (1983). Error bounds for derivative estimates based on spline smoothing of exact or noisy data. Journal of Approximation Theory, 37(4), 335–355.
Sakamoto, Y., Ishiguro, M., Kitagawa, G. (1986). Akaike information criterion statistics. Tokyo: KTK Scientific Publishers (KTK).
Shen, J., Wang, X. (2011). Estimation of monotone functions via P-splines: A constrained dynamical optimization approach. SIAM Journal on Control and Optimization, 49(2), 646–671.
Silverman, B. W. (1984). Spline smoothing: The equivalent variable kernel method. The Annals of Statistics, 12(3), 898–916.
Speckman, P. L. (1985). Spline smoothing and optimal rates of convergence in nonparametric regression models. The Annals of Statistics, 13(3), 970–983.
Speckman, P. L., Sun, D. (2001). Asymptotic properties of smoothing parameter selection in spline smoothing, Technical report, Department of Statistics, University of Missouri. http://www.stat.missouri.edu/~speckman/pub.html.
Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 10(4), 1040–1053.
Thorpe, M., Johansen, A. M. (2016). Convergence and rates for fixed-interval multiple-track smoothing using \(k\)-means type optimization. Electronic Journal of Statistics, 10(2), 3693–3722.
Thorpe, M., Theil, F., Johansen, A. M., Cade, N. (2015). Convergence of the \(k\)-means minimization problem using \(\Gamma \)-convergence. SIAM Journal on Applied Mathematics, 75(6), 2444–2474.
Utreras, F. I. (1981). Optimal smoothing of noisy data using spline functions. SIAM Journal on Scientific and Statistical Computing, 2(3), 349–362.
Utreras, F. I. (1983). Natural spline functions, their associated eigenvalue problem. Numerische Mathematik, 42(1), 107–117.
Utreras, F. I. (1985). Smoothing noisy data under monotonicity constraints existence, characterization and convergence rates. Numerische Mathematik, 47(4), 611–625.
Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. The Annals of Statistics, 13(4), 1378–1402.
Wahba, G. (1990). Spline models for observational data. Philadelphia: Society for Industrial and Applied Mathematics (SIAM).
Wahba, G., Wold, S. (1975). A completely automatic French curve: Fitting spline functions by cross-validation. Communications in Statistics, 4(1), 1–17.
Wand, M. P. (1999). On the optimal amount of smoothing in penalised spline regression. Biometrika, 86(4), 936–940.
Wang, X., Shen, J., Ruppert, D. (2011). On the asymptotics of penalized spline smoothing. Electronic Journal of Statistics, 5, 1–17.
Xiao, L., Li, Y., Apanasovich, T. V., Ruppert, D. (2012). Local asymptotics of P-splines. arXiv:1201.0708
Xiao, L., Li, Y., Ruppert, D. (2013). Fast bivariate P-splines: The sandwich smoother. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75(3), 577–599.
Yoshida, T., Naito, K. (2012). Asymptotics for penalized additive \(B\)-spline regression. Journal of the Japan Statistical Society, 42(1), 81–107.
Yoshida, T., Naito, K. (2014). Asymptotics for penalised splines in generalised additive models. Journal of Nonparametric Statistics, 26(2), 269–289.
Acknowledgements
This work was carried out whilst MT was part of MASDOC at the University of Warwick and supported by an EPSRC Industrial CASE Award Ph.D. Studentship with Selex ES Ltd.
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Thorpe, M., Johansen, A.M. Pointwise convergence in probability of general smoothing splines. Ann Inst Stat Math 70, 717–744 (2018). https://doi.org/10.1007/s10463-017-0609-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-017-0609-x