Skip to main content
SpringerLink
Log in

Pointwise convergence in probability of general smoothing splines

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

Establishing the convergence of splines can be cast as a variational problem which is amenable to a \(\Gamma \)-convergence approach. We consider the case in which the regularization coefficient scales with the number of observations, n, as \(\lambda _n=n^{-p}\). Using standard theorems from the \(\Gamma \)-convergence literature, we prove that the general spline model is consistent in that estimators converge in a sense slightly weaker than weak convergence in probability for \(p\le \frac{1}{2}\). Without further assumptions, we show this rate is sharp. This differs from rates for strong convergence using Hilbert scales where one can often choose \(p>\frac{1}{2}\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aerts, M., Claeskens, G., Wand, M. P. (2002). Some theory for penalized spline generalized additive models. Journal of Statistical Planning and Inference, 103(1–2), 455–470.

  • Agapiou, S., Larsson, S., Stuart, A. M. (2013). Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems. Stochastic Processes and their Applications, 123(10), 3828–3860.

  • Bissantz, N., Hohage, T., Munk, A. (2004). Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise. Inverse Problems, 20(6), 1773–1789.

  • Bissantz, N., Hohage, T., Munk, A., Ruymgaart, F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM Journal on Numerical Analysis, 45(6), 2610–2636.

  • Bogachev, V. I. (1998). Gaussian measures. Providence: The American Mathematical Society.

    Book  MATH  Google Scholar 

  • Braides, A. (2002). \(\Gamma \) -convergence for beginners. Oxford: Oxford University Press.

  • Brown, L. D., Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. The Annals of Statistics, 24(6), 2384–2398.

  • Carroll, R. J., Van Rooij, A. C. M., Ruymgaart, F. H. (1991). Theoretical aspects of ill-posed problems in statistics. Acta Applicandae Mathematica, 24(2), 113–140.

  • Claeskens, G., Krivobokova, T., Opsomer, J. D. (2009). Asymptotic properties of penalized spline estimators. Biometrika, 96(3), 529–544.

  • Cox, D. D. (1983). Asymptotics for \(M\)-type smoothing splines. The Annals of Statistics, 11(2), 530–551.

    Article  MathSciNet  MATH  Google Scholar 

  • Cox, D. D. (1988). Approximation of method of regularization estimators. The Annals of Statistics, 16(2), 694–712.

    Article  MathSciNet  MATH  Google Scholar 

  • Craven, P., Wahba, G. (1979). Smoothing noisy data with spline functions. Numerische Mathematik, 31(4), 377–403.

  • Dudley, R. M. (2002). Real analysis and probability. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Eilers, P. H. C., Marx, B. D. (1996). Flexible smoothing with B-splines and penalties. Statistical Science, 11(2), 89–121.

  • Feinberg, E. A., Kasyanov, P. O., Zadoianchuk, N. V. (2014). Fatou’s lemma for weakly converging probabilities. Theory of Probability & Its Applications, 58(4), 683–689.

  • Goldenshluger, A., Pereverzev, S. V. (2000). Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probability Theory and Related Fields, 118(2), 169–186.

  • Hall, P., Opsomer, J. D. (2005). Theory for penalised spline regression. Biometrika, 92(1), 105–118.

  • Hastie, T. J., Tibshirani, R. J. (1990). Generalized additive models. Boca Raton: Chapman and Hall.

  • Hurvich, C. M., Simonoff, J. S., Tsai, C.-L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 60(2), 271–293.

  • Kauermann, G., Krivobokova, T., Fahrmeir, L. (2009). Some asymptotic results on generalized penalized spline smoothing. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2), 487–503.

  • Kimeldorf, G. S., Wahba, G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2), 495–502.

  • Kimeldorf, G. S., Wahba, G. (1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications, 33(1), 82–95.

  • Kou, S. C., Efron, B. (2002). Smoothers and the \(C_p\), generalized maximum likelihood, and extended exponential criteria: A geometric approach. Journal of the American Statistical Association, 97(459), 766–782.

  • Lai, M.-J., Wang, L. (2013). Bivariate penalized splines for regression. Statistica Sinica, 23, 1399–1417.

  • Li, K.-C. (1987). Asymptotic optimality for \(C_p\), \(C_L\), cross-validation and generalized cross-validation: Discrete index set. The Annals of Statistics, 15(3), 958–975.

    Article  MathSciNet  MATH  Google Scholar 

  • Li, Y., Ruppert, D. (2008). On the asymptotics of penalized splines. Biometrika, 95(2), 415–436.

  • Lukas, M. A. (2006). Robust generalized cross-validation for choosing the regularization parameter. Inverse Problems, 22(5), 1883–1902.

    Article  MathSciNet  MATH  Google Scholar 

  • Mair, B. A., Ruymgaart, F. H. (1996). Statistical inverse estimation in Hilbert scales. SIAM Journal on Applied Mathematics, 56(5), 1424–1444.

  • Mallows, C. L. (1973). Some comments on \(C_P\). Technometrics, 15(4), 661–675.

    MATH  Google Scholar 

  • Nychka, D. W., Cox, D. D. (1989). Convergence rates for regularized solutions of integral equations from discrete noisy data. The Annals of Statistics, 17(2), 556–572.

  • Ragozin, D. L. (1983). Error bounds for derivative estimates based on spline smoothing of exact or noisy data. Journal of Approximation Theory, 37(4), 335–355.

    Article  MathSciNet  MATH  Google Scholar 

  • Sakamoto, Y., Ishiguro, M., Kitagawa, G. (1986). Akaike information criterion statistics. Tokyo: KTK Scientific Publishers (KTK).

  • Shen, J., Wang, X. (2011). Estimation of monotone functions via P-splines: A constrained dynamical optimization approach. SIAM Journal on Control and Optimization, 49(2), 646–671.

  • Silverman, B. W. (1984). Spline smoothing: The equivalent variable kernel method. The Annals of Statistics, 12(3), 898–916.

    Article  MathSciNet  MATH  Google Scholar 

  • Speckman, P. L. (1985). Spline smoothing and optimal rates of convergence in nonparametric regression models. The Annals of Statistics, 13(3), 970–983.

    Article  MathSciNet  MATH  Google Scholar 

  • Speckman, P. L., Sun, D. (2001). Asymptotic properties of smoothing parameter selection in spline smoothing, Technical report, Department of Statistics, University of Missouri. http://www.stat.missouri.edu/~speckman/pub.html.

  • Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 10(4), 1040–1053.

    Article  MathSciNet  MATH  Google Scholar 

  • Thorpe, M., Johansen, A. M. (2016). Convergence and rates for fixed-interval multiple-track smoothing using \(k\)-means type optimization. Electronic Journal of Statistics, 10(2), 3693–3722.

  • Thorpe, M., Theil, F., Johansen, A. M., Cade, N. (2015). Convergence of the \(k\)-means minimization problem using \(\Gamma \)-convergence. SIAM Journal on Applied Mathematics, 75(6), 2444–2474.

  • Utreras, F. I. (1981). Optimal smoothing of noisy data using spline functions. SIAM Journal on Scientific and Statistical Computing, 2(3), 349–362.

    Article  MathSciNet  MATH  Google Scholar 

  • Utreras, F. I. (1983). Natural spline functions, their associated eigenvalue problem. Numerische Mathematik, 42(1), 107–117.

    Article  MathSciNet  MATH  Google Scholar 

  • Utreras, F. I. (1985). Smoothing noisy data under monotonicity constraints existence, characterization and convergence rates. Numerische Mathematik, 47(4), 611–625.

    Article  MathSciNet  MATH  Google Scholar 

  • Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. The Annals of Statistics, 13(4), 1378–1402.

    Article  MathSciNet  MATH  Google Scholar 

  • Wahba, G. (1990). Spline models for observational data. Philadelphia: Society for Industrial and Applied Mathematics (SIAM).

    Book  MATH  Google Scholar 

  • Wahba, G., Wold, S. (1975). A completely automatic French curve: Fitting spline functions by cross-validation. Communications in Statistics, 4(1), 1–17.

  • Wand, M. P. (1999). On the optimal amount of smoothing in penalised spline regression. Biometrika, 86(4), 936–940.

    Article  MathSciNet  MATH  Google Scholar 

  • Wang, X., Shen, J., Ruppert, D. (2011). On the asymptotics of penalized spline smoothing. Electronic Journal of Statistics, 5, 1–17.

  • Xiao, L., Li, Y., Apanasovich, T. V., Ruppert, D. (2012). Local asymptotics of P-splines. arXiv:1201.0708

  • Xiao, L., Li, Y., Ruppert, D. (2013). Fast bivariate P-splines: The sandwich smoother. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75(3), 577–599.

  • Yoshida, T., Naito, K. (2012). Asymptotics for penalized additive \(B\)-spline regression. Journal of the Japan Statistical Society, 42(1), 81–107.

  • Yoshida, T., Naito, K. (2014). Asymptotics for penalised splines in generalised additive models. Journal of Nonparametric Statistics, 26(2), 269–289.

Download references

Acknowledgements

This work was carried out whilst MT was part of MASDOC at the University of Warwick and supported by an EPSRC Industrial CASE Award Ph.D. Studentship with Selex ES Ltd.

Author information

Authors and Affiliations

  1. Department of Mathematics, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Matthew Thorpe

  2. Department of Statistics, University of Warwick, Coventry, CV4 7AL, UK

    Adam M. Johansen

Authors
  1. Matthew Thorpe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam M. Johansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Thorpe.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thorpe, M., Johansen, A.M. Pointwise convergence in probability of general smoothing splines. Ann Inst Stat Math 70, 717–744 (2018). https://doi.org/10.1007/s10463-017-0609-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-017-0609-x

Keywords

Search

Navigation