Skip to main content
Log in

Smoothing noisy data with spline functions

Estimating the correct degree of smoothing by the method of generalized cross-validation

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Summary

Smoothing splines are well known to provide nice curves which smooth discrete, noisy data. We obtain a practical, effective method for estimating the optimum amount of smoothing from the data. Derivatives can be estimated from the data by differentiating the resulting (nearly) optimally smoothed spline.

We consider the modely i (t i )+ε i ,i=1, 2, ...,n,t i∈[0, 1], wheregW (m)2 ={f:f,f′, ...,f (m−1) abs. cont.,f (m)∈ℒ2[0,1]}, and the {ε i } are random errors withEε i =0,Eε i ε j 2δ ij . The error variance σ2 may be unknown. As an estimate ofg we take the solutiong n, λ to the problem: Findf∈W (m)2 to minimize\(\frac{1}{n}\sum\limits_{j = 1}^n {(f(t_j ) - y_j )^2 + \lambda \int\limits_0^1 {(f^{(m)} (u))^2 du} }\). The functiong n, λ is a smoothing polynomial spline of degree 2m−1. The parameter λ controls the tradeoff between the “roughness” of the solution, as measured by\(\int\limits_0^1 {[f^{(m)} (u)]^2 du}\), and the infidelity to the data as measured by\(\frac{1}{n}\sum\limits_{j = 1}^n {(f(t_j ) - y_j )^2 }\), and so governs the average square errorR(λ; g)=R(λ) defined by

$$R(\lambda ) = \frac{1}{n}\sum\limits_{j = 1}^n {(g_{n,\lambda } (t_j ) - g(t_j ))^2 }$$

. We provide an estimate\(\hat \lambda\), called the generalized cross-validation estimate, for the minimizer ofR(λ). The estimate\(\hat \lambda\) is the minimizer ofV(λ) defined by\(V(\lambda ) = \frac{1}{n}\parallel (I - A(\lambda ))y\parallel ^2 /\left[ {\frac{1}{n}{\text{Trace(}}I - A(\lambda ))} \right]^2\), wherey=(y 1, ...,y n)t andA(λ) is then×n matrix satisfying(g n, λ (t 1), ...,g n, λ (t n))t=A (λ) y. We prove that there exist a sequence of minimizers\(\tilde \lambda = \tilde \lambda (n)\) ofEV(λ), such that as the (regular) mesh{t i} ni=1 becomes finer,\(\mathop {\lim }\limits_{n \to \infty } ER(\tilde \lambda )/\mathop {\min }\limits_\lambda ER(\lambda ) \downarrow 1\). A Monte Carlo experiment with several smoothg's was tried withm=2,n=50 and several values of σ2, and typical values of\(R(\hat \lambda )/\mathop {\min }\limits_\lambda R(\lambda )\) were found to be in the range 1.01–1.4. The derivativeg′ ofg can be estimated by\(g'_{n,\hat \lambda } (t)\). In the Monte Carlo examples tried, the minimizer of\(R_D (\lambda ) = \frac{1}{n}\sum\limits_{j = 1}^n {(g'_{n,\lambda } (t_j ) - } g'(t_j ))\) tended to be close to the minimizer ofR(λ), so that\(\hat \lambda\) was also a good value of the smoothing parameter for estimating the derivative.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abramowitz, M., Stegun, I.: Handbook of mathematical functions with formulas, graphs, and mathematical tables. U.S. Department of Commerce, National Bureau of Standards Applied Mathematics Series No.55, pp. 803–819, 1964

    Google Scholar 

  2. Aronszajn, N.: Theory of reproducing kernels. Trans. Amer. Math. Soc.68, 337–404 (1950)

    Google Scholar 

  3. Golomb, M.: Approximation by periodic spline interpolants on uniform meshes. J. Approximation Theory1, 26–65 (1968)

    Google Scholar 

  4. Golub, G., Heath, M., Wahba, G.: Generalized cross validation as a method for choosing a good ridge parameter, to appear, Technometrics

  5. Golub, G., Reinsch, C.: Singular value decomposition and least squares solutions. Numer. Math.14, 403–420 (1970)

    Google Scholar 

  6. Hudson, H.M.: Empirical Bayes estimation. Technical Report #58, Stanford University, Department of Statistics, Stanford, Cal., 1974

    Google Scholar 

  7. Kimeldorf, G., Wahba, G.: A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Inst. Statist. Math.41, 495–502 (1970)

    Google Scholar 

  8. Mallows, C.L.: Some comments onC p. Technometrics15, 661–675 (1973)

    Google Scholar 

  9. Reinsch, C.M.: Smoothing by spline functions. Numer. Math.10, 177–183 (1967)

    Google Scholar 

  10. Reinsch, C.M.: Smoothing by spline functions, II. Numer. Math.16, 451–454 (1971)

    Google Scholar 

  11. Schoenberg, I.J.: Spline functions and the problem of graduation. Proc. Nat. Acad. Sci. (USA)52, 947–950 (1964)

    Google Scholar 

  12. Wahba, G.: Convergence rates for certain approximate solutions to first kind integral equations. J. Approximation Theory7, 167–185 (1973)

    Google Scholar 

  13. Wahba, G.: Smoothing noisy data with spline functions. Numer. Math.24, 383–393 (1975)

    Google Scholar 

  14. Wahba, G.: Practical approximate solutions to linear operator equations when the data are noisy. SIAM J. Numer. Anal.14, 651–667 (1977)

    Google Scholar 

  15. Wahba, G., Wold, S.: A completely automatic French curve: Fitting spline functions by crossvalidation. Comm. Statist.4, 1–17 (1975)

    Google Scholar 

  16. Wahba, G., Wold, S.: Periodic splines for spectral density estimation: The use of cross-validation for determining the degree of smoothing. Comm. Statist.4, 125–141 (1975)

    Google Scholar 

  17. Wahba, G.: A survey of some smoothing problems and the method of generalized cross validation for solving them. University of Wisconsin-Madison, Statistics Dept., Technical Report #457. In: Proceedings of the Conference on Applications of Statistics, Dayton, Ohio (P.R. Krishnaiah, ed.) June 14–18, 1976

  18. Wahba, G.: Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. Roy. Statist. Soc., Ser. B. To appear

Download references

Author information

Authors and Affiliations

Authors

Additional information

Research supported in part under U.S. Air Force Grant AF-AFOSR-77-3272 and by the Science Research Council (GB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Craven, P., Wahba, G. Smoothing noisy data with spline functions. Numer. Math. 31, 377–403 (1978). https://doi.org/10.1007/BF01404567

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01404567

Subject Classifications

Navigation