International Journal of Computer Vision

, Volume 70, Issue 3, pp 241–255 | Cite as

Splines in Higher Order TV Regularization

  • Gabriele Steidl
  • Stephan Didas
  • Julia Neumann


Splines play an important role as solutions of various interpolation and approximation problems that minimize special functionals in some smoothness spaces. In this paper, we show in a strictly discrete setting that splines of degree m−1 solve also a minimization problem with quadratic data term and m-th order total variation (TV) regularization term. In contrast to problems with quadratic regularization terms involving m-th order derivatives, the spline knots are not known in advance but depend on the input data and the regularization parameter λ. More precisely, the spline knots are determined by the contact points of the m–th discrete antiderivative of the solution with the tube of width 2λ around the m-th discrete antiderivative of the input data. We point out that the dual formulation of our minimization problem can be considered as support vector regression problem in the discrete counterpart of the Sobolev space W 2,0 m . From this point of view, the solution of our minimization problem has a sparse representation in terms of discrete fundamental splines.


higher order TV regularization splines support vector regression Legendre-Fenchel dualization taut-string algorithm 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Chambolle, A. 2004. An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision, (20):89–97.MathSciNetCrossRefGoogle Scholar
  2. Chambolle, A. and Lions, P.-L. 1997 Image recovery via total variation minimization and related problems. Numerische Mathematik, 76:167–188.zbMATHMathSciNetCrossRefGoogle Scholar
  3. Chan, R. H., Ho, C. W., and Nikolova, M. Salt-and-pepper noise removal by median noise detectors and detail preserving regularization. IEEE Transactions on Image Processing, page to appear.Google Scholar
  4. Chan, T. F., Golub, G. H., and Mulet, P. 1999. A nonlinear primal—dual method for total-variation based image restoration. SIAM Journal on Scientific Computing, 20(6):1964–1977.zbMATHMathSciNetCrossRefGoogle Scholar
  5. Chan, T. F., Marquina, A., and Mulet, P. 2000. High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 22(2):503–516.zbMATHMathSciNetCrossRefGoogle Scholar
  6. Davies, P. L. and Kovac, A. 2001. Local extremes, runs, strings and multiresolution. Annals of Statistics, 29:1–65.zbMATHMathSciNetCrossRefGoogle Scholar
  7. Didas, S. 2004. Higher order variational methods for noise removal in signals and images. Diplomarbeit, Universität des Saarlandes.Google Scholar
  8. Duchon, J. 1997. Splines minimizing rotation-invariant seminorms in sobolev spaces. In Constructive Theory of Functions of Several Variables, pages 85–100, Berlin, Springer—Verlag.Google Scholar
  9. Girosi, F. 1998. An equivalence between sparse approximation and support vector machines. Neural computation, 10(6):1455–1480.CrossRefGoogle Scholar
  10. Hinterberger, W., Hintermüller, M., Kunisch K., von Oehsen M., and Scherzer, O. 2003. Tube methods for BV regularization. Journal of Mathematical Imaging and Vision, 19:223–238.CrossRefGoogle Scholar
  11. Hinterberger, W. and Scherzer, O. 2003. Variational methods on the space of functions of bounded Hessian for convexification and denoising. Technical report, University of Innsbruck, Austria.Google Scholar
  12. Hintermüller, W. and Kunisch, K. May 2004. Total bounded variation regularization as a bilaterally constrained optimization problem. SIAM Journal on Applied Mathematics, 64(4):1311–1333.CrossRefGoogle Scholar
  13. Hyman, J. M. and Shashkov, M. J. 1997. Natural discretizations for the divergence, gradient, and curl on logically rectangular grids. Comput. Math. Appl., 33(4):81–104.zbMATHMathSciNetCrossRefGoogle Scholar
  14. Kimmeldorf, G. S. and Wahba, G. 1971. Some results on Tchebycheffian spline functions. J. Anal. Appl., 33:82–95.CrossRefGoogle Scholar
  15. Lysaker, M., Lundervold, A., and Tai, X. 2003. Noise removal using fourth-order partial differential equations with applications to medical magnetic resonance images in space and time. IEEE Transactions on Image Processing, 12(12):1579–1590.CrossRefGoogle Scholar
  16. Mammen, E. and van de Geer, S. 1997. Locally adaptive regression splines. Annals of Statistics, 25(1):387–413.zbMATHMathSciNetCrossRefGoogle Scholar
  17. Mangasarian, O. L. and Schumaker, L. L. 1971. Mangasarian, O. L. and Schumaker, L. L. 1971. Discrete splines via mathematical programming. SIAM Journal on Control, 9(2):174–183.zbMATHMathSciNetCrossRefGoogle Scholar
  18. Mangasarian, O. L. and Schumaker, L. L. 1973. Best summation formulae and discrete splines via mathematical programming. SIAM Journal on Numerical Analysis, 10(3):448–459.zbMATHMathSciNetCrossRefGoogle Scholar
  19. Mehrotra, S. 1992. On the implementation of a primal-dual interior point method. SIAM Journal on Optimization, 2(4):575–601.zbMATHMathSciNetCrossRefGoogle Scholar
  20. Nielsen, M., Florack, L., and Deriche, R. 1997. Regularization, scale-space and edge detection filters. Journal of Mathematical Imaging and Vision, 7:291–307.MathSciNetCrossRefGoogle Scholar
  21. Nikolova, M. 2004. A variational approach to remove outliers and impulse noise. Journal of Mathematical Imaging and Vision, 20:99–120.MathSciNetCrossRefGoogle Scholar
  22. Rockafellar, R. T. 1970. Convex Analysis. Princeton University Press, Princeton.Google Scholar
  23. Rudin, L. I., Osher S., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Physica D, 60:259–268.zbMATHCrossRefGoogle Scholar
  24. Schnörr, C. 1998. A study of a convex variational diffusion approach for image segmentation and feature extraction. Journal of Mathematical Imaging and Vision, 8(3):271–292.zbMATHMathSciNetCrossRefGoogle Scholar
  25. Steidl, G. 2006. A note on the dual treatment of higher order regularization functionals. Computing, 76:135–148.zbMATHMathSciNetCrossRefGoogle Scholar
  26. Steidl, G., Didas, S., and Neumann, J. 2005. Relations between higher order TV regularization and support vector regression. In R. Kimmel, N. Sochen, and J. Weickert, editors, Scale-Space and PDE Methods in Computer Vision, volume 3459 of Lecture Notes in Computer Science, pages 515–527. Springer, Berlin.Google Scholar
  27. Steidl, G., Weickert, J., Brox, T., Mrázek, P., and Welk, M. 2004. On the equivalence of soft wavelet shrinkage, total variation diffusion, total variation regularization, and SIDEs. SIAM Journal on Numerical Analysis, 42(2):686–713.zbMATHMathSciNetCrossRefGoogle Scholar
  28. Unser, M. and Blu, T. 2005. Generalized smoothing splines and the optimal discretization of the Wiener filter. IEEE Transactions on Signal Processing, 53(6):2146–2159.MathSciNetCrossRefGoogle Scholar
  29. Vapnik, V. N. 1998. Statistical Learning Theory. John Wiley and Sons, Inc.Google Scholar
  30. Vogel, C. R. 2002. Computational Methods for Inverse Problems. SIAM, Philadelphia.Google Scholar
  31. Wahba, G. 1990. Spline Models for Observational Data. SIAM, Philadelphia.Google Scholar
  32. Welk, M., Weickert, J., and Steidl, G. 2005. A four-pixel scheme for singular differential equations. In R. Kimmel, N. Sochen, and J. Weickert, editors, Scale-Space and PDE Methods in Computer Vision, Lecture Notes in Computer Science. Springer, Berlin, to appear.Google Scholar
  33. Yip, A. M. and Park, F. 2003. Solution dynamics, causality, and critical behaviour of the regularization parameter in total variation denoising problems. UCLA Report.Google Scholar
  34. You, Y.-L. and Kaveh, M. 2000. Fourth-order partial differential equations for noise removal. IEEE Transactions on Image Processing, 9(10):1723–1730.zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Faculty of Mathematics and Computer ScienceUniversity of MannheimMannheimGermany
  2. 2.Mathematical Image Analysis Group, Faculty of Mathematics and Computer ScienceSaarland UniversitySaarbrückenGermany

Personalised recommendations