Computational Optimization and Applications

, Volume 58, Issue 3, pp 523–541 | Cite as

Adaptively refined dynamic program for linear spline regression

  • Noam Goldberg
  • Youngdae Kim
  • Sven Leyffer
  • Thomas D. Veselka
Article

Abstract

The linear spline regression problem is to determine a piecewise linear function for estimating a set of given points while minimizing a given measure of misfit or error. This is a classical problem in computational statistics and operations research; dynamic programming was proposed as a solution technique more than 40 years ago by Bellman and Roth (J Am Stat Assoc 64:1079–1084, 1969). The algorithm requires a discretization of the solution space to define a grid of candidate breakpoints. This paper proposes an adaptive refinement scheme for the grid of candidate breakpoints in order to allow the dynamic programming method to scale for larger instances of the problem. We evaluate the quality of solutions found on small instances compared with optimal solutions determined by a novel integer programming formulation of the problem. We also consider a generalization of the linear spline regression problem to fit multiple curves that share breakpoint horizontal coordinates, and we extend our method to solve the generalized problem. Computational experiments verify that our nonuniform grid construction schemes are useful for computing high-quality solutions for both the single-curve and two-curve linear spline regression problem.

Keywords

Piecewise regression Least squares Change point detection  Dynamic programming Mixed-integer programming 

References

  1. 1.
    Aronov, B., Asanov, T., Katoh, N., Mehlhorn, K.: Polyline fitting of planar points under min-sum criteria. Int. J. Comput. Geom. Appl. 16(2&3), 97–116 (2006)CrossRefMATHGoogle Scholar
  2. 2.
    Bache, K., Lichman, M.: UCI machine learning repository. School of Information and Computer Sciences, University of California, Irvine (2013). http://archive.ics.uci.edu/ml
  3. 3.
    Bai, J., Perron, P.: Computation and analysis of multiple structural change models. J. Appl. Econom. 18, 1–22 (2003)Google Scholar
  4. 4.
    Bellman, R., Roth, R.: Curve fitting by segmented straight lines. J. Am. Stat. Assoc. 64, 1079–1084 (1969)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Ertel, J.E., Fowlkes, E.B.: Some algorithms for linear spline and piecewise multiple linear regression. J. Am. Stat. Assoc. 71(355), 640–648 (1976)CrossRefMATHGoogle Scholar
  6. 6.
    Friedman, J.H.: Multivariate adaptive regression splines. Ann. Stat. 19(1), 1–141 (1991)CrossRefMATHGoogle Scholar
  7. 7.
    Guthery, S.B.: Partition regression. J. Am. Stat. Assoc. 348, 945–947 (1974)CrossRefGoogle Scholar
  8. 8.
    Hannah, L.A., Dunson, D.B.: Multivariate convex regression with adaptive partitioning. J. Mach. Learn. Res. 14, 3261–3294 (2013)MathSciNetGoogle Scholar
  9. 9.
    Kehagias, A., Nidelkou, E., Petridis, V.: A dynamic programming segmentation procedure for hydrological and environmental time series. Stoch. Environ. Res. Risk Assess. 20(1), 77–94 (2005)MathSciNetGoogle Scholar
  10. 10.
    Toriello, A., Vielma, J.P.: Fitting piecewise linear continuous functions. Eur. J. Oper. Res. 219(1), 89–95 (2012)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Weber, G.-W., Batmaz, I., Köksal, G., Taylan, P., Yelikaya-Özkurt, F.: CMARS: a new contribution to nonparametric regression with multivariate adaptive regression splines supported by continuous optimization. Inverse Probl. Sci. Eng. 20(3), 371–400 (2012)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2014

Authors and Affiliations

  • Noam Goldberg
    • 1
  • Youngdae Kim
    • 2
  • Sven Leyffer
    • 3
  • Thomas D. Veselka
    • 4
  1. 1.Department of ManagementBar-Ilan University Ramat GanIsrael
  2. 2.Department of Computer SciencesUniversity of Wisconsin-MadisonMadisonUSA
  3. 3.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA
  4. 4.Decision and Information SciencesArgonne National LaboratoryArgonneUSA

Personalised recommendations