Journal of Global Optimization

, Volume 56, Issue 2, pp 265–277 | Cite as

A hybrid global optimization algorithm for non-linear least squares regression

  • Antanas ŽilinskasEmail author
  • Julius Žilinskas


A hybrid global optimization algorithm is proposed aimed at the class of objective functions with properties typical of the problems of non-linear least squares regression. Three components of hybridization are considered: simplicial partition of the feasible region, indicating and excluding vicinities of the main local minimizers from global search, and computing the indicated local minima by means of an efficient local descent algorithm. The performance of the algorithm is tested using a collection of non-linear least squares problems evaluated by other authors as difficult global optimization problems.


Global optimization Simplicial partition Non-linear least squares 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bjórck A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)CrossRefGoogle Scholar
  2. 2.
    Demidenko E.: Criteria for global minimum of sum of squares in nonlinear regression. Comput. Stat. Data Anal. 51(3), 1739–1753 (2006)CrossRefGoogle Scholar
  3. 3.
    Dorsey R.E., Mayer W.J.: Genetic algorithms for estimation problems with multiple optima, nondifferentiability, and other irregular features. J. Bus. Econ. Stat. 13(1), 53–66 (1995)Google Scholar
  4. 4.
    Gasimov R.: Augmented lagrangian duality and nondifferentiable optimization methods in nonconvex programming. J. Glob. Optim. 24, 187–203 (2002)CrossRefGoogle Scholar
  5. 5.
    Goffe W., Ferrier G., Rogers J.: Global optimization of statistical functions with simulated annealing. J. Econ. 60, 65–99 (1994)Google Scholar
  6. 6.
    Griffits J.D., Hunt J.G.: Vehicle headways in urban areas. Traffic Eng. Control 32, 458–462 (1991)Google Scholar
  7. 7.
    Hansen E.R.: Global Optimization Using Interval Analysis. Marcel Dekker, New York (1992)Google Scholar
  8. 8.
    Horst R., Pardalos P., Thoai N.: Introduction to Global Optimization. Kluwer, New York (1995)Google Scholar
  9. 9.
    Křivý I., Tvrdík J., Krepec R.: Stochastic algorithms in nonlinear regression. Comput. Stat. Data Anal. 33, 277–290 (2000)CrossRefGoogle Scholar
  10. 10.
    Křivý I., Tvrdík J., Misik L.: Adaptive population-based search: application to estimation of nonlinear regression parameters. Comput. Stat. Data Anal. 52(2), 713–724 (2007)CrossRefGoogle Scholar
  11. 11.
    Mathar R., Žilinskas A.: A class of test functions for global optimization. J. Glob. Optim. 5, 195–199 (1994)CrossRefGoogle Scholar
  12. 12.
    Mishra, S.: Performance of differential evolution method in least squares fitting of some typical nonlinear curves.
  13. 13.
    Nelles O.: Nonlinear System Identification From Classical Approaches to Neural Networks and Fuzzy Models. Springer, New York (2001)Google Scholar
  14. 14.
    Paulavičius R., Žilinskas J., Grothey A.: Investigation of selection strategies in branch and bound algorithm with simplicial partitions and combination of Lipschitz bounds. Optim. Lett. 4(2), 173–183 (2010)CrossRefGoogle Scholar
  15. 15.
    Pinter J.: Global Optimization in Action, Continuous and Lipshitz Optimization: Algorithms, Implementations and Applications. Kluwer, New York (1996)Google Scholar
  16. 16.
    Schittkowski K.: Data Fitting in Dynamical Systems. Kluwer, New York (2002)CrossRefGoogle Scholar
  17. 17.
    Sergeyev Y.: An information global optimization algorithm with local tuning. SIAM J. Optim. 5(4), 858–870 (1995)CrossRefGoogle Scholar
  18. 18.
    Sergeyev Y.: Efficient partitions of n-dimensional intervals in the framework of diagonal algorithms. J. Optim. Theory Appl. 124, 503–510 (2005)CrossRefGoogle Scholar
  19. 19.
    Sergeyev Y., Kvasov D.: Global search based on efficient diagonal partitions and a set of Lipschitz costants. SIAM J. Optim. 16, 910–937 (2006)CrossRefGoogle Scholar
  20. 20.
    Sergeyev Y., Pugliese P., Famularo D.: Index information algorithm with local tuning for solving multidimensional global optimization problems with multiextremal constraints. Math. Program. 96(3), 489–512 (2003)CrossRefGoogle Scholar
  21. 21.
    Strongin R., Sergeyev Y.: Global Optimization with Non-convex Constraints: Sequential and Parallel Algorithms. Kluwer, New York (2000)Google Scholar
  22. 22.
    Törn A., Žilinskas A.: Global optimization. Lect. Notes Comput. Sci. 350, 1–255 (1989)CrossRefGoogle Scholar
  23. 23.
    Tvrdík, J.: Global optimization, evolutionary algorithms and their application to computational statistics.
  24. 24.
    Velzquez L., Phillips G.N., Tapia R.A., Zhang Y.: Selective search for global optimization of zero or small residual least-squares problems: a numerical study. Comput. Optim. Appl. 20(3), 299–315 (1989)CrossRefGoogle Scholar
  25. 25.
    Žilinskas A.: Optimization of one-dimensional multimodal functions, Algorithm 133. J. R. Stat. Soc. Ser. C 23, 367–385 (1978)Google Scholar
  26. 26.
    Žilinskas A., Baronas D.: Optimization-based evaluation of concentrations in modeling the biosensor-aided measurement. Informatica 22(4), 589–600 (2011)Google Scholar
  27. 27.
    Žilinskas A., Žilinskas J.: Global optimization based on a statistical model and simplicial partitioning. Comput. Math. Appl. 44(7), 957–967 (2002). doi: 10.1016/S0898-1221(02)00206-7 CrossRefGoogle Scholar
  28. 28.
    Žilinskas A., Žilinskas J.: Interval arithmetic based optimization in nonlinear regression. Informatica 21(1), 149–158 (2010)Google Scholar
  29. 29.
    Žilinskas A., Žilinskas J.: P-algorithm based on a simplicial statistical model of multimodal functions. TOP 18(2), 396–412 (2010)CrossRefGoogle Scholar
  30. 30.
    Žilinskas J.: Branch and bound with simplicial partitions for global optimization. Math. Model. Anal. 13(1), 145–159 (2008). doi: 10.3846/1392-6292.2008.13.145-159 CrossRefGoogle Scholar
  31. 31.
    Žilinskas J., Bogle I.D.L.: Balanced random interval arithmetic in market model estimation. Eur. J. Oper. Res. 175(3), 1367–1378 (2006). doi: 10.1016/j.ejor.2005.02.013 CrossRefGoogle Scholar
  32. 32.
    Zhou W., Chen X.: Global convergence of a new hybrid gauss-newton structured BFGS method for nonlinear least squares problems. SIAM J. Optim. 20(5), 2422–2441 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2012

Authors and Affiliations

  1. 1.Institute of Mathematics and InformaticsVilnius UniversityVilniusLithuania

Personalised recommendations