On the search of the shape parameter in radial basis functions using univariate global optimization methods

  • R. Cavoretto
  • A. De Rossi
  • M. S. Mukhametzhanov
  • Ya. D. SergeyevEmail author


In this paper we consider the problem of finding an optimal value of the shape parameter in radial basis function interpolation. In particular, we propose the use of a leave-one-out cross validation (LOOCV) technique combined with univariate global optimization methods, which involve strategies of global optimization with pessimistic improvement (GOPI) and global optimization with optimistic improvement (GOOI). This choice is carried out to overcome serious issues of commonly used optimization routines that sometimes result in shape parameter values are not globally optimal. New locally-biased versions of geometric and information Lipschitz global optimization algorithms are presented. Numerical experiments and applications to real-world problems show a promising performance and efficacy of the new algorithms, called LOOCV-GOPI and LOOCV-GOOI, in comparison with their direct competitors.


RBF interpolation Shape parameter Global optimization 

Mathematics Subject Classification

65D05 65D15 90C26 65K05 



  1. 1.
    Allasia, G., Cavoretto, R., De Rossi, A.: Lobachevsky spline functions and interpolation to scattered data. Comput. Appl. Math. 32, 71–87 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Barkalov, K., Gergel, V., Lebedev, I.: Solving global optimization problems on GPU cluster. In: Simos, T.E. (ed.) AIP Conference Proceedings, vol. 1738 (400006) (2016)Google Scholar
  3. 3.
    Barkalov, K., Strongin, R.: Solving a set of global optimization problems by the parallel technique with uniform convergence. J. Global Optim. 71(1), 21–36 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Biazar, B., Hosami, M.: An interval for the shape parameter in radial basis function approximation. Appl. Math. Comput. 315, 131–149 (2017)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Buhmann, M.D.: Radial Basis Functions: Theory and Implementation. Cambridge Monographs on Applied and Computational Mathematics, vol. 12. Cambridge University Press, Cambridge (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    Calvin, J., Žilinskas, A.: On the convergence of the P-algorithm for one-dimensional global optimization of smooth functions. J. Optim. Theory Appl. 102(3), 479–495 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Carlson, R.E., Foley, T.A.: Interpolation of track data with radial basis methods. Comput. Math. Appl. 24, 27–34 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Cavoretto, R., De Rossi, A.: A trivariate interpolation algorithm using a cube-partition searching procedure. SIAM J. Sci. Comput. 37, A1891–A1908 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Cavoretto, R., De Rossi, A., Perracchione, E.: Optimal selection of local approximants in RBF-PU interpolation. J. Sci. Comput. 74, 1–22 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Cavoretto, R., Schneider, T., Zulian, P.: OpenCL based parallel algorithm for RBF-PUM interpolation. J. Sci. Comput. 74, 267–289 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Cheng, A.D.: Multiquadric and its shape parameter—a numerical investigation of error estimate, condition number, and round-off error by arbitrary precision computation. Eng. Anal. Bound. Elem. 36, 220–239 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    De Marchi, S., Santin, G.: Fast computation of orthonormal basis for rbf spaces through krylov space methods. BIT Numer. Math. 55, 949–966 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Fasshauer, G., McCourt, M.: Kernel-Based Approximation Methods Using Matlab, Interdisciplinary Mathematical Sciences, vol. 19. World Scientific, Singapore (2015)zbMATHGoogle Scholar
  14. 14.
    Fasshauer, G.E.: Meshfree Approximation Methods with Matlab, Interdisciplinary Mathematical Sciences, vol. 6. World Scientific, Singapore (2007)zbMATHCrossRefGoogle Scholar
  15. 15.
    Fasshauer, G.E.: Positive definite kernels: past, present and future. Dolomit. Res. Notes Approx. 4, 21–63 (2011)Google Scholar
  16. 16.
    Fasshauer, G.E., Zhang, J.G.: On choosing “optimal” shape parameters for RBF approximation. Numer. Algorithms 45, 345–368 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Floudas, C.A., Pardalos, P.M. (eds.): Encyclopedia of Optimization, vol. 6, 2nd edn. Springer, New York (2009)zbMATHGoogle Scholar
  18. 18.
    Fornberg, B., Larsson, E., Flyer, N.: Stable computations with Gaussian radial basis functions. SIAM J. Sci. Comput. 33, 869–892 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Fowkes, J.M., Gould, N.I.M., Farmer, C.L.: A branch and bound algorithm for the global optimization of Hessian Lipschitz continuous functions. J. Global Optim. 56, 1791–1815 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Gergel, V.P., Grishagin, V.A., Israfilov, R.A.: Local tuning in nested scheme of global optimization. Proc. Comput. Sci. 51, 865–874 (2015)CrossRefGoogle Scholar
  21. 21.
    Gergel, V.P., Kuzmin, M.I., Solovyov, N.A., Grishagin, V.A.: Recognition of surface defects of cold-rolling sheets based on method of localities. Int. Rev. Autom. Control 8(1), 51–55 (2015)Google Scholar
  22. 22.
    Gillard, J.W., Zhigljavsky, A.A.: Stochastic algorithms for solving structured low-rank matrix approximation problems. Commun. Nonlinear Sci. Numer. Simul. 21(1–3), 70–88 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Golbabai, A., Mohebianfar, E., Rabiei, H.: On the new variable shape parameter strategies for radial basis functions. Comput. Appl. Math. 34, 691–704 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Grishagin, V.A., Israfilov, R.A., Sergeyev, Y.D.: Convergence conditions and numerical comparison of global optimization methods based on dimensionality reduction schemes. Appl. Math. Comput. 318, 270–280 (2018)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Kvasov, D.E., Mukhametzhanov, M.S.: Metaheuristic versus deterministic global optimization algorithms: the univariate case. Appl. Math. Comput. 318, 245–259 (2018)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Kvasov, D.E., Pizzuti, C., Sergeyev, Y.D.: Local tuning and partition strategies for diagonal GO methods. Numer. Math. 94(1), 93–106 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Kvasov, D.E., Sergeyev, Y.D.: Deterministic approaches for solving practical black-box global optimization problems. Adv. Eng. Softw. 80, 58–66 (2015)CrossRefGoogle Scholar
  28. 28.
    Lera, D., Sergeyev, Y.D.: Acceleration of univariate global optimization algorithms working with Lipschitz functions and Lipschitz first derivatives. SIAM J. Optim. 23(1), 508–529 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Luh, L.T.: The shape parameter in the Gaussian function. Comput. Math. Appl. 63, 687–694 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Paulavičius, R., Žilinskas, J.: Simplicial Global Optimization. Springer Briefs in Optimization. Springer, New York (2014)zbMATHCrossRefGoogle Scholar
  31. 31.
    Pazouki, M., Schaback, R.: Bases for kernel-based spaces. J. Comput. Appl. Math. 236, 575–588 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Piyavskij, S.A.: An algorithm for finding the absolute extremum of a function. USSR Comput. Math. Math. Phys. 12(4), 57–67 (1972). In Russian: Zh. Vychisl. Mat. Mat. Fiz., 12(4) (1972), pp. 888–896Google Scholar
  33. 33.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2014)Google Scholar
  34. 34.
    Rippa, S.: An algorithm for selecting a good value for the parameter \(c\) in radial basis function interpolation. Adv. Comput. Math. 11, 193–210 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Sanyasiraju, Y., Satyanarayana, C.: On optimization of the RBF shape parameter in a grid-free local scheme for convection dominated problems over non-uniform centers. Appl. Math. Model. 37, 7245–7272 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Scheuerer, M.: An alternative procedure for selecting a good value for the parameter c in RBF-interpolation. Adv. Comput. Math. 34, 105–126 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  37. 37.
    Sergeyev, Y.D.: An information global optimization algorithm with local tuning. SIAM J. Optim. 5(4), 858–870 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  38. 38.
    Sergeyev, Y.D.: A one-dimensional deterministic global minimization algorithm. Comput. Math. Math. Phys. 35(5), 705–717 (1995)MathSciNetGoogle Scholar
  39. 39.
    Sergeyev, Y.D.: Global one-dimensional optimization using smooth auxiliary functions. Math. Program. 81(1), 127–146 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Sergeyev, Y.D., Kvasov, D.E., Mukhametzhanov, M.S.: On the least-squares fitting of data by sinusoids. In: Pardalos, P.M., Zhigljavsky, A., Žilinskas, J. (eds.) Advances in Stochastic and Deterministic Global Optimization, pp. 209–226. Springer, Berlin (2016)CrossRefGoogle Scholar
  41. 41.
    Sergeyev, Y.D., Kvasov, D.E., Mukhametzhanov, M.S.: Operational zones for comparing metaheuristic and deterministic one-dimensional global optimization algorithms. Math. Comput. Simul. 141, 96–109 (2017)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Sergeyev, Y.D., Kvasov, D.E., Mukhametzhanov, M.S.: On strong homogeneity of a class of global optimization algorithms working with infinite and infinitesimal scales. Commun. Nonlinear Sci. Numer. Simul. 59, 319–330 (2018)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Sergeyev, Y.D., Kvasov, D.E., Mukhametzhanov, M.S.: On the efficiency of nature-inspired metaheuristics in expensive global optimization with limited budget. Sci. Rep. 8, 453 (2018)CrossRefGoogle Scholar
  44. 44.
    Sergeyev, Y.D., Kvasov, D.E., Mukhametzhanov, M.S., De Franco, A.: Acceleration techniques in the univariate Lipschitz global optimization. In: Sergeyev, Y.D., Kvasov, D.E., Dell’Accio F., Mukhametzhanov, M.S. (eds.) AIP Conference Proceedings, vol. 1776 (090051) (2016)Google Scholar
  45. 45.
    Sergeyev, Y.D., Mukhametzhanov, M.S., Kvasov, D.E., Lera, D.: Derivative-free local tuning and local improvement techniques embedded in the univariate global optimization. J. Optim. Theory Appl. 171(1), 186–208 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Strongin, R.G.: Numerical Methods in Multiextremal Problems (Information-Statistical Algorithms). Nauka, Moscow (1978). In RussianGoogle Scholar
  47. 47.
    Strongin, R.G., Sergeyev, Y.D.: Global optimization: fractal approach and non-redundant parallelism. J. Global Optim. 27(1), 25–50 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Trahan, C.J., Wyatt, R.W.: Radial basis function interpolation in the quantum trajectory method: optimization of the multi-quadric shape parameter. J. Comput. Phys. 185, 27–49 (2003)zbMATHCrossRefGoogle Scholar
  49. 49.
    Uddin, M.: On the selection of a good value of shape parameter in solving time-dependent partial differential equations using RBF approximation method. Appl. Math. Model. 38, 135–144 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Wahba, G.: Spline Models for Observational Data, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990)Google Scholar
  51. 51.
    Wendland, H.: Scattered Data Approximation. Cambridge Monographs on Applied and Computational Mathematics, vol. 17. Cambridge University Press, Cambridge (2005)zbMATHGoogle Scholar
  52. 52.
    Yao, G., Duo, J., Chen, C., Shen, L.: Implicit local radial basis function interpolations based on function values. Appl. Math. Comput. 265, 91–102 (2015)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Zhigljavsky, A.A., Žilinskas, A.: Stochastic Global Optimization. Springer, New York (2008)zbMATHGoogle Scholar
  54. 54.
    Zhigljavsky, A.A., Žilinskas, A.: Selection of a covariance function for a Gaussian random field aimed for modeling global optimization problems. Optim. Lett. 13, 249–259 (2019)MathSciNetzbMATHCrossRefGoogle Scholar
  55. 55.
    Žilinskas, A.: On similarities between two models of global optimization: statistical models and radial basis functions. J. Global Optim. 48(1), 173–182 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Žilinskas, A., Zhigljavsky, A.A.: Stochastic global optimization: a review on the occasion of 25 years of Informatica. Informatica 27(2), 229–256 (2016)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • R. Cavoretto
    • 1
  • A. De Rossi
    • 1
  • M. S. Mukhametzhanov
    • 2
    • 3
  • Ya. D. Sergeyev
    • 2
    • 3
    Email author
  1. 1.Department of Mathematics “Giuseppe Peano”University of TorinoTorinoItaly
  2. 2.DIMESUniversity of CalabriaRendeItaly
  3. 3.ITMMLobachevsky Nizhni Novgorod State UniversityNizhni NovgorodRussia

Personalised recommendations