Advertisement

Journal of Global Optimization

, Volume 61, Issue 3, pp 429–457 | Cite as

Branching and bounding improvements for global optimization algorithms with Lipschitz continuity properties

  • Coralia Cartis
  • Jaroslav M. FowkesEmail author
  • Nicholas I. M. Gould
Article

Abstract

We present improvements to branch and bound techniques for globally optimizing functions with Lipschitz continuity properties by developing novel bounding procedures and parallelisation strategies. The bounding procedures involve nonconvex quadratic or cubic lower bounds on the objective and use estimates of the spectrum of the Hessian or derivative tensor, respectively. As the nonconvex lower bounds are only tractable if solved over Euclidean balls, we implement them in the context of a recent branch and bound algorithm (Fowkes et al. in J Glob Optim 56:1791–1815, 2013) that uses overlapping balls. Compared to the rectangular tessellations of traditional branch and bound, overlapping ball coverings result in an increased number of subproblems that need to be solved and hence makes the need for their parallelization even more stringent and challenging. We develop parallel variants based on both data- and task-parallel paradigms, which we test on an HPC cluster on standard test problems with promising results.

Keywords

Global optimization Lipschitzian optimization Parallel branch and bound Nonconvex programming 

Notes

Acknowledgments

The work of the first and second authors was supported by EPSRC grants EP/I028854/1 and NAIS EP/G036136/1 and the work of the third author by EP/I013067/1. We are also grateful to NAIS for funding computing time.

References

  1. 1.
    Alba, E., Almeida, F., Blesa, M., Cotta, C., Díaz, M., Dorta, I., Gabarró, J., León, C., Luque, G., Petit, J., Rodríguez, C., Rojas, A., Xhafa, F.: Efficient parallel LAN/WAN algorithms for optimization. The MALLBA project. Parallel Comput. 32(5–6), 415–440 (2006). doi: 10.1016/j.parco.2006.06.007 CrossRefGoogle Scholar
  2. 2.
    Ananth, G.Y., Kumar, V., Pardalos, P.M.: Parallel processing of discrete optimization problems. In: Kent, A., Williams, J. (eds.) Encyclopedia of Microcomputers, vol. 13, pp. 129–147, Dekker (1993). http://books.google.co.uk/books?id=Rx3hqGdXcooC
  3. 3.
    Baritompa, W., Cutler, A.: Accelerations for global optimization covering methods using second derivatives. J. Glob. Optim. 4(3), 329–341 (1994). doi: 10.1007/BF01098365 CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Breiman, L., Cutler, A.: A deterministic algorithm for global optimization. Math. Program. 58(1–3), 179–199 (1993). doi: 10.1007/BF01581266 CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Cartis, C., Fowkes, J.M., Gould, N.I.M.: Branching and bounding improvements for global optimization algorithms with Lipschitz continuity properties. Tech. rep., Optimization. http://www.optimization-online.org/DB_HTML/2013/06/3914.html (2013)
  6. 6.
    Casado, L.G., Martìnez, J.A., Garcìa, I., Hendrix, E.M.T.: Branch-and-bound interval global optimization on shared memory multiprocessors. Optim. Methods Softw. 23(5), 689–701 (2008). doi: 10.1080/10556780802086300 CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Conn, A.R., Gould, N.I.M., Toint P.L.: Trust region methods. MPS-SIAM series on optimization, SIAM. http://books.google.co.uk/books?id=5kNC4fqssYQC (2000)
  8. 8.
    Crainic, T.G., Le Cun, B., Roucairol, C.: Parallel branch-and-bound algorithms. In: Talbi, E. (ed.) Parallel Combinatorial Optimization, Wiley Series on Parallel and Distributed Computing, pp. 1–28. Wiley, New York. http://books.google.co.uk/books?id=rYtuk_sm23UC (2006)
  9. 9.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002). doi: 10.1007/s101070100263 CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Evtushenko, Y.G.: Numerical methods for finding global extrema (case of a non-uniform mesh). USSR Comput. Math. Math. Phys. 11(6), 38–54 (1971)CrossRefGoogle Scholar
  11. 11.
    Evtushenko, Y.G., Posypkin, M.A.: An application of the nonuniform covering method to global optimization of mixed integer nonlinear problems. Comput. Math. Math. Phys. 51(8), 1286–1298 (2011). doi: 10.1134/S0965542511080082 CrossRefMathSciNetGoogle Scholar
  12. 12.
    Evtushenko, Y.G., Posypkin, M.A.: A deterministic approach to global box-constrained optimization. Optim. Lett. 7(4), 819–829 (2013). doi: 10.1007/s11590-012-0452-1 CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Evtushenko, Y.G., Posypkin, M.A., Sigal, I.: A framework for parallel large-scale global optimization. Comput. Sci. Res. Dev. 23(3–4), 211–215 (2009). doi: 10.1007/s00450-009-0083-7 CrossRefGoogle Scholar
  14. 14.
    Floudas, C.: Deterministic Global Optimization: Theory, Methods and Applications. Nonconvex Optimization and Its Applications. Springer, Berlin. http://books.google.co.uk/books?id=qZSpq27TsOcC (1999)
  15. 15.
    Fowkes, J.M.: Bayesian Numerical Analysis: Global Optimization and Other Applications. PhD thesis, Mathematical Institute, University of Oxford. http://ora.ox.ac.uk/objects/uuid:ab268fe7-f757-459e-b1fe-a4a9083c1cba (2012)
  16. 16.
    Fowkes, J.M., Gould, N.I.M., Farmer, C.L.: A branch and bound algorithm for the global optimization of Hessian Lipschitz continuous functions. J. Glob. Optim. 56(4), 1791–1815 (2013). doi: 10.1007/s10898-012-9937-9 CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Gaviano, M., Lera, D.: A global minimization algorithm for Lipschitz functions. Optim. Lett. 2(1), 1–13 (2008). doi: 10.1007/s11590-006-0036-z CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Gendron, B., Crainic, T.G.: Parallel branch-and-bound algorithms: survey and synthesis. Oper. Res. 42(6), 1042–1066 (1994). doi: 10.1287/opre.42.6.1042 CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Grishagin, V.A.: Operating characteristics of some global search algorithms. Probl. Stoch. Search 7, 198–206 (1978). (In Russian)zbMATHGoogle Scholar
  20. 20.
    Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches. Springer, Berlin. http://books.google.co.uk/books?id=usFjGFvuBDEC (1996)
  21. 21.
    Knuth, D.: The Art of Computer Programming: Sorting and Searching, the Art of Computer Programming, vol. 3. Addison-Wesley. http://books.google.co.uk/books?id=ePzuAAAAMAAJ (1998)
  22. 22.
    Kolda, T., Mayo, J.: Shifted power method for computing tensor eigenpairs. SIAM J. Matrix Anal. Appl. 32(4), 1095–1124 (2011). doi: 10.1137/100801482 CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Kreinovich, V., Kearfott, R.: Beyond convex? Global optimization is feasible only for convex objective functions: a theorem. J. Glob. Optim. 33(4), 617–624 (2005). doi: 10.1007/s10898-004-2120-1 CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Kvasov, D.E., Sergeyev, Y.D.: A univariate global search working with a set of Lipschitz constants for the first derivative. Optim. Lett. 3(2), 303–318 (2009). doi: 10.1007/s11590-008-0110-9 CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Kvasov, D.E., Sergeyev, Y.D.: Lipschitz gradients for global optimization in a one-point-based partitioning scheme. J. Comput. Appl. Math. 236(16), 4042–4054 (2012). doi: 10.1016/j.cam.2012.02.020 CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Kvasov, D.E., Sergeyev, Y.D.: Univariate geometric Lipschitz global optimization algorithms. Numer. Algebr. Control Optim. 2(1), 69–90 (2012). doi: 10.3934/naco.2012.2.69 CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Lera, D., Sergeyev, Y.D.: Acceleration of univariate global optimization algorithms working with Lipschitz functions and Lipschitz first derivatives. SIAM J. Optim. 23(1), 508–529 (2013). doi: 10.1137/110859129 CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Lim, L.H.: Singular values and eigenvalues of tensors: a variational approach. In: Computational Advances in Multi-Sensor Adaptive Processing, 2005 1st IEEE International Workshop on, pp. 129–132 (2005). doi: 10.1109/CAMAP.2005.1574201
  29. 29.
    Neumaier, A.: Complete search in continuous global optimization and constraint satisfaction. Acta Numer. 13, 271–369 (2004). doi: 10.1017/S0962492904000194 CrossRefMathSciNetGoogle Scholar
  30. 30.
    Pardalos, P.M., Horst, R., Thoai, N.V.: Introduction to Global Optimization, Nonconvex Optimization and its Applications, vol. 3. Springer, Berlin. http://www.springer.com/mathematics/book/978-0-7923-3556-6 (1995)
  31. 31.
    Paulavičius, R., Žilinskas, J., Grothey, A.: Parallel branch and bound for global optimization with combination of Lipschitz bounds. Optim. Methods Softw. 26(3), 487–498 (2011). doi: 10.1080/10556788.2010.551537 CrossRefMathSciNetGoogle Scholar
  32. 32.
    Pintér, J.D.: Global Optimization in Action, Nonconvex Optimization and its Applications, vol. 6. Springer, Berlin. http://www.springer.com/mathematics/book/978-0-7923-3757-7 (1996)
  33. 33.
    Piyavskii, S.A.: An algorithm for finding the absolute extremum of a function. USSR Comput. Math. Math. Phys. 12(4), 57–67 (1972). doi: 10.1016/0041-5553(72)90115-2 CrossRefGoogle Scholar
  34. 34.
    Qi, L.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40(6), 1302–1324 (2005). doi: 10.1016/j.jsc.2005.05.007. http://www.sciencedirect.com/science/article/pii/S0747717105000817
  35. 35.
    Sergeyev, Y.D.: Global one-dimensional optimization using smooth auxiliary functions. Math. Program. 81(1), 127–146 (1998). doi: 10.1007/BF01584848 CrossRefzbMATHMathSciNetGoogle Scholar
  36. 36.
    Sergeyev, Y.D., Strongin, R.G., Lera, D.: Introduction to Global Optimization Exploiting Space-Filling Curves. Springer Briefs in Optimization, Springer, Berlin. http://books.google.co.uk/books?id=IqyYnAEACAAJ (2013)
  37. 37.
    Shcherbina, O., Neumaier, A., Sam-Haroud, D., Vu, X.H., Nguyen, T.V.: Benchmarking global optimization and constraint satisfaction codes. In: Bliek, C., Jermann, C., Neumaier, A. (eds.) Global Optimization and Constraint Satisfaction, Lecture Notes in Computer Science, vol. 2861, pp. 211–222. Springer, Berlin, Heidelberg (2003). doi: 10.1007/978-3-540-39901-8_16
  38. 38.
    Shubert, B.: A sequential method seeking the global maximum of a function. SIAM J. Numer. Anal. 9(3), 379–388 (1972). doi: 10.1137/0709036 CrossRefzbMATHMathSciNetGoogle Scholar
  39. 39.
    Stephens, C.P., Baritompa, W.: Global optimization requires global information. J. Optim. Theory Appl. 96(3), 575–588 (1998). doi: 10.1023/A:1022612511618 CrossRefzbMATHMathSciNetGoogle Scholar
  40. 40.
    Strongin, R.G., Sergeyev, Y.D.: Global Optimization with Non-convex Constraints: Sequential and Parallel Algorithms. Nonconvex Optimization and its Applications. Springer, Berlin. http://books.google.co.uk/books?id=xh_GF9Dor3AC (2000)
  41. 41.
    Zhang, X., Qi, L., Ye, Y.: The cubic spherical optimization problems. Math. Comput. 81(279), 1513–1525 (2012). doi: 10.1090/S0025-5718-2012-02577-4 CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Coralia Cartis
    • 1
  • Jaroslav M. Fowkes
    • 1
    Email author
  • Nicholas I. M. Gould
    • 2
  1. 1.School of MathematicsUniversity of EdinburghEdinburghScotland, UK
  2. 2.Computational Science and Engineering DepartmentRutherford Appleton LaboratoryOxfordshireUK

Personalised recommendations