, Volume 21, Issue 1, pp 133–162 | Cite as

Benchmarking nonlinear optimization software in technical computing environments

Global optimization in Mathematica with MathOptimizer Professional
  • János D. PintérEmail author
  • Frank J. Kampas
Original Paper


Our strategic objective is to develop a broadly categorized, expandable collection of test problems, to support the benchmarking of nonlinear optimization software packages in integrated technical computing environments (ITCEs). ITCEs—such as Maple, Mathematica, and MATLAB—support concise, modular and scalable model development: their built-in documentation and visualization features can be put to good use also in test model selection and analysis. ITCEs support the flexible inclusion of both new models and general-purpose solver engines for future studies. Within this broad context, in this article we review a collection of global optimization problems coded in Mathematica, and present illustrative and summarized numerical results obtained using the MathOptimizer Professional software package.


Nonlinear optimization in integrated technical computing environments Optimization software benchmarking Model library in Mathematica Lipschitz Global Optimizer (LGO) solver suite for nonlinear optimization MathOptimizer Professional (LGO linked to MathematicaNumerical performance results 

Mathematics Subject Classification (2000)

65K30 90C05 90C31 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Addis B, Locatelli M (2007) A new class of test functions for global optimization. J Glob Optim 38:479–501 CrossRefGoogle Scholar
  2. Addis B, Locatelli M, Schoen F (2008) Efficiently packing unequal disks in a circle. Oper Res Lett 36(1):37–42 CrossRefGoogle Scholar
  3. Ali MM, Khompatraporn Ch, Zabinsky ZB (2005) A numerical evaluation of several global optimization algorithms on selected test problems. J Glob Optim 31:635–672 CrossRefGoogle Scholar
  4. AMPL LLC (2011) AMPL;
  5. Averick BM, Moré JJ (1991) The Minpack-2 test problem collection. Technical report ANL/MCS-TM-157, Argonne National Laboratory, Argonne, IL Google Scholar
  6. Barr RS, Golden BL, Kelly JP, Resende MGC, Stewart WR (1995) Designing and reporting on computational experiments with heuristic methods. J Heuristics 1:9–32 CrossRefGoogle Scholar
  7. Berna TJ, Locke MH, Westerberg AW (1980) A new approach to optimization of chemical processes. AIChE J 26(1):37–43 CrossRefGoogle Scholar
  8. Bhatti AM (2000) Practical optimization methods with Mathematica applications. Springer, New York CrossRefGoogle Scholar
  9. Bongartz I, Conn AR, Gould NIM, Toint PhL (1995) CUTE: constrained and unconstrained testing environment. ACM Trans Math Softw 21(1):123–160 CrossRefGoogle Scholar
  10. Buckley AG (1992) Algorithm 709: testing algorithm implementations. ACM Trans Math Softw 18(4):375–391 CrossRefGoogle Scholar
  11. Casado LC, García I, Martínez JA, Sergeyev YD (2003) New interval analysis support functions using gradient information in a global minimization algorithm. J Glob Optim 25:345–362 CrossRefGoogle Scholar
  12. Castillo I, Kampas FJ, Pintér JD (2008) Solving circle packing problems by global optimization: numerical results and industrial applications. Eur J Oper Res 191:786–802 CrossRefGoogle Scholar
  13. Csendes T, Pál L, Sendin OH, Banga JR (2008) The GLOBAL optimization method revisited. Optim Lett 2:445–454 CrossRefGoogle Scholar
  14. Dimmer PR, Cutteridge OPD (1980) Second derivative Gauss–Newton-based method for solving nonlinear simultaneous equations. IEE Proc G, Electron Circuits Syst 127:278–283 CrossRefGoogle Scholar
  15. Dixon L, Szegö G (eds) (1975) Towards global optimisation, vol 1. North-Holland, Amsterdam Google Scholar
  16. Dixon L, Szegö G (eds) (1978) Towards global optimisation, vol 2. North-Holland, Amsterdam Google Scholar
  17. Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program, Ser A 91:201–213 CrossRefGoogle Scholar
  18. Dolan ED, Moré JJ, Munson TS (2004) Benchmarking optimization software with COPS 3.0. Technical report ANL/MCS-TM-273, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL Google Scholar
  19. Doye JPK, Leary RH, Locatelli M, Schoen F (2004) The global optimization of Morse clusters by potential energy transformations. INFORMS J Comput 16:371–379 CrossRefGoogle Scholar
  20. Ebers JJ, Moll JL (1954) Large signal behaviour of junction transistors. Proc IRE 42:1761–1772 CrossRefGoogle Scholar
  21. Fasano G (2008) MIP-based heuristic for non-standard 3D-packing problems. 4OR 6:291–310 CrossRefGoogle Scholar
  22. Fasano G, Saia D, Piras A (2010) Columbus stowage optimization by CAST (Cargo Accommodation Support Tool). Acta Astronaut 67(3):489–496 CrossRefGoogle Scholar
  23. Floudas CA, Pardalos PM, Adjiman CS, Esposito WR, Gümüş ZH, Harding ST, Klepeis JL, Meyer CA, Schweiger CA (1999) Handbook of test problems in local and global optimization. Kluwer Academic, Dordrecht CrossRefGoogle Scholar
  24. GAMS Development Corporation (2011) GAMS;
  25. GAMS Global World (2011) GLOBAL Library: a collection of nonlinear programming models;
  26. GAMS Performance World (2011)
  27. Gaviano M, Kvasov DE, Lera D, Sergeyev YD (2003) Algorithm 829: software for generation of classes of test functions with known local and global minima for global optimization. ACM Trans Math Softw 29(4):469–480 CrossRefGoogle Scholar
  28. Gould NIM, Orban D, Toint PL (2003) CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans Math Softw 29(4):373–394 CrossRefGoogle Scholar
  29. Grossmann IE (ed) (1996) Global optimization in engineering design. Kluwer Academic, Dordrecht Google Scholar
  30. Hansen P, Jaumard B, Lu S-H (1992) Global optimization of univariate Lipschitz functions: II. New algorithms and computational comparison. Math Program 55:273–292 CrossRefGoogle Scholar
  31. Hare WL, Wang Y (2010) Fairer benchmarking of optimization algorithms via derivative free optimization. Available from Optimization Online,
  32. Hock W, Schittkowski K (1981) Test examples for nonlinear programming codes. Lecture notes in economics and mathematical systems, vol 187. Springer, Berlin CrossRefGoogle Scholar
  33. Hock W, Schittkowski K (1983) A comparative performance evaluation of 27 nonlinear programming codes. Computing 30:335–358 CrossRefGoogle Scholar
  34. Jackson RHF, Boggs PT, Nash SG, Powell S (1991) Guidelines for reporting results of computation experiments. Report of the ad hoc committee. Math Program 49:413–425 CrossRefGoogle Scholar
  35. Kampas FJ, Pintér JD (2006) Configuration analysis and design by using optimization tools in Mathematica. Math J 10:128–154 Google Scholar
  36. Khompatraporn Ch, Pintér JD, Zabinsky ZB (2005) Comparative assessment of algorithms and software for global optimization. J Glob Optim 31:613–633 CrossRefGoogle Scholar
  37. Lahey Computer Systems (2004) Lahey/Fujitsu FORTRAN 95 (Release 5.70f),
  38. Lavor C, Liberti L, Maculan N (2006) Computational experience with the molecular distance geometry problem. In: Pintér JD (ed) Global optimization: selected case studies. Springer, New York Google Scholar
  39. LINDO Systems (2011) LINGO;
  40. Liu D, Zhang X-S (2000) Test problem generator by neural network for algorithms that try solving nonlinear programming problems globally. J Glob Optim 16:229–243 CrossRefGoogle Scholar
  41. Luksan L, Vlçek J (2000) Test problems for non-smooth unconstrained and linearly constrained optimization. Technical report 798, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague Google Scholar
  42. Maplesoft (2011a) Maple. Maplesoft, Inc, Waterloo, ON;
  43. Maplesoft (2011b) Maple Global Optimization Toolbox. Maplesoft, Inc, Waterloo, ON;
  44. Maximal Software (2011) MPL. Maximal Software, Inc, Arlington, VA;
  45. Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs. Springer, Heidelberg CrossRefGoogle Scholar
  46. Mittelmann HD (2011) Decision tree for optimization software;
  47. Moré JJ, Garbow BS, Hillström KE (1981) Testing unconstrained optimization software. ACM Trans Math Softw 7:17–41 CrossRefGoogle Scholar
  48. Moré JJ, Wild SM (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20(1):172–191 CrossRefGoogle Scholar
  49. Müller A, Schneider JJ, Schömer E (2009) Packing a multidisperse system of hard disks in a circular environment. Phys Rev E 79:021102 (14 pages) CrossRefGoogle Scholar
  50. Neumaier A (2011) Global optimization;
  51. Neumaier A, Shcherbina O, Huyer W, Vinkó T (2005) A comparison of complete global optimization solvers. Math Program, Ser B 103:335–356 CrossRefGoogle Scholar
  52. Papalambros PM, Wilde DJ (2000) Principles of optimal design—modeling and computation, 2nd edn. Cambridge University Press, Cambridge CrossRefGoogle Scholar
  53. Paragon Decision Technology (2011) AIMMS. Paragon Decision Technology BV, Haarlem, The Netherlands;
  54. Pál L, Csendes T (2009) INTLAB implementation of an interval global optimization algorithm. Optim Methods Softw 24(4):749–759 CrossRefGoogle Scholar
  55. Pintér JD (1996a) Global optimization in action. Kluwer Academic, Dordrecht CrossRefGoogle Scholar
  56. Pintér JD (1996b) Continuous global optimization software: a brief review. Optima 52:1–8. See also at Google Scholar
  57. Pintér JD (1997) LGO: A program system for continuous and Lipschitz optimization. In: Bomze IM, Csendes T, Horst R, Pardalos PM (eds) Developments in global optimization. Kluwer Academic, Dordrecht, pp 183–197 CrossRefGoogle Scholar
  58. Pintér JD (2001) Globally optimized spherical point arrangements: model variants and illustrative results. Ann Oper Res 104:213–230 CrossRefGoogle Scholar
  59. Pintér JD (2002) Global optimization: software, test problems, and applications. In: Pardalos PM, Romeijn HE (eds) Handbook of global optimization, vol 2. Kluwer Academic, Dordrecht, pp 515–569 Google Scholar
  60. Pintér JD (2005) Nonlinear optimization in modeling environments: software implementations for compilers, spreadsheets, modeling languages, and integrated computing systems. In: Jeyakumar V, Rubinov AM (eds) Continuous optimization: current trends and applications. Springer, New York, pp 147–173 CrossRefGoogle Scholar
  61. Pintér JD (2006a) Global optimization with Maple: an introduction with illustrative examples. An electronic book published by Pintér Consulting Services, Canada Google Scholar
  62. Pintér JD (ed) (2006b) Global optimization: selected case studies. Springer, New York Google Scholar
  63. Pintér JD (2007) Nonlinear optimization with GAMS/LGO. J Glob Optim 38:79–101 CrossRefGoogle Scholar
  64. Pintér JD (2009a) Global optimization in practice: state-of-the-art and perspectives. In: Gao DY, Sherali HD (eds) Advances in applied mathematics and global optimization. Springer, New York, pp 377–404 CrossRefGoogle Scholar
  65. Pintér JD (2009b) Software development for global optimization. In: Pardalos PM, Coleman TF (eds) Global optimization: methods and applications. Fields institute communications, vol 55. American Mathematical Society, Providence, pp 183–204 Google Scholar
  66. Pintér JD, Bagirov A, Zhang J (2003) An illustrated collection of global optimization test problems. Research report, School of Information Technology & Mathematical Sciences, University of Ballarat, Victoria, Australia Google Scholar
  67. Pintér JD, Kampas FJ (2005) Nonlinear optimization in Mathematica with MathOptimizer Professional. Math Educ Res 10(2):1–18 Google Scholar
  68. Pintér JD, Kampas FJ (2006) MathOptimizer Professional: key features and illustrative applications. In: Liberti L, Maculan N (eds) Global optimization: from theory to implementation. Springer, New York, pp 263–279 CrossRefGoogle Scholar
  69. Pintér JD, Kampas FJ (2010a) MathOptimizer—an advanced modeling and optimization system for Mathematica users. User’s guide. (MathOptimizer is developed and maintained since 2002 by Pintér Consulting Services, Inc, Canada;
  70. Pintér JD, Kampas FJ (2010b) MathOptimizer Professional—an advanced modeling and optimization system for Mathematica, using the LGO Solver Engine. User’s guide. (MathOptimizer Professional is developed and maintained since 2003 by Pintér Consulting Services, Inc, Canada;
  71. Pintér JD, Linder D, Chin P (2006) Global optimization toolbox for Maple: an introduction with illustrative applications. Optim Methods Softw 21(4):565–582 CrossRefGoogle Scholar
  72. Rakhmanov EA, Saff EB, Zhou YM (1995) Electrons on the sphere. In: Ali RM, Ruscheweyh S, Saff EB (eds) Computational methods and function theory. World Scientific, Singapore, pp 111–127 Google Scholar
  73. Ratschek H, Rokne J (1993) Experiments using interval analysis for solving a circuit design problem. J Glob Optim 3:501–518 CrossRefGoogle Scholar
  74. Rios LM, Sahinidis NV (2010) Derivative-free optimization: a review of algorithms and comparison of software implementations. Technical report, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA (to appear) Google Scholar
  75. Shcherbina O, Neumaier A, Sam-Haroud D, Vu X-H, Tuan-Viet N (2003) Benchmarking global optimization and constraint satisfaction codes. In: Bliek Ch et al. (eds) Global optimization and constraint satisfaction. Springer, Berlin, pp 211–222 CrossRefGoogle Scholar
  76. Schichl H (2003) Global optimization in the COCONUT project. Manuscript,
  77. Schittkowski K (1987) More test examples for nonlinear programming. Lecture notes in economics and mathematical systems, vol 182. Springer, Berlin CrossRefGoogle Scholar
  78. Schittkowski K (2008) An updated set of 306 test problems for nonlinear programming with validated optimal solutions—user’s guide. Research report, Department of Computer Science, University of Bayreuth Google Scholar
  79. Schoen F (1993) A wide class of test functions for global optimization. J Glob Optim 3:133–137 CrossRefGoogle Scholar
  80. Schoen F (2008) Personal communication Google Scholar
  81. Stortelder WJH, de Swart JJB, Pintér JD (2001) Finding elliptic Fekete points sets: two numerical solution approaches. J Comput Appl Math 130:205–216 CrossRefGoogle Scholar
  82. Tang K, Li X, Suganthan PN, Yang Z, Weise T (2010) Benchmark functions for the CEC’2010 special session and competition on large-scale global optimization. Online research document, January 8, 2010. For possible updates, visit
  83. Tawarmalani M, Sahinidis NV (2002) Convexification and global optimization in continuous and mixed-integer nonlinear programming. Kluwer Academic, Dordrecht Google Scholar
  84. The MathWorks (2011a) MATLAB. The MathWorks, Inc, Natick, MA;
  85. The MathWorks (2011b) MATLAB global optimization toolbox 3 user’s guide. The MathWorks, Inc, Natick, MA;
  86. TOMLAB Optimization (2011) TOMLAB. TOMLAB Optimization AB, Västerås, Sweden;
  87. Trott M (2004, 2006) The Mathematica GuideBooks. Programming, graphics, numerics, symbolics, vols 1–4. Springer, New York Google Scholar
  88. Vanderbei RJ (2011) Nonlinear optimization models;
  89. Weise T (2011) Global optimization algorithms—theory and application;
  90. Wolfram Research (2011) Mathematica. Wolfram Research, Inc, Champaign, IL;
  91. Wolfram S (2003) The Mathematica book, 5th edn. Wolfram Media, Inc, Champaign. Online reference Google Scholar
  92. Zabinsky ZB (2003) Stochastic adaptive search for global optimization. Kluwer Academic, Dordrecht Google Scholar

Copyright information

© Sociedad de Estadística e Investigación Operativa 2011

Authors and Affiliations

  1. 1.PCS Inc.HalifaxCanada
  2. 2.Department of Industrial EngineeringÖzyeğin UniversityIstanbulTurkey
  3. 3.Physicist at Large ConsultingAmblerUSA

Personalised recommendations