How to Deal with Mixed-Variable Optimization Problems: An Overview of Algorithms and Formulations

Conference paper


Real world engineering optimization problems often involve discrete variables (e.g., categorical variables) characterizing choices such as the type of material to be used or the presence of certain system components. From an analytical perspective, these particular variables determine the definition of the objective and constraint functions, as well as the number and type of parameters that characterize the problem. Furthermore, due to the inherent discrete and potentially non-numerical nature of these variables, the concept of metrics is usually not definable within their domain, thus resulting in an unordered set of possible choices. Most modern optimization algorithms were developed with the purpose of solving design problems essentially characterized by integer and continuous variables and by consequence the introduction of these discrete variables raises a number of new challenges. For instance, in case an order can not be defined within the variables domain, it is unfeasible to use optimization algorithms relying on measures of distances, such as Particle Swarm Optimization. Furthermore, their presence results in non-differentiable objective and constraint functions, thus limiting the use of gradient-based optimization techniques. Finally, as previously mentioned, the search space of the problem and the definition of the objective and constraint functions vary dynamically during the optimization process as a function of the discrete variables values.

This paper presents a comprehensive survey of the scientific work on the optimization of mixed-variable problems characterized by continuous and discrete variables. The strengths and limitations of the presented methodologies are analyzed and their adequacy for mixed-variable problems with regards to the particular needs of complex system design is discussed, allowing to identify several ways of improvements to be further investigated.


Mixed-variable optimization Variable-size design space Categorical variables 


  1. 1.
    Abdelkhalik, O.: Autonomous planning of multigravity-assist trajectories with deep space Maneuvers using a differential evolution approach. Int. J. Aerosp. Eng. 2013, 1–11 (2013)CrossRefGoogle Scholar
  2. 2.
    Abdelkhalik, O.: Hidden genes genetic optimization for variable-size design space problems. J. Optim. Theor. Appl. 156(2), 450–468 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Abramson, M.A.: Mixed variable optimization of a load-bearing thermal insulation system using a filter pattern search algorithm. Optim. Eng. 5(2), 157–177 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Abramson, M.A., Audet, C., Chrissis, J.W., Walston, J.G.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3(1), 35–47 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Abramson, M.A., Audet, C., Dennis Jr., J.E.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Audet, C., Dennis Jr., J.E.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11(3), 573–594 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bartz-Beielstein, T., Zaefferer, M.: Model-based methods for continuous and discrete global optimization. Appl. Soft Comput. 55, 154–167 (2017)CrossRefGoogle Scholar
  9. 9.
    Coello, C.A.C.: Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput. Meth. Appl. Mech. Eng. 191(11–12), 1245–1287 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Deb, K., Pratap, A., Agarwal, S.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  11. 11.
    Dorigo, M.: Optimization, learning and natural algorithms. Ph.D. thesis, Politecnico di Milano, Italy (1992)Google Scholar
  12. 12.
    Emmerich, M., Grötzner, M., Groß, B., Schütz, M.: Mixed-integer evolution strategy for chemical plant optimization with simulators. In: Evolutionary Design and Manufacture, pp. 55–67. Springer, London (2000)Google Scholar
  13. 13.
    Coelho, R.F.: Metamodels for mixed variables based on moving least squares. Optim. Eng. 15(2), 311–329 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Frank, C., Marlier, R., Pinon-Fischer, O.J., Mavris, D.N.: An evolutionary multi-architecture multi-objective optimization algorithm for design space exploration. In: 57th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Reston, Virginia, January 2016. American Institute of Aeronautics and AstronauticsGoogle Scholar
  15. 15.
    Goldberg, D.E.: Genetic Algorithms in Search Optimization & Machine Learning. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)Google Scholar
  16. 16.
    Herrera, M., Guglielmetti, A., Xiao, M., Coelho, R.F.: Metamodel-assisted optimization based on multiple Kernel regression for mixed variables. Struct. Multi. Optim. 49(6), 979–991 (2014)CrossRefGoogle Scholar
  17. 17.
    Hooke, R., Jeeves, T.A.: Direct search solution of numerical, statistical problems. J. ACM 8(2), 212–229 (1961)CrossRefzbMATHGoogle Scholar
  18. 18.
    Isebor, O.J.: Derivative-free optimization for generalized oil field development. Ph.D. thesis, Stanford University, USA (2010)Google Scholar
  19. 19.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN 1995 - International Conference on Neural Networks, pp. 1942–1948 (1995)Google Scholar
  20. 20.
    Kirkpatrick, S., Gelatt, D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Kokkolaras, M., Audet, C., Dennis Jr., J.E.: Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2(1), 5–29 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Land, A.H., Doig, A.G.: An automatic method for solving discrete programming problems. In: 50 Years of Integer Programming 1958–2008, pp. 105–132. Springer, Heidelberg (2010)Google Scholar
  23. 23.
    Li, R., Emmerich, M.T.M., Eggermont, J., Bovenkamp, E.G.P., Bäck, T., Dijkstra, J., Reiber, J.H.C.: Metamodel-assisted mixed integer evolution strategies and their application to intravascular ultrasound image analysis. In: 2008 IEEE Congress on Evolutionary Computation, CEC 2008, Hong Kong, June 2008. IEEE, pp. 2764–2771Google Scholar
  24. 24.
    Liao, C.-J., Tseng, C.-T., Luarn, P.: A discrete version of particle swarm optimization for flowshop scheduling problems. Comput. Oper. Res. 34(10), 3099–3111 (2007)CrossRefzbMATHGoogle Scholar
  25. 25.
    Liao, T., Socha, K., de Oca, M.A.M., Stutzle, T., Dorigo, M.: Ant colony optimization for mixed-variable optimization problems. IEEE Trans. Evol. Comput. 18(4), 503–518 (2014)CrossRefGoogle Scholar
  26. 26.
    Lucidi, S., Piccialli, V., Sciandrone, M.: An algorithm model for mixed variable programming. SIAM J. Optim. 15(4), 1057–1084 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1964)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Nyew, H.M., Abdelkhalik, O., Onder, N.: Structured-chromosome evolutionary algorithms for variable-size autonomous interplanetary trajectory planning optimization. J. Aerosp. Inf. Syst. 12(3), 314–328 (2015)Google Scholar
  29. 29.
    Roy, S., Moore, K., Hwang, J.T., Gray, J.S., Crossley, W.A., Martins, J.: A mixed integer efficient global optimization algorithm for the simultaneous aircraft allocation-mission-design problem. In: 58th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Reston, Virginia, January 2017. American Institute of Aeronautics and AstronauticsGoogle Scholar
  30. 30.
    Stelmack, M., Nakashima, N., Batill, S.: Genetic algorithms for mixed discrete/continuous optimization in multidisciplinary design. In: 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Reston, Virigina, September 1998. American Institute of Aeronautics and AstronauticsGoogle Scholar
  31. 31.
    Storn, R., Price, K.: Differential evolution a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Sun, C., Zeng, J., Pan, J.-S.: A modified particle swarm optimization with feasibility-based rules for mixed-variable optimization problems. Int, J. Innov. Comput. Inf. Contr. 7(6), 3081–3096 (2011)Google Scholar
  33. 33.
    Talbi, E.-G.: Metaheuristics: From Design to Implementation. Wiley, New York (2009)CrossRefzbMATHGoogle Scholar
  34. 34.
    Venter, G., Sobieszczanski-Sobieski, J.: Multidisciplinary optimization of a transport aircraft wing using particle swarm optimization. Struct. Multi. Optim. 26(1–2), 121–131 (2004)CrossRefGoogle Scholar
  35. 35.
    Wang, J., Yin, Z.: A ranking selection-based particle swarm optimizer for engineering design optimization problems. Struct. Multi. Optim. 37(2), 131–147 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.ONERA-The French Aerospace Lab/CNESToulouseFrance
  2. 2.ONERA-The French Aerospace LabPalaiseauFrance
  3. 3.INRIAParisFrance
  4. 4.CNESToulouseFrance

Personalised recommendations