Abstract
When a black-box optimization objective can only be evaluated with costly or noisy measurements, most standard optimization algorithms are unsuited to find the optimal solution. Specialized algorithms that deal with exactly this situation make use of surrogate models. These models are usually continuous and smooth, which is beneficial for continuous optimization problems, but not necessarily for combinatorial problems. However, by choosing the basis functions of the surrogate model in a certain way, we show that it can be guaranteed that the optimal solution of the surrogate model is integer. This approach outperforms random search, simulated annealing and a Bayesian optimization algorithm on the problem of finding robust routes for a noise-perturbed traveling salesman benchmark problem, with similar performance as another Bayesian optimization algorithm, and outperforms all compared algorithms on a convex binary optimization problem with a large number of variables.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. Journal of Global optimization 13(4), 455–492 (1998)
Gosavi, A.: Simulation-based optimization: parametric optimization techniques and reinforcement learning, Springer, 55 (2015)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to derivative-free optimization, Siam, 8 (2009)
Bliek, L., Verstraete, H.R.G.W., Verhaegen, M., Wahls, S.: Online optimization with costly and noisy measurements using random Fourier expansions. IEEE Transactions on Neural Networks and Learning Systems 29(1), 167–182 (2018)
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp 2951–2959 (2012)
Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., Doucet, A.: A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Auton. Robot. 27(2), 93–103 (2009)
Garrido-Merchán, EC, Hernández-Lobato, D: Dealing with integer-valued variables in Bayesian optimization with Gaussian processes. arXiv:1706.03673 (June 2017)
Verwer, S., Zhang, Y., Ye, Q.C.: Auction optimization using regression trees and linear models as integer programs. Artif. Intell. 244, 368–395 (2017)
Verbeeck, D., Maes, F., De Grave, K., Blockeel, H.: Multi-objective optimization with surrogate trees. In: Proceedings of the 15th annual conference on Genetic and evolutionary computation, pp 679–686, ACM (2013)
Bliek, L., Verhaegen, M., Wahls, S.: Online function minimization with convex random ReLU expansions. In: Machine Learning for Signal Processing (MLSP), 2017 IEEE 27th International Workshop on, pp 1–6, IEEE (2017)
Baptista, R., Poloczek, M.: Bayesian optimization of combinatorial structures. In: International Conference on Machine Learning, pp 471–480 (2018)
Ueno, T., Rhone, T D, Hou, Z., Mizoguchi, T., Tsuda, K.: Combo: An efficient Bayesian optimization library for materials science. Materials discovery 4, 18–21 (2016)
Aarts, E.H.L., Lenstra, J.K.: Local search in combinatorial optimization, Princeton University Press (2003)
Rajeev, S., Krishnamoorthy, C.S.: Discrete optimization of structures using genetic algorithms. Journal of structural engineering 118(5), 1233–1250 (1992)
Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. In: 1997 IEEE International Conference on Systems, Man, and Cybernetics: Computational Cybernetics and Simulation, vol. 5, pp 4104–4108, IEEE (1997)
Dorigo, M., Caro, G.D., Gambardella, L.M.: Ant algorithms for discrete optimization. Artificial life 5(2), 137–172 (1999)
Hong, L.J., Nelson, B.L.: Discrete optimization via simulation using COMPASS. Oper. Res. 54(1), 115–129 (2006)
Shapiro, A., Dentcheva, D., Ruszczyński, A: Lectures on stochastic programming: modeling and theory, SIAM (2014)
Wolsey, L.A.: Integer programming, John Wiley & Sons, vol. 52 (1998)
Schrijver, A.: Theory of linear and integer programming, John Wiley & Sons (1998)
Li, D., Sun, X.: Nonlinear integer programming, Springer Science & Business Media, 84 (2006)
Mockus, J.: Bayesian approach to global optimization: theory and applications, Springer Science & Business Media, 37 (2012)
Bergstra, J., Yamins, D., Cox, D.D.: Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In: Proceedings of the 12th Python in science conference, pp 13–20 (2013)
Bergstra, J., Yamins, D., Cox, D D: Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In: Proceedings of the 30th International Conference on Machine Learning. Jmlr (2013)
Rahimi, A., Recht, B.: Uniform approximation of functions with random bases. In: 46th Annual Allerton Conference on Communication, Control, and Computing, pp 555–561, IEEE (2008)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Sayed, A.H., Kailath, T.: Recursive least-squares adaptive filters, The Digital Signal Processing Handbook, 21, 1 (1998)
Wright, S., Nocedal, J.: Numerical optimization. Springer Science 35, 67–68 (1999)
TSPlib: http://elib.zib.de/pub/mp-testdata/tsp/tsplib/tsplib.html (2019)
Acknowledgments
This work is part of the research programme Real-time data-driven maintenance logistics with project number 628.009.012, which is financed by the Dutch Research Council (NWO). The authors would also like to thank Arthur Guijt for helping with the python code.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bliek, L., Verwer, S. & de Weerdt, M. Black-box combinatorial optimization using models with integer-valued minima. Ann Math Artif Intell 89, 639–653 (2021). https://doi.org/10.1007/s10472-020-09712-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-020-09712-4