Skip to main content

Real-time solving of computationally hard problems using optimal algorithm portfolios


Various hard real-time systems have a desired requirement which is impossible to fulfill: to solve a computationally hard optimization problem within a short and fixed amount of time T, e.g., T = 0.5 seconds. For such a task, the exact, exponential algorithms, as well as various Polynomial-Time Approximation Schemes, are irrelevant because they can exceed T. What is left in practice is to combine various anytime algorithms in a parallel portfolio. The question is how to build such an optimal portfolio, given a budget of K computing cores. It is certainly not as simple as choosing the K best performing algorithms, because their results are possibly correlated (e.g., there is no point in choosing two good algorithm for the portfolio if they win on a similar set of instances). We prove that the decision variant of this problem is NP-complete, and furthermore that the optimization problem is approximable. On the practical side, our main contribution is a solution of the optimization problem of choosing K algorithms out of n, for a machine with K computing cores, and the related problem of detecting the minimum number of required cores to achieve an optimal portfolio, with respect to a given training set of instances. As a benchmark, we took instances of a hard optimization problem that is prevalent in the real-time industry, in which the challenge is to decide on the best action within time T. We include the results of numerous experiments that compare the various methods. Hence, a side effect of our tests is that it gives the first systematic empirical evaluation of the relative success of various known stochastic-search algorithms in coping with a hard combinatorial optimization problems under a very short and fixed timeout.

This is a preview of subscription content, access via your institution.


  1. 1.

    Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: International Conference on Principles and Practice of Constraint Programming, pp 142–157. Springer, Berlin (2009)

  2. 2.

    Brooks, S.H.: A discussion of random methods for seeking maxima. Oper. Res. 6, 244–251 (1958)

    Article  Google Scholar 

  3. 3.

    Cohen, E., Dahlweid, M., Hillebrand, M., Leinenbach, D., Moskal, M., Santen, T., Schulte, W., Tobies, S.: VCC: a Practical System for Verifying Concurrent C, pp 23–42. Springer, Berlin (2009).

    Google Scholar 

  4. 4.

    Crescenzi, P., Kann, V.: A compendium of NP optimization problems. In: WWW Spring 1994 (1994)

  5. 5.

    Dechter, R.: Constraints Processing. The Morgan Kaufmann Series in Artificial Intelligence. Morgan Kaufmann, Burlington (2003)

    Google Scholar 

  6. 6.

    Di Gaspero, L., Schaerf, A.: Easylocal++: an object-oriented framework for flexible design of local search algorithms. Software — Practice & Experience 33(8), 733–765 (2003)

    Article  Google Scholar 

  7. 7.

    Dutertre, B., de Moura, L.M.: A fast linear-arithmetic solver for DPLL(T). In: Ball, T., Jones, R.B. (eds.) Computer Aided Verification, 18th International Conference, CAV 2006, Seattle, WA, USA, August 17-20, 2006, Proceedings, Lecture Notes in Computer Science, vol. 4144, pp 81–94. Springer (2006).

  8. 8.

    Forrest, S., Mitchell, M.: Relative Building-Block fitness and the Building-Block hypothesis. Foundations of Genetic Algorithms 2, 109–126 (1993)

    Google Scholar 

  9. 9.

    Glover, F.: Future paths for integer programming and links to artificial intelligence. Comput. Oper. Res. 13(5), 533–549 (1986)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Gomes, C.P., Selman, B.: Algorithm portfolios. Artif. Intell. 126 (1-2), 43–62 (2001)

    MathSciNet  Article  Google Scholar 

  11. 11.

    Hentenryck, P.V., Michel, L.: Constraint-Based Local Search. MIT Press, Cambridge (2005)

    MATH  Google Scholar 

  12. 12.

    Hoos, H., Leyton-Brown, K., Schaub, T., Schneider, M.: Algorithm configuration for portfolio-based parallel sat-solving. In: Workshop on Combining Constraint Solving with Mining and Learning (2012)

  13. 13.

    Hoos, H.H., Stutzle, T.: Stochastic Local Search: Foundations and Applications. Morgan Kaufmann Massachusetts , Burlington (2004)

    MATH  Google Scholar 

  14. 14.

    Huberman, B.A., Lukose, R.M., Hogg, T.: An economics approach to hard computational problems. Science 275(5296), 51–54 (1997)

    Article  Google Scholar 

  15. 15.

    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: International Conference on Learning and Intelligent Optimization, pp 507–523. Springer (2011)

  16. 16.

    Hutter, F., Hoos, H.H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. J. Artif. Intell. Res. 36, 267–306 (2009)

    Article  Google Scholar 

  17. 17.

    Kirkpatrick, S., Gelatt, C.D. Jr., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)

    MathSciNet  Article  Google Scholar 

  18. 18.

    Karp, R.M.: Reducibility among combinatorial problems. In: Complexity of Computer Computations. Springer, US (1972)

  19. 19.

    Kroening, D., Strichman, O.: Decision Procedures: an Algorithmic Point of View. Texts in Theoretical Computer Science. An EATCS Series. Springer, Berlin (2010).

    Google Scholar 

  20. 20.

    Leino, K.R.M.: Dafny: an automatic program verifier for functional correctness. In: International Conference on Logic for Programming Artificial Intelligence and Reasoning, pp 348–370. Springer (2010)

  21. 21.

    López-Ibáñez, M., Dubois-Lacoste, J., Cáceres, L. P., Birattari, M., Stützle, T.: The irace package: iterated racing for automatic algorithm configuration. Operations Research Perspectives 3, 43–58 (2016)

    MathSciNet  Article  Google Scholar 

  22. 22.

    Malitsky, Y., Sellmann, M.: Instance-specific algorithm configuration as a method for non-model-based portfolio generation. Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimzation Problems.7298 244–259

  23. 23.

    Michel, L., Van Hentenryck, P.: Localizer a modeling language for local search. In: Smolka, G. (ed.) Principles and Practice of Constraint Programming-CP97, pp 237–251. Springer, Berlin (1997)

  24. 24.

    de Moura, L., Bjørner, N.: Z3: an Efficient SMT Solver, pp 337–340. Springer, Berlin (2008).

    Google Scholar 

  25. 25.

    Nemhauser, G.L., Wolsey, L.A., Fisher, M.L.: An analysis of approximations for maximizing submodular set functions - I. Math. Program. 14(1), 265–294 (1978)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Nof, Y.: Real time solving of discrete optimization problems. Master’s thesis, Technion, Israel Institute of Technology. Available online in (2017)

  27. 27.

    Petrik, M., Zilberstein, S.: Learning parallel portfolios of algorithms. Ann. Math. Artif. Intell. 48(1), 85–106 (2006)

    MathSciNet  MATH  Google Scholar 

  28. 28.

    Rubinstein, R., Kroese, D.: The Cross-Entropy Method: a Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer, New York (2004)

    Book  Google Scholar 

  29. 29.

    Sinz, C.: Towards an optimal cnf encoding of boolean cardinality constraints. CP 3709, 827–831 (2005)

    MATH  Google Scholar 

  30. 30.

    Wei, Y., Pei, Y., Furia, C.A., Silva, L.S., Buchholz, S., Meyer, B., Zeller, A.: Automated fixing of programs with contracts. In: Proceedings of the 19th International Symposium on Software Testing and Analysis, pp 61–72. ACM (2010)

  31. 31.

    Williamson, D.P., Shmoys, D.B.: The Design of Approximation Algorithms. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  32. 32.

    Yang, X.S.: Nature-Inspired Metaheuristic Algorithms, 2nd edn., pp 12–13. Luniver Press, United Kingdom (2010)

    Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Yair Nof.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nof, Y., Strichman, O. Real-time solving of computationally hard problems using optimal algorithm portfolios. Ann Math Artif Intell 89, 693–710 (2021).

Download citation


  • Algorithm portfolios
  • NP-optimization
  • Real-time

Mathematics Subject Classification (2010)

  • 68T20