Skip to main content
SpringerLink
Log in

Why CP Portfolio Solvers Are (under)Utilized? Issues and Challenges

  • Conference paper
  • First Online:
Book cover Logic-Based Program Synthesis and Transformation (LOPSTR 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9527))

Abstract

It is well recognized that a single, arbitrarily efficient solver can be significantly outperformed by a portfolio solver exploiting a combination of possibly slower on-average different solvers. Despite the success of portfolio solvers within the context of solving competitions, they are rarely used in practice. In this paper we give an overview of the main limitations that hinder the practical adoption and development of portfolio solvers within the Constraint Programming (CP) paradigm, discussing also possible ways to overcome them and potential extensions outside the CP field.

Supported by the EU project FP7-644298 HyVar: Scalable Hybrid Variability for Distributed, Evolving Software Systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The model file is 2DPacking.mzn while the data file is Class7_40_3.dzn.

  2. 2.

    sunny-cp attended the MiniZinc Challenge 2014 with respectable results (4th out of 18). It has also been awarded with an honourable mention by the challenge organizers.

References

  1. Abío, I., Stuckey, P.J.: Encoding linear constraints into SAT. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 75–91. Springer, Heidelberg (2014)

    Google Scholar 

  2. Amadini, R., Gabbrielli, M., Mauro J.: An enhanced features extractor for a portfolio of constraint solvers. In: SAC, pp. 1357–1359. ACM (2014)

    Google Scholar 

  3. Amadini, R., Gabbrielli, M., Mauro, J.: SUNNY: a lazy portfolio approach for constraint solving. TPLP 14(4–5), 509–524 (2014)

    MATH  Google Scholar 

  4. Amadini, R., Gabbrielli, M., Mauro J.: A multicore tool for constraint solving. In: IJCAI, pp. 232–238. AAAI Press (2015)

    Google Scholar 

  5. Amadini, R., Gabbrielli, M., Mauro, J.: Portfolio approaches for constraint optimization problems. In: AMAI, pp. 1–18 (2015)

    Google Scholar 

  6. Amadini, R., Gabbrielli, M., Mauro, J.: SUNNY-CP: a sequential CP portfolio solver. In: SAC, pp. 1861–1867. ACM (2015)

    Google Scholar 

  7. Amadini, R., Stuckey, P.J.: Sequential time splitting and bounds communication for a portfolio of optimization solvers. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 108–124. Springer, Heidelberg (2014)

    Google Scholar 

  8. Arbelaez, A., Hamadi, Y., Sebag, M.: Online heuristic selection in constraint programming. In: SoCS (2009)

    Google Scholar 

  9. Arbelaez, A., Hamadi, Y., Sebag, M.: Continuous search in constraint programming. In: ICTAI, pp. 219–243. IEEE Computer Society (2010)

    Google Scholar 

  10. Audemard, G., Hoessen, B., Jabbour, S., Lagniez, J-M., Piette, C.: PeneLoPe, a parallel clause-freezer solver. In: SAT Challenge, pp. 43–44 (2012)

    Google Scholar 

  11. Barahona, P., Hölldobler, S., Nguyen, V.-H.: Representative encodings to translate finite CSPs into SAT. In: Simonis, H. (ed.) CPAIOR 2014. LNCS, vol. 8451, pp. 251–267. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  12. Le Berre, D., Simon, L.: Preface to the special volume on the SAT 2005 competitions and evaluations. In: JSAT, 2(1–4), (2006)

    Google Scholar 

  13. Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M.T., Malitsky, Y., Fréchette, A., Hoos, H.H., Hutter, F., Leyton-Brown, K., Tierney, K., Vanschoren, J.: Aslib: A benchmark library for algorithm selection. CoRR, abs/1506.02465 (2015)

  14. Cenamor, I., de la Rosa, T., Fernández, F.: IBACOP and IBACOP2 Planner (2014). http://www.plg.inf.uc3m.es/~icenamor/files/IBaCoPPlanner.pdf

  15. Chu, G., de la Banda, M.G., Stuckey, P.J.: Automatically exploiting subproblem equivalence in constraint programming. In: Lodi, A., Milano, M., Toth, P. (eds.) CPAIOR 2010. LNCS, vol. 6140, pp. 71–86. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  16. Cipriano, R., Dovier, A., Mauro, J.: Compiling and executing declarative modeling languages to gecode. In: Garcia de la Banda, M., Pontelli, E. (eds.) ICLP 2008. LNCS, vol. 5366, pp. 744–748. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  17. claspfolio: http://www.cs.uni-potsdam.de/claspfolio/

  18. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2000)

    MATH  Google Scholar 

  19. Fawcett, C., Vallati, M., Hutter, F., Hoffmann, J., Hoos, H.H., Leyton-Brown, K.: Improved features for runtime prediction of domain-independent planners. In: ICAPS. AAAI (2014)

    Google Scholar 

  20. Frisch, A.M., Harvey, W., Jefferson, C., Martínez-Hernández, B., Miguel, I.: Essence: a constraint language for specifying combinatorial problems. Constraints 13(3), 268–306 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. FullContact. How We Used Immutable Servers to Simplify Our Cloud Infrastructure. (2014) http://www.fullcontact.com/blog/immutable-servers-benefits/

  22. Van Gelder, A.: Careful ranking of multiple solvers with timeouts and ties. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS, vol. 6695, pp. 317–328. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  23. Gent, I.P., Walsh, T.: CSPlib: a benchmark library for constraints. In: Jaffar, J. (ed.) CP 1999. LNCS, vol. 1713, pp. 480–481. Springer, Heidelberg (1999)

    Google Scholar 

  24. Geschwender, D., Hutter, F., Kotthoff, L., Malitsky, Y., Hoos, H.H., Leyton-Brown, K.: Algorithm configuration in the cloud: a feasibility study. In: Pardalos, P.M., Resende, M.G.C., Vogiatzis, C., Walteros, J.L. (eds.) Learning and Intelligent Optimization. Lecture Notes in Computer Science, pp. 41–46. Springer, Heidelberg (2014)

    Google Scholar 

  25. Gomes, C.P., Selman, B.: Algorithm portfolios. Artif. Intell. 126(1–2), 43–62 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  26. Guo, H., Hsu, W.H.: A machine learning approach to algorithm selection for NP-hard optimization problems: a case study on the MPE problem. Ann. OR 156(1), 61–82 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  27. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  28. Hamadi, Y., Jabbour, S., Sais, L.: ManySAT: a parallel SAT solver. JSAT 6(4), 245–262 (2009)

    MATH  Google Scholar 

  29. Hebrard, E., O’Mahony, E., O’Sullivan, B.: Constraint programming and combinatorial optimisation in numberjack. In: Lodi, A., Milano, M., Toth, P. (eds.) CPAIOR 2010. LNCS, vol. 6140, pp. 181–185. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  30. Hoos, H.H., Kaminski, R., Lindauer, M.T., Schaub, T.: aspeed: solver scheduling via answer set programming. In: TPLP (2015)

    Google Scholar 

  31. Hoos, H.H., Kaufmann, B., Schaub, T., Schneider, M.: Robust benchmark set selection for boolean constraint solvers. In: Nicosia, G., Pardalos, P. (eds.) LION 7. LNCS, vol. 7997, pp. 138–152. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  32. Lindauer, M., Hoos, H., Hutter, F.: From sequential algorithm selection to parallel portfolio selection. In: Jourdan, L., Dhaenens, C., Marmion, M.-E. (eds.) LION 9 2015. LNCS, vol. 8994, pp. 1–16. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  33. Hurley, B., Kotthoff, L., Malitsky, Y., O’Sullivan, B.: Proteus: a hierarchical portfolio of solvers and transformations. In: Simonis, H. (ed.) CPAIOR 2014. LNCS, vol. 8451, pp. 301–317. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  34. Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  35. Hutter, F., Hoos, H.H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. J. Artif. Intell. Res. (JAIR) 36, 267–306 (2009)

    MATH  Google Scholar 

  36. Hutter, F., Xu, L., Hoos, H.H., Leyton-Brown, K.: Algorithm runtime prediction: the state of the art. CoRR, abs/1211.0906 (2012)

  37. International planning competition: http://ipc.icaps-conference.org/

  38. Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee, J. (ed.) CP 2011. LNCS, vol. 6876, pp. 454–469. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  39. Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC - instance-specific algorithm configuration. In: Coelho, H., Studer, R., Wooldridge, M. (eds.) ECAI, Frontiers in Artificial Intelligence and Applications, vol. 215 pp. 751–756. IOS Press (2010)

    Google Scholar 

  40. Kiziltan, Z., Mauro, J.: Service-oriented volunteer computing for massively parallel constraint solving using portfolios. In: Lodi, A., Milano, M., Toth, P. (eds.) CPAIOR 2010. LNCS, vol. 6140, pp. 246–251. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  41. Kotthoff, L.: LLAMA: leveraging learning to automatically manage algorithms. CoRR, abs/1306.1031 (2013)

  42. Kotthoff, L.: Algorithm selection for combinatorial search problems: a survey. AI Mag. 35(3), 48–60 (2014)

    Google Scholar 

  43. Kotthoff, L.: Reliability of computational experiments on virtualised hardware. J. Exp. Theor. Artif. Intell. 26(1), 33–49 (2014)

    Article  Google Scholar 

  44. Kroer, C., Malitsky, Y.: Feature filtering for instance-specific algorithm configuration. In: ICTAI, pp. 849–855. IEEE (2011)

    Google Scholar 

  45. Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Parallel SAT solver selection and scheduling. In: Milano, M. (ed.) CP 2012. LNCS, vol. 7514, pp. 512–526. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  46. Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm portfolios based on cost-sensitive hierarchical clustering. In: IJCAI. AAAI (2013)

    Google Scholar 

  47. Maratea, M., Pulina, L., Ricca, F.: Multi-engine ASP solving with policy adaptation. J. Logic Comput. 1–22 (2013)

    Google Scholar 

  48. Maratea, M., Pulina, L., Ricca, F.: A multi-engine approach to answer-set programming. TPLP 14(6), 841–868 (2014)

    MathSciNet  Google Scholar 

  49. Max-SAT 2013: http://maxsat.ia.udl.cat/introduction/

  50. Morris, K.: Immutable server web page (2013). http://martinfowler.com/bliki/ImmutableServer.html

  51. mzn2feat-1.0 web page: http://www.cs.unibo.it/ amadini/mzn2feat-1.0.tar.bz2

  52. Nethercote, N., Stuckey, P.J., Becket, R., Brand, S., Duck, G.J., Tack, G.: MiniZinc: towards a standard CP modelling language. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741, pp. 529–543. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  53. Ohrimenko, O., Stuckey, P.J., Codish, M.: Propagation via lazy clause gener. Constraints 14(3), 357–391 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  54. O’Mahony, E., Hebrard, E., Holland, A., Nugent C., O’Sullivan, B.: Using case-based reasoning in an algorithm portfolio for constraint solving. In: AICS 2008 (2009)

    Google Scholar 

  55. Pulina, L., Tacchella, A.: A multi-engine solver for quantified boolean formulas. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741, pp. 574–589. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  56. Pulina, L., Tacchella, A.: A self-adaptive multi-engine solver for quantified boolean formulas. Constraints 14(1), 80–116 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  57. Rice, J.R.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)

    Article  Google Scholar 

  58. Roussel, O.: ppfolio. http://www.cril.univ-artois.fr/~roussel/ppfolio/

  59. Roussel O., Lecoutre, C.: XML representation of constraint networks: format XCSP 2.1. CoRR, abs/0902.2362 (2009)

  60. Sabharwal, A., Samulowitz, H.: Insights into parallelism with intensive knowledge sharing. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 655–671. Springer, Heidelberg (2014)

    Google Scholar 

  61. Samulowitz, H., Reddy, C., Sabharwal, A., Sellmann, M.: Snappy: a simple algorithm portfolio. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 422–428. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  62. SAT Challenge 2012: http://baldur.iti.kit.edu/SAT-Challenge-2012/

  63. Zinc Interface library(zinc). https://sicstus.sics.se/sicstus/docs/4.1.0/html/sicstus/lib_002dzinc.html#lib_002dzinc

  64. Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1) (2009)

    Google Scholar 

  65. Stojadinovic M., Maric, F.: Instance-based selection of CSP solvers using short training. In: Pragmatics of SAT (2014)

    Google Scholar 

  66. Stojadinovic, M., Maric, F.: meSAT: multiple encodings of CSP to SAT. Constraints 19(4), 380–403 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  67. Stuckey, P.J., Becket, R., Fischer, J.: Philosophy of the miniZinc challenge. Constraints 15(3), 307–316 (2010)

    Article  MATH  Google Scholar 

  68. Tamura, N., Taga, A., Kitagawa, S., Banbara, M.: Compiling finite linear CSP into SAT. Constraints 14(2), 254–272 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  69. Telelis, O., Stamatopoulos, P.: Combinatorial optimization through statistical instance-based learning. In: ICTAI, pp. 203–209 (2001)

    Google Scholar 

  70. Valenzano, R.A., Nakhost, H., Müller, M., Schaeffer, J., Sturtevant, N.R.: ArvandHerd: parallel planning with a portfolio. In: ECAI, pp. 786–791 (2012)

    Google Scholar 

  71. Xu, L., Hutter, F., Shen, J., Hoos, H., Leyton-Brown, K.: SATzilla2012: improved algorithm selection based on cost-sensitive classification models. In: SAT Challenge (2012)

    Google Scholar 

  72. Lin, X., Hutter, F., Hoos, H.H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. JAIR 32, 565–606 (2008)

    MATH  Google Scholar 

  73. Yun, X., Epstein, S.L.: Learning algorithm portfolios for parallel execution. In: Hamadi, Y., Schoenauer, M. (eds.) LION 2012. LNCS, vol. 7219, pp. 323–338. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering/Laboratory Focus INRIA, University of Bologna, Bologna, Italy

    Roberto Amadini & Maurizio Gabbrielli

  2. Department of Informatics, University of Oslo, Oslo, Norway

    Jacopo Mauro

Authors
  1. Roberto Amadini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Gabbrielli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jacopo Mauro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto Amadini .

Editor information

Editors and Affiliations

  1. Università di Siena, Siena, Italy

    Moreno Falaschi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Amadini, R., Gabbrielli, M., Mauro, J. (2015). Why CP Portfolio Solvers Are (under)Utilized? Issues and Challenges. In: Falaschi, M. (eds) Logic-Based Program Synthesis and Transformation. LOPSTR 2015. Lecture Notes in Computer Science(), vol 9527. Springer, Cham. https://doi.org/10.1007/978-3-319-27436-2_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27436-2_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27435-5

  • Online ISBN: 978-3-319-27436-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation