Advertisement

Continuous Search in Constraint Programming

  • Alejandro Arbelaez
  • Youssef Hamadi
  • Michèle Sebag

Abstract

This work presents the concept of Continuous Search (CS), whose objective is to allow any user to eventually get their constraint solver achieving a top performance on their problems. Continuous Search comes in two modes: the functioning mode solves the user’s problem instances using the current heuristics model; the exploration mode reuses these instances to train and improve the heuristics model through Machine Learning during the computer idle time. Contrasting with previous approaches, Continuous Search thus does not require that the representative instances needed to train a good heuristics model be available beforehand. It achieves lifelong learning, gradually becoming an expert on the user’s problem instance distribution. Experimental validation suggests that Continuous Search can design efficient mixed strategies after considering a moderate number of problem instances.

Keywords

Support Vector Machine Problem Instance Constraint Program Constraint Satisfaction Problem Constraint Solver 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Akbani, R., Kwek, S., Japkowicz, N.: Applying support vector machines to imbalanced datasets. In: J. F. Boulicaut, F. Esposito, F. Giannotti, D. Pedreschi (eds.) 15th European Conference on Machine Learning, Lecture Notes in Computer Science, vol. 3201, pp. 39–50. Springer, Pisa, Italy (2004) Google Scholar
  2. [2]
    Al-Shahib, A., Breitling, R., Gilbert, D. R.: Predicting protein function by machine learning on amino acid sequences – a critical evaluation. BMC Genomics 78(2) (2007) Google Scholar
  3. [3]
    Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: I. P. Gent (ed.) 15th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 5732, pp. 142–157. Springer, Lisbon, Portugal (2009) Google Scholar
  4. [4]
    Arbelaez, A., Hamadi, Y., Sebag, M.: Online heuristic selection in constraint programming. In: International Symposium on Combinatorial Search. Lake Arrowhead, USA (2009) Google Scholar
  5. [5]
    Bailey, T. L., Elkan, C.: Estimating the accuracy of learned concepts. In: IJCAI, pp. 895–901 (1993) Google Scholar
  6. [6]
    Beldiceanu, N., Carlsson, M., Demassey, S., Petit, T.: Global constraint catalogue: Past, present and future. Constraints 12(1), 21–62 (2007) CrossRefzbMATHMathSciNetGoogle Scholar
  7. [7]
    Birattari, M., Stützle, T., Paquete, L., Varrentrapp, K.: A racing algorithm for configuring metaheuristics. In: W. B. Langdon, E. Cantú-Paz, K. E. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. K. Burke, N. Jonoska (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, pp. 11–18. Morgan Kaufmann, New York, USA (2002) Google Scholar
  8. [8]
    Bordeaux, L., Hamadi, Y., Zhang, L.: Propositional satisfiability and constraint programming: A comparative survey. ACM Comput. Surv. 38(4) (2006) Google Scholar
  9. [9]
    Bordes, A., Ertekin, S., Weston, J., Bottou, L.: Fast kernel classifiers with online and active learning. Journal of Machine Learning Research 6, 1579–1619 (2005) zbMATHMathSciNetGoogle Scholar
  10. [10]
    Borrett, J. E., Tsang, E. P. K., Walsh, N. R.: Adaptive constraint satisfaction: The quickest first principle. In: W. Wahlster (ed.) 12th European Conference on Artificial Intelligence, pp. 160–164. John Wiley and Sons, Chichester, Budapest, Hungary (1996) Google Scholar
  11. [11]
    Boussemart, F., Hemery, F., Lecoutre, C., Sais, L.: Boosting systematic search by weighting constraints. In: R. L. de Mántaras, L. Saitta (eds.) Proceedings of the 16th European Conference on Artificial Intelligence, pp. 146–150. IOS Press, Valencia, Spain (2004) Google Scholar
  12. [12]
    Carchrae, T., Beck, J. C.: Low-knowledge algorithm control. In: D. L. McGuinness, G. Ferguson (eds.) Proceedings of the Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence, pp. 49–54. AAAI Press / The MIT Press, San Jose, California, USA (2004) Google Scholar
  13. [13]
    Carchrae, T., Beck, J. C.: Applying machine learning to low-knowledge control of optimization algorithms. Computational Intelligence 21(4), 372–387 (2005) CrossRefMathSciNetGoogle Scholar
  14. [14]
    Chang, C. C., Lin, C. J.: LIBSVM: A library for support vector machines (2001). Software from http://www.csie.ntu.edu.tw/~cjlin/libsvm
  15. [15]
    Correira, M., Barahona, P.: On the efficiency of impact based heuristics. In: P. J. Stuckey (ed.) 14th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 5202, pp. 608–612. Springer, Sydney, Australia (2008) CrossRefGoogle Scholar
  16. [16]
    Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press (2000) Google Scholar
  17. [17]
    Dasgupta, S., Hsu, D., Monteleoni, C.: A general agnostic active learning algorithm. In: J. C. Platt, D. Koller, Y. Singer, S. T. Roweis (eds.) Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems. MIT Press, Vancouver, British Columbia, Canada (2007) Google Scholar
  18. [18]
    Epstein, S. L., Freuder, E. C., Wallace, R., Morozov, A., Samuels, B.: The Adaptive Constraint Engine. In: P. Van Hentenryck (ed.) 8th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 2470, pp. 525–542. Springer, NY, USA (2002) Google Scholar
  19. [19]
    Gebruers, C., Hnich, B., Bridge, D. G., Freuder, E.C.: Using CBR to select solution strategies in constraint programming. In: H. Muñoz-Avila, F. Ricci (eds.) 6th International Conference on Case-Based Reasoning, Research and Development, Lecture Notes in Computer Science, vol. 3620, pp. 222–236. Springer, Chicago, IL, USA (2005) CrossRefGoogle Scholar
  20. [20]
    Gecode Team: Gecode: Generic constraint development environment (2006). Available from http://www.gecode.org
  21. [21]
    Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: Z. Ghahramani (ed.) Proceedings of the Twenty-Fourth International Conference on Machine Learning, ACM International Conference Proceeding Series, vol. 227, pp. 273–280. ACM, Corvalis, Oregon, USA (2007) Google Scholar
  22. [22]
    Gent, I. P., Miguel, I., Rendl, A.: Tailoring solver-independent constraint models: A case study with essence’ and minion. In: I. Miguel, W. Ruml (eds.) 7th International Symposium on Abstraction, Reformulation, and Approximation, Lecture Notes in Computer Science, vol. 4612, pp. 184–199. Springer, Whistler, Canada (2007) CrossRefGoogle Scholar
  23. [23]
    Gomes, C., Selman, B., Kautz, H.: Boosting combinatorial search through randomization. In: AAAI/IAAI, pp. 431–437 (1998) Google Scholar
  24. [24]
    Gomes, C. P., Selman, B.: Algorithm portfolios. Artif. Intell. 126(1-2), 43–62 (2001) CrossRefzbMATHMathSciNetGoogle Scholar
  25. [25]
    Haim, S., Walsh, T.: Restart strategy selection using machine learning techniques. In: O. Kullmann (ed.) 12th International Conference on Theory and Applications of Satisfiability Testing, Lecture Notes in Computer Science, vol. 5584, pp. 312–325. Springer, Swansea, UK (2009) Google Scholar
  26. [26]
    Haralick, R. M., Elliott, G. L.: Increasing tree search efficiency for constraint satisfaction problems. In: IJCAI, pp. 356–364. San Francisco, CA, USA (1979) Google Scholar
  27. [27]
    Hutter, F., Hamadi, Y., Hoos, H. H., Leyton-Brown, K.: Performance prediction and automated tuning of randomized and parametric algorithms. In: F. Benhamou (ed.) 12th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 4204, pp. 213–228. Springer, Nantes, France (2006) CrossRefGoogle Scholar
  28. [28]
    Hutter, F., Hoos, H. H., Stützle, T.: Automatic algorithm configuration based on local search. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, pp. 1152–1157. AAAI Press, Vancouver, British Columbia, Canada (2007) Google Scholar
  29. [29]
    Kautz, H. A., Horvitz, E., Ruan, Y., Gomes, C. P., Selman, B.: Dynamic restart policies. In: AAAI/IAAI, pp. 674–681 (2002) Google Scholar
  30. [30]
    Larochelle, H., Bengio, Y.: Classification using discriminative restricted Boltzmann machines. In: W. W. Cohen, A. McCallum, S. T. Roweis (eds.) Proceedings of the Twenty-Fifth International Conference on Machine Learning, ACM International Conference Proceeding Series, vol. 307, pp. 536–543. ACM, Helsinki, Finland (2008) Google Scholar
  31. [31]
    Luby, M., Sinclair, A., Zuckerman, D.: Optimal speedup of Las Vegas algorithms. In: ISTCS, pp. 128–133 (1993) Google Scholar
  32. [32]
    O’Mahony, E., Hebrard, E., Holland, A., Nugent, C., O’Sullivan, B.: Using case-based reasoning in an algorithm portfolio for constraint solving. In: Proceedings of the 19th Irish Conference on Artificial Intelligence and Cognitive Science (2008) Google Scholar
  33. [33]
    Petrovic, S., Epstein, S. L.: Random subsets support learning a mixture of heuristics. International Journal on Artificial Intelligence Tools 17(3), 501–520 (2008) CrossRefGoogle Scholar
  34. [34]
    Prasad, M. R., Biere, A., Gupta, A.: A survey of recent advances in SAT-based formal verification. STTT 7(2), 156–173 (2005) CrossRefGoogle Scholar
  35. [35]
    Pulina, L., Tacchella, A.: A multi-engine solver for quantified Boolean formulas. In: C. Bessiere (ed.) 13th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 4741, pp. 574–589. Springer, Providence, RI, USA (2007) Google Scholar
  36. [36]
    Refalo, P.: Impact-based search strategies for constraint programming. In: M. Wallace (ed.) 10th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 2004, pp. 557–571. Springer, Toronto, Canada (2004) Google Scholar
  37. [37]
    Richter, M. M., Aamodt, A.: Case-based reasoning foundations. Knowledge Eng. Review 20(3), 203–207 (2005) CrossRefGoogle Scholar
  38. [38]
    Rish, I., Brodie, M., Ma, S., et al.: Adaptive diagnosis in distributed systems. IEEE Trans. on Neural Networks 16, 1088–1109 (2005) CrossRefGoogle Scholar
  39. [39]
    Roussel, O., Lecoutre, C.: XML representation of constraint networks: Format XCSP 2.1. CoRR abs/0902.2362 (2009) Google Scholar
  40. [40]
    Samulowitz, H., Memisevic, R.: Learning to solve QBF. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence. AAAI Press, Vancouver, British Columbia (2007) Google Scholar
  41. [41]
    Shalabi, L. A., Shaaban, Z., Kasasbeh, B.: Data mining: A preprocessing engine. Journal of Computer Science 2, 735–739 (2006) CrossRefGoogle Scholar
  42. [42]
    Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1) (2008) Google Scholar
  43. [43]
    Streeter, M., Golovin, D., Smith, S. F.: Combining multiple heuristics online. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, pp. 1197–1203. AAAI Press, Vancouver, British Columbia, Canada (2007) Google Scholar
  44. [44]
    Vapnik, V.: The Nature of Statistical Learning. Springer Verlag, New York, NY, USA (1995) zbMATHGoogle Scholar
  45. [45]
    Witten, I. H., Frank, E.: Data Mining, Practical Machine Learning Tools and Techniques. Elsevier (2005) zbMATHGoogle Scholar
  46. [46]
    Wu, H., Beek, P. V.: Portfolios with deadlines for backtracking search. International Journal on Artificial Intelligence Tools 17, 835–856 (2008) CrossRefGoogle Scholar
  47. [47]
    Xu, L., Hutter, F., Hoos, H. H., Leyton-Brown, K.: The design and analysis of an algorithm portfolio for SAT. In: C. Bessiere (ed.) 13th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 4741, pp. 712–727. Springer, Providence, RI, USA (2007) Google Scholar
  48. [48]
    Xu, L., Hutter, F., Hoos, H. H., Leyton-Brown, K.: Satzilla: Portfolio-based algorithm selection for sat. Journal of Artificial Intelligence Research 32, 565–606 (2008) zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Alejandro Arbelaez
    • 1
  • Youssef Hamadi
  • Michèle Sebag
  1. 1.Microsoft-INRIA joint labOrsayFrance

Personalised recommendations