Evolving Parameterised Policies for Stochastic Constraint Programming

  • Steven Prestwich
  • S. Armagan Tarim
  • Roberto Rossi
  • Brahim Hnich
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5732)


Stochastic Constraint Programming is an extension of Constraint Programming for modelling and solving combinatorial problems involving uncertainty. A solution to such a problem is a policy tree that specifies decision variable assignments in each scenario. Several solution methods have been proposed but none seems practical for large multi-stage problems. We propose an incomplete approach: specifying a policy tree indirectly by a parameterised function, whose parameter values are found by evolutionary search. On some problems this method is orders of magnitude faster than a state-of-the-art scenario-based approach, and it also provides a very compact representation of policy trees.


Policy Tree Constraint Program Hard Constraint Chance Constraint Evolutionary Search 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balafoutis, T., Stergiou, K.: Algorithms for Stochastic CSPs. In: Benhamou, F. (ed.) CP 2006. LNCS, vol. 4204, pp. 44–58. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    Benoist, T., Bourreau, E., Caseau, Y., Rottembourg, B.: Towards Stochastic Constraint Programming: A Study of On-Line Multi-Choice Knapsack with Deadlines. In: Walsh, T. (ed.) CP 2001. LNCS, vol. 2239, pp. 61–76. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  3. 3.
    Beyer, H.-G.: Evolutionary Algorithms in Noisy Environments: Theoretical Issues and Guidelines for Practice. Computer Methods in Applied Mechanics and Engineering 186(2-4), 239–267 (2000)CrossRefzbMATHGoogle Scholar
  4. 4.
    Bordeaux, L., Samulowitz, H.: On the Stochastic Constraint Satisfaction Framework. In: ACM Symposium on Applied Computing, pp. 316–320 (2007)Google Scholar
  5. 5.
    Fogel, D.B., Chellapilla, K.: Verifying Anaconda’s Expert Rating by Competing Against Chinook: Experiments in Co-Evolving a Neural Checkers Player. Neurocomputing 42(1-4), 69–86 (2002)CrossRefzbMATHGoogle Scholar
  6. 6.
    Fogel, D.B., Hays, T.J., Hahn, S.L., Quon, J.: A Self-Learning Evolutionary Chess Program. Proceedings of the IEEE 92(12), 1947–1954 (2004)CrossRefGoogle Scholar
  7. 7.
    Gomez, F., Schmidhuber, J., Miikkulainen, R.: Efficient Non-Linear Control Through Neuroevolution. Journal of Machine Learning Research 9, 937–965 (2008)zbMATHGoogle Scholar
  8. 8.
    Hewahi, N.M.: Engineering Industry Controllers Using Neuroevolution. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 19(1), 49–57 (2005)CrossRefGoogle Scholar
  9. 9.
    Jin, Y., Branke, J.: Evolutionary Optimization in Uncertain Environments — a Survey. IEEE Transactions on Evolutionary Computation 9(3), 303–317 (2005)CrossRefGoogle Scholar
  10. 10.
    Littman, M.L., Majercik, S.M., Pitassi, T.: Stochastic Boolean Satisfiability. Journal of Automated Reasoning 27(3), 251–296 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Lubberts, A., Miikkulainen, R.: Co-Evolving a Go-Playing Neural Network. In: Genetic and Evolutionary Computation Conference, pp. 14–19. Kaufmann, San Francisco (2001)Google Scholar
  12. 12.
    Majercik, S.M.: APPSSAT: Approximate Probabilistic Planning Using Stochastic Satisfiability. International Journal of Approximate Reasoning 45(2), 402–419 (2007)CrossRefzbMATHGoogle Scholar
  13. 13.
    Majercik, S.M.: Stochastic Boolean Satisfiability. In: Handbook of Satisfiability, ch. 27, pp. 887–925. IOS Press, Amsterdam (2009)Google Scholar
  14. 14.
    Pollack, J.B., Blair, A.D.: Co-Evolution in the Successful Learning of Backgammon Strategy. Machine Learning 32(3), 225–240 (1998)CrossRefzbMATHGoogle Scholar
  15. 15.
    Prestwich, S.D., Tarim, S.A., Rossi, R., Hnich, B.: A Steady-State Genetic Algorithm With Resampling for Noisy Inventory Control. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 559–568. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  16. 16.
    Rossi, R., Tarim, S.A., Hnich, B., Prestwich, S.D.: Cost-Based Domain Filtering for Stochastic Constraint Programming. In: Stuckey, P.J. (ed.) CP 2008. LNCS, vol. 5202, pp. 235–250. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  17. 17.
    Stanley, K.O., Miikkulainen, R.: Evolving Neural Networks Through Augmenting Topologies. Evolutionary Computation 10(2), 99–127 (2002)CrossRefGoogle Scholar
  18. 18.
    Tarim, S.A., Manandhar, S., Walsh, T.: Stochastic Constraint Programming: A Scenario-Based Approach. Constraints 11(1), 1383–7133 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Tarim, S.A., Miguel, I.: A Hybrid Bender’s Decomposition Method for Solving Stochastic Constraint Programs with Linear Recourse. In: Hnich, B., Carlsson, M., Fages, F., Rossi, F. (eds.) CSCLP 2005. LNCS (LNAI), vol. 3978, pp. 133–148. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  20. 20.
    Walsh, T.: Stochastic Constraint Programming. In: 15th European Conference on Artificial Intelligence (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Steven Prestwich
    • 1
  • S. Armagan Tarim
    • 2
  • Roberto Rossi
    • 3
  • Brahim Hnich
    • 4
  1. 1.Cork Constraint Computation CentreUniversity College CorkIreland
  2. 2.Operations Management DivisionNottingham University Business SchoolNottinghamUK
  3. 3.Logistics, Decision and Information Sciences GroupWageningen URThe Netherlands
  4. 4.Faculty of Computer ScienceIzmir University of EconomicsTurkey

Personalised recommendations