Skip to main content

Classifier-based constraint acquisition

Abstract

Modeling a combinatorial problem is a hard and error-prone task requiring significant expertise. Constraint acquisition methods attempt to automate this process by learning constraints from examples of solutions and (usually) non-solutions. Active methods query an oracle while passive methods do not. We propose a known but not widely-used application of machine learning to constraint acquisition: training a classifier to discriminate between solutions and non-solutions, then deriving a constraint model from the trained classifier. We discuss a wide range of possible new acquisition methods with useful properties inherited from classifiers. We also show the potential of this approach using a Naive Bayes classifier, obtaining a new passive acquisition algorithm that is considerably faster than existing methods, scalable to large constraint sets, and robust under errors.

References

  1. 1.

    Asdi, H.A., Bessiere, C., Ezzahir, R., Lazaar, N.: Time-bounded query generator for constraint acquisition. In: Proceedings of the 15th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research, Lecture Notes in Computer Science, vol. 10848, pp 1–17 (2018)

  2. 2.

    Arcangioli, R., Bessiere, C., Lazaar, N.: Multiple constraint acquisition. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence (2016)

  3. 3.

    Asafu-Adjei, J.K., Betensky, R.A.: A pairwise Naïve Bayes approach to Bayesian classification. Intern. J. Pattern Recognit. Artif. Intell. 29(7) (2015)

  4. 4.

    Bartolini, A., Lombardi, M., Milano, M., Benini, L.: Neuron constraints to model complex real-world problems. In: Proceedings of the 17th International Conference on Principles and Practice of Constraint Programming Lecture Notes in Computer Science, vol. 6876, pp 115–129 (2011)

  5. 5.

    Beldiceanu, N., Simonis, H.: Modelseeker: Extracting global constraint models from positive examples. In: Data Mining and Constraint Programming, Lecture Notes in Computer Science, vol. 10101, pp 77–95. Springer (2016)

  6. 6.

    Bessiere, C., Koriche, F., Lazaara, N., O’Sullivan, B.: Constraint acquisition. Artif. Intell. 244, 315–342 (2017)

    MathSciNet  Article  Google Scholar 

  7. 7.

    Bessiere, C., Coletta, R., Freuder, E.C., O’Sullivan, B.: Leveraging the learning power of examples in automated constraint acquisition. In: Proceedings of the 10th International Conference on Principles and Practice of Constraint Programming Lecture Notes in Computer Science, vol. 3258, pp 123–137 (2004)

  8. 8.

    Bessiere, C., Coletta, R., Daoudi, A., Lazaar, N., Bouyakhf, E.H.: Boosting constraint acquisition via generalization queries. In: Proceedings of the 21st European Conference on Artificial Intelligence, pp 99–104 (2014)

  9. 9.

    Bessiere, C., Coletta, R., Hebrard, E., Katsirelos, G., Lazaar, N., Narodytska, N., Quimper, C.-G., Walsh, T.: Constraint acquisition via partial queries. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pp 475–481. AAAI Press (2013)

  10. 10.

    Bonfietti, A., Lombardi, M., Milano, M.: Embedding decision trees and random forests in constraint programming. In: Proceedings of the International Conference on AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, Lecture Notes in Computer Science, vol. 9075, pp 74–90. Springer (2015)

  11. 11.

    Browne, D., Giering, M., Prestwich, S.D.: Pulse-net: Dynamic compression of convolutional neural networks. In: Proceedings of the IEEE 5th World Forum on Internet of Things (2019)

  12. 12.

    Casale, P., Pujol, O., Radeva, P.: Approximate convex hulls family for One-Class classification. In: proceedings of the International Workshop on Multiple Classifier Systems Lecture in Notes Computer Sci, vol. 6713, pp 106–115 (2011)

  13. 13.

    Cheng, B.M.W., Choi, K.M.F., Lee, H.H.M., Wu, J.C.K.: Increasing constraint propagation by redundant modeling: An experience report. Constraints 4, 167–192 (1999)

    Article  Google Scholar 

  14. 14.

    Domingos, P., Pazzani, M.: On the optimality of the simple bayesian classifier under Zero-One loss. Mach. Learn. 29, 103–130 (1997)

    Article  Google Scholar 

  15. 15.

    Fischetti, M., Jo, J.: Deep neural networks as 0-1 mixed integer linear programs: A feasibility study. Constraints 23(3), 296–309 (2018)

    MathSciNet  Article  Google Scholar 

  16. 16.

    Frankle, J., Carbin, M.: The lottery ticket hypothesis: Finding sparse, trainable neural networks. In: Proceedings of the International Conference on Learning Representations. to appear (2019)

  17. 17.

    Freuder, E.C.: Constraints: The ties that bind. In: Proceedings of the 21st National Conference on Artificial Intelligence, pp 1520–1523. AAAI Press (2006)

  18. 18.

    Freuder, E.C.: Progress towards the holy grail. Constraints 23, 158–171 (2018)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Freuder, E.C., Wallace, R.J.: Suggestion strategies for Constraint-Based matchmaker agents. Int. J. Artif. Intell. Tools 11(1), 3–18 (2002)

    Article  Google Scholar 

  20. 20.

    Gent, I.P., Petrie, K.E., Puget, J.-F.: Handbook of Constraint Programming. Elsevier, Amsterdam (2006)

    Google Scholar 

  21. 21.

    Good, I.J.: Turing’s anticipation of empirical Bayes in connection with the cryptanalysis of the naval enigma. J. Stat. Comput. Simul. 66(2), 101–111 (2000)

    MathSciNet  Article  Google Scholar 

  22. 22.

    Hnich, B., Prestwich, S.D., Selensky, E., Smith, B.M.: Constraint models for the covering test problem. Constraints 11(3), 199–219 (2006)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Kass, R.E., Raftery, A.E.: Bayes Factors. J. Amer. Stat. Assoc. 90(430), 773–795 (1995)

    MathSciNet  Article  Google Scholar 

  24. 24.

    Khan, S., Madden, M.: One-Class Classification: Taxonomy of study and review of techniques. Knowl. Eng. Rev. 29(3), 345–374 (2014)

    Article  Google Scholar 

  25. 25.

    Kolb, S., Paramonov, S., Guns, T., De Raedt, L.: Learning constraints in spreadsheets and tabular data. Mach. Learn. 106, 1441–1468 (2017)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Lallouet, A., Lopez, M., Martin, L., Vrain, C.: On learning constraint problems. In: Proceedings of the IEEE International Conference on Tools With Artificial Intelligence, pp 45–52 (2010)

  27. 27.

    Lallouet, A., Legtchenko, A.: Two contributions of constraint programming to machine learning. In: Proceedings of the European Conference on Machine Learning Lecture Notes in Artificial Intelligence, vol. 3720, pp 617–624. Springer (2005)

  28. 28.

    Lombardi, M., Milano, M.: Boosting combinatorial problem modeling with machine learning. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp 5472–5478 (2018)

  29. 29.

    Lombardi, M., Milano, M., Bartolini, A.: Empirical decision model learning. Artif. Intell. 244(Supplement C), 343–367 (2017)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Manning, C.D., Raghavan, P., Schütze, M.: Introduction to information retrieval. Cambridge University Press, Cambridge (2008)

    Book  Google Scholar 

  31. 31.

    Prestwich, S.D.: Robust constraint acquisition by sequential analysis. In: Proceedings of the 24th European Conference on Artificial Intelligence, Frontiers in Artificial Intelligence and Applications, vol. 325, pp 355–362. IOS Press (2020)

  32. 32.

    O’Sullivan, B.: Automated modelling and solving in constraint programming. In: Proceedings of the 24th AAAI Conference on Artificial Intelligence, pp 1493–1497 (2010)

  33. 33.

    Pawlak, T.P., Krawiec, K.: Automatic synthesis of constraints from examples using mixed integer linear programming. Eur. J. Oper. Res. 261(3), 1141–1157 (2017)

    MathSciNet  Article  Google Scholar 

  34. 34.

    De Raedt, L., Dehaspe, L.: Clausal discovery. Mach. Learn. 26, 99–146 (1997)

    Article  Google Scholar 

  35. 35.

    De Raedt, L., Dz̆eroski, S.: First Order jk-clausal Theories are PAC-learnable. Artif. Intell. 70, 375–392 (1994)

    MathSciNet  Article  Google Scholar 

  36. 36.

    De Raedt, L., Passerini, A., Reso, S.: Learning constraints from examples. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pp 7965–7970 (2018)

  37. 37.

    Say, B., Wu, G., Zhou, Y.Q., Sanner, S.: Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear programs. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp 750–756 (2017)

  38. 38.

    Smith, B.M., Stergiou, K., Walsh, T.: Modelling the Golomb Ruler Problem. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence (1999)

  39. 39.

    Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Proceedings of the 31st Conference on Neural Information Processing Systems (2017)

  40. 40.

    Tjeng, V., Tedrake, R.: Verifying neural networks with mixed integer programming. coRR (2017)

  41. 41.

    Tseitin, G.: On the complexity of derivation in propositional calculus. In: Siekmann, J., Wrightson, G. (eds.) Automation of Reasoning: Classical Papers in Computational Logic, vol. 2, pp 466–483. Springer (1983)

  42. 42.

    Tsouros, D.C., Stergiou, K., Sarigiannidis, P.G.: Efficient methods for constraint acquisition. In: Proceedings of the 24th International Conference on Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, vol. 11008, pp 373–388 (2018)

  43. 43.

    Tsouros, D.C., Stergiou, K., Bessiere, C.: Structure-driven multiple constraint acquisition. In: 25th International Conference on Principles and Practice of Constraint Programming Lecture Notes in Computer Science, vol. 11802, pp 709–725 (2019)

  44. 44.

    Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)

    Article  Google Scholar 

  45. 45.

    Verwer, S., Zhang, Y., Ye, Q.C.: Auction optimization using regression trees and linear models as integer programs. Artif. Intell. 244, 368–395 (2017)

    MathSciNet  Article  Google Scholar 

  46. 46.

    Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: Proceedings of the 30th Conference on Neural Information Processing Systems, pp 3637–3645 (2016)

  47. 47.

    Vu, X.-H., O’Sullivan, B.: A unifying framework for generalized constraint acquisition. Int. J. Artif. Intell. Tools 17(5), 803–833 (2008)

    Article  Google Scholar 

Download references

Acknowledgements

This material is based upon works supported by the Science Foundation Ireland under Grant No. 12/RC/2289-P2 which is co-funded under the European Regional Development Fund.

Funding

Open Access funding provided by the IReL Consortium.

Author information

Affiliations

Authors

Corresponding author

Correspondence to S. D. Prestwich.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Prestwich, S.D., Freuder, E.C., O’Sullivan, B. et al. Classifier-based constraint acquisition. Ann Math Artif Intell 89, 655–674 (2021). https://doi.org/10.1007/s10472-021-09736-4

Download citation

Keywords

  • Constraint acquisition
  • Classifier
  • Bayesian
  • Boolean satisfiability

Mathematics Subject Classification 2010

  • 68T99
  • 68Q32
  • 68R99