Advertisement

Symbolic Bucket Elimination for Piecewise Continuous Constrained Optimization

  • Zhijiang Ye
  • Buser Say
  • Scott Sanner
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10848)

Abstract

Bucket elimination and its approximation extensions have proved to be effective techniques for discrete optimization. This paper addresses the extension of bucket elimination to continuous constrained optimization by leveraging the recent innovation of the extended algebraic decision diagram (XADD). XADDs support symbolic arithmetic and optimization operations on piecewise linear or univariate quadratic functions that permit the solution of continuous constrained optimization problems with a symbolic form of bucket elimination. The proposed framework is an efficient alternative for solving optimization problems with low tree-width constraint graphs without using a big-M formulation for piecewise, indicator, or conditional constraints. We apply this framework to difficult constrained optimization problems including XOR’s of linear constraints and temporal constraint satisfaction problems with “repulsive” preferences, and show that this new approach significantly outperforms Gurobi. Our framework also enables symbolic parametric optimization where closed-form solutions cannot be computed with tools like Gurobi, where we demonstrate a final novel application to parametric optimization of learned Relu-based deep neural networks.

Keywords

Bucket elimination Decision diagram Constrained optimization Symbolic dynamic programming 

References

  1. 1.
    Bahar, R.I., Frohm, E.A., Gaona, C.M., Hachtel, G.D., Macii, E., Pardo, A., Somenzi, F.: Algebraic decision diagrams and their applications. In: Proceedings of the 1993 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 1993, pp. 188–191. IEEE Computer Society Press, Los Alamitos (1993). http://dl.acm.org/citation.cfm?id=259794.259826
  2. 2.
    Bertele, U., Brioschi, F.: Nonserial Dynamic Programming. Academic Press Inc., Orlando (1972)zbMATHGoogle Scholar
  3. 3.
    Bistarelli, S., Montanari, U., Rossi, F.: Semiring-based constraint satisfaction and optimization. J. ACM 44(2), 201–236 (1997).  https://doi.org/10.1145/256303.256306MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bryant, R.E.: Graph-based algorithms for Boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986).  https://doi.org/10.1109/TC.1986.1676819CrossRefzbMATHGoogle Scholar
  5. 5.
    Davis, M., Putnam, H.: A computing procedure for quantification theory. J. ACM 7(3), 201–215 (1960).  https://doi.org/10.1145/321033.321034MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Dechter, R., Pearl, J.: Network-based heuristics for constraint satisfaction problems. In: Kanal, L., Kumar, V. (eds.) Search in Artificial Intelligence. SYMBOLIC, pp. 370–425. Springer, London (1988).  https://doi.org/10.1007/978-1-4613-8788-6_11. http://dl.acm.org/citation.cfm?id=60727.60738CrossRefGoogle Scholar
  7. 7.
    Dechter, R.: Bucket elimination: a unifying framework for reasoning. Artif. Intell. 113(1), 41–85 (1999). http://www.sciencedirect.com/science/article/pii/S0004370299000594MathSciNetCrossRefGoogle Scholar
  8. 8.
    Dechter, R.: Reasoning with Probabilistic and Deterministic Graphical Models: Exact Algorithms. Morgan & Claypool Publishers, San Rafael (2013)zbMATHGoogle Scholar
  9. 9.
    Dechter, R., Meiri, I., Pearl, J.: Temporal constraint networks. Artif. Intell. 49(1–3), 61–95 (1991).  https://doi.org/10.1016/0004-3702(91)90006-6MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2016). http://www.gurobi.com
  11. 11.
    Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-63387-9_5CrossRefGoogle Scholar
  12. 12.
    Khatib, L., Morris, P., Morris, R., Rossi, F.: Temporal constraint reasoning with preferences. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, IJCAI 2001, vol. 1, pp. 322–327. Morgan Kaufmann Publishers Inc., San Francisco (2001). http://dl.acm.org/citation.cfm?id=1642090.1642135
  13. 13.
    Larrosa, J., Dechter, R.: Boosting search with variable elimination in constraint optimization and constraint satisfaction problems. Constraints 8, 303–326 (2003)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Larrosa, J., Morancho, E., Niso, D.: On the practical use of variable elimination in constraint optimization problems: ‘still-life’ as a case study. J. Artif. Intell. Res. 23, 421–440 (2005)zbMATHGoogle Scholar
  15. 15.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML, pp. 807–814 (2010). http://www.icml2010.org/papers/432.pdf
  16. 16.
    Rollón, E., Larrosa, J.: Bucket elimination for multiobjective optimization problems. J. Heuristics 12(4), 307–328 (2006).  https://doi.org/10.1007/s10732-006-6726-yCrossRefzbMATHGoogle Scholar
  17. 17.
    Sanner, S., Abbasnejad, E.: Symbolic variable elimination for discrete and continuous graphical models. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1954–1960. AAAI Press (2012). http://dl.acm.org/citation.cfm?id=2900929.2901004
  18. 18.
    Sanner, S., Delgado, K., Barros, L.: Symbolic dynamic programming for discrete and continuous state MDPs. In: UAI, pp. 643–652, January 2011Google Scholar
  19. 19.
    Say, B., Wu, G., Zhou, Y.Q., Sanner, S.: Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear programming. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 750–756 (2017).  https://doi.org/10.24963/ijcai.2017/104
  20. 20.
    Zamani, Z., Sanner, S., Fang, C.: Symbolic dynamic programming for continuous state and action MDPs. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1839–1845. AAAI Press (2012). http://dl.acm.org/citation.cfm?id=2900929.2900988

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of TorontoTorontoCanada

Personalised recommendations