Symbolic Bucket Elimination for Piecewise Continuous Constrained Optimization

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10848)


Bucket elimination and its approximation extensions have proved to be effective techniques for discrete optimization. This paper addresses the extension of bucket elimination to continuous constrained optimization by leveraging the recent innovation of the extended algebraic decision diagram (XADD). XADDs support symbolic arithmetic and optimization operations on piecewise linear or univariate quadratic functions that permit the solution of continuous constrained optimization problems with a symbolic form of bucket elimination. The proposed framework is an efficient alternative for solving optimization problems with low tree-width constraint graphs without using a big-M formulation for piecewise, indicator, or conditional constraints. We apply this framework to difficult constrained optimization problems including XOR’s of linear constraints and temporal constraint satisfaction problems with “repulsive” preferences, and show that this new approach significantly outperforms Gurobi. Our framework also enables symbolic parametric optimization where closed-form solutions cannot be computed with tools like Gurobi, where we demonstrate a final novel application to parametric optimization of learned Relu-based deep neural networks.


Bucket elimination Decision diagram Constrained optimization Symbolic dynamic programming 


  1. 1.
    Bahar, R.I., Frohm, E.A., Gaona, C.M., Hachtel, G.D., Macii, E., Pardo, A., Somenzi, F.: Algebraic decision diagrams and their applications. In: Proceedings of the 1993 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 1993, pp. 188–191. IEEE Computer Society Press, Los Alamitos (1993).
  2. 2.
    Bertele, U., Brioschi, F.: Nonserial Dynamic Programming. Academic Press Inc., Orlando (1972)zbMATHGoogle Scholar
  3. 3.
    Bistarelli, S., Montanari, U., Rossi, F.: Semiring-based constraint satisfaction and optimization. J. ACM 44(2), 201–236 (1997). Scholar
  4. 4.
    Bryant, R.E.: Graph-based algorithms for Boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986). Scholar
  5. 5.
    Davis, M., Putnam, H.: A computing procedure for quantification theory. J. ACM 7(3), 201–215 (1960). Scholar
  6. 6.
    Dechter, R., Pearl, J.: Network-based heuristics for constraint satisfaction problems. In: Kanal, L., Kumar, V. (eds.) Search in Artificial Intelligence. SYMBOLIC, pp. 370–425. Springer, London (1988). Scholar
  7. 7.
    Dechter, R.: Bucket elimination: a unifying framework for reasoning. Artif. Intell. 113(1), 41–85 (1999). Scholar
  8. 8.
    Dechter, R.: Reasoning with Probabilistic and Deterministic Graphical Models: Exact Algorithms. Morgan & Claypool Publishers, San Rafael (2013)zbMATHGoogle Scholar
  9. 9.
    Dechter, R., Meiri, I., Pearl, J.: Temporal constraint networks. Artif. Intell. 49(1–3), 61–95 (1991). Scholar
  10. 10.
    Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2016).
  11. 11.
    Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). Scholar
  12. 12.
    Khatib, L., Morris, P., Morris, R., Rossi, F.: Temporal constraint reasoning with preferences. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, IJCAI 2001, vol. 1, pp. 322–327. Morgan Kaufmann Publishers Inc., San Francisco (2001).
  13. 13.
    Larrosa, J., Dechter, R.: Boosting search with variable elimination in constraint optimization and constraint satisfaction problems. Constraints 8, 303–326 (2003)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Larrosa, J., Morancho, E., Niso, D.: On the practical use of variable elimination in constraint optimization problems: ‘still-life’ as a case study. J. Artif. Intell. Res. 23, 421–440 (2005)zbMATHGoogle Scholar
  15. 15.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML, pp. 807–814 (2010).
  16. 16.
    Rollón, E., Larrosa, J.: Bucket elimination for multiobjective optimization problems. J. Heuristics 12(4), 307–328 (2006). Scholar
  17. 17.
    Sanner, S., Abbasnejad, E.: Symbolic variable elimination for discrete and continuous graphical models. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1954–1960. AAAI Press (2012).
  18. 18.
    Sanner, S., Delgado, K., Barros, L.: Symbolic dynamic programming for discrete and continuous state MDPs. In: UAI, pp. 643–652, January 2011Google Scholar
  19. 19.
    Say, B., Wu, G., Zhou, Y.Q., Sanner, S.: Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear programming. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 750–756 (2017).
  20. 20.
    Zamani, Z., Sanner, S., Fang, C.: Symbolic dynamic programming for continuous state and action MDPs. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1839–1845. AAAI Press (2012).

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of TorontoTorontoCanada

Personalised recommendations