Abstract
Bucket elimination and its approximation extensions have proved to be effective techniques for discrete optimization. This paper addresses the extension of bucket elimination to continuous constrained optimization by leveraging the recent innovation of the extended algebraic decision diagram (XADD). XADDs support symbolic arithmetic and optimization operations on piecewise linear or univariate quadratic functions that permit the solution of continuous constrained optimization problems with a symbolic form of bucket elimination. The proposed framework is an efficient alternative for solving optimization problems with low tree-width constraint graphs without using a big-M formulation for piecewise, indicator, or conditional constraints. We apply this framework to difficult constrained optimization problems including XOR’s of linear constraints and temporal constraint satisfaction problems with “repulsive” preferences, and show that this new approach significantly outperforms Gurobi. Our framework also enables symbolic parametric optimization where closed-form solutions cannot be computed with tools like Gurobi, where we demonstrate a final novel application to parametric optimization of learned Relu-based deep neural networks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For simplicity of exposition, we presume that non-binary discrete variables of cardinality k are encoded in binary with \(\lceil {\log _2(k)}\rceil \) boolean variables.
- 2.
We use \(\varvec{b}_{\setminus i}\) to denote the set \(\varvec{b}\) with the variable \(b_i\) excluded. Similarly \(\varvec{x}_{\setminus i}\) denotes exclusion of \(x_i\) from \(\varvec{x}\).
References
Bahar, R.I., Frohm, E.A., Gaona, C.M., Hachtel, G.D., Macii, E., Pardo, A., Somenzi, F.: Algebraic decision diagrams and their applications. In: Proceedings of the 1993 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 1993, pp. 188–191. IEEE Computer Society Press, Los Alamitos (1993). http://dl.acm.org/citation.cfm?id=259794.259826
Bertele, U., Brioschi, F.: Nonserial Dynamic Programming. Academic Press Inc., Orlando (1972)
Bistarelli, S., Montanari, U., Rossi, F.: Semiring-based constraint satisfaction and optimization. J. ACM 44(2), 201–236 (1997). https://doi.org/10.1145/256303.256306
Bryant, R.E.: Graph-based algorithms for Boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986). https://doi.org/10.1109/TC.1986.1676819
Davis, M., Putnam, H.: A computing procedure for quantification theory. J. ACM 7(3), 201–215 (1960). https://doi.org/10.1145/321033.321034
Dechter, R., Pearl, J.: Network-based heuristics for constraint satisfaction problems. In: Kanal, L., Kumar, V. (eds.) Search in Artificial Intelligence. SYMBOLIC, pp. 370–425. Springer, London (1988). https://doi.org/10.1007/978-1-4613-8788-6_11. http://dl.acm.org/citation.cfm?id=60727.60738
Dechter, R.: Bucket elimination: a unifying framework for reasoning. Artif. Intell. 113(1), 41–85 (1999). http://www.sciencedirect.com/science/article/pii/S0004370299000594
Dechter, R.: Reasoning with Probabilistic and Deterministic Graphical Models: Exact Algorithms. Morgan & Claypool Publishers, San Rafael (2013)
Dechter, R., Meiri, I., Pearl, J.: Temporal constraint networks. Artif. Intell. 49(1–3), 61–95 (1991). https://doi.org/10.1016/0004-3702(91)90006-6
Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2016). http://www.gurobi.com
Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_5
Khatib, L., Morris, P., Morris, R., Rossi, F.: Temporal constraint reasoning with preferences. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, IJCAI 2001, vol. 1, pp. 322–327. Morgan Kaufmann Publishers Inc., San Francisco (2001). http://dl.acm.org/citation.cfm?id=1642090.1642135
Larrosa, J., Dechter, R.: Boosting search with variable elimination in constraint optimization and constraint satisfaction problems. Constraints 8, 303–326 (2003)
Larrosa, J., Morancho, E., Niso, D.: On the practical use of variable elimination in constraint optimization problems: ‘still-life’ as a case study. J. Artif. Intell. Res. 23, 421–440 (2005)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML, pp. 807–814 (2010). http://www.icml2010.org/papers/432.pdf
Rollón, E., Larrosa, J.: Bucket elimination for multiobjective optimization problems. J. Heuristics 12(4), 307–328 (2006). https://doi.org/10.1007/s10732-006-6726-y
Sanner, S., Abbasnejad, E.: Symbolic variable elimination for discrete and continuous graphical models. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1954–1960. AAAI Press (2012). http://dl.acm.org/citation.cfm?id=2900929.2901004
Sanner, S., Delgado, K., Barros, L.: Symbolic dynamic programming for discrete and continuous state MDPs. In: UAI, pp. 643–652, January 2011
Say, B., Wu, G., Zhou, Y.Q., Sanner, S.: Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear programming. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 750–756 (2017). https://doi.org/10.24963/ijcai.2017/104
Zamani, Z., Sanner, S., Fang, C.: Symbolic dynamic programming for continuous state and action MDPs. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1839–1845. AAAI Press (2012). http://dl.acm.org/citation.cfm?id=2900929.2900988
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Ye, Z., Say, B., Sanner, S. (2018). Symbolic Bucket Elimination for Piecewise Continuous Constrained Optimization. In: van Hoeve, WJ. (eds) Integration of Constraint Programming, Artificial Intelligence, and Operations Research. CPAIOR 2018. Lecture Notes in Computer Science(), vol 10848. Springer, Cham. https://doi.org/10.1007/978-3-319-93031-2_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-93031-2_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93030-5
Online ISBN: 978-3-319-93031-2
eBook Packages: Computer ScienceComputer Science (R0)