Abstract
In automatic differentiation, vertex elimination is one of the many methods for Jacobian accumulation and in general it can be much more efficient than the forward mode or reverse mode (Forth et al. ACM Trans Math Softw 30(3):266–299, 2004; Griewank and Walther, Evaluating derivatives: principles and techniques of algorithmic differentiation, SIAM, Philadelphia, 2008). However, finding the optimal vertex elimination sequence of a computational graph is a hard combinatorial optimization problem. In this paper, we propose to tackle this problem with an integer programming (IP) technique, and we develop an IP formulation for it. This enables us to use a standard integer optimization solver to find an optimal vertex elimination strategy. In addition, we have developed several bound-tightening and symmetry-breaking constraints to strengthen the basic IP formulation. We demonstrate the effectiveness of these enhancements through computational experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xpress-Optimizer Reference Manual (2009). URLhttp://fico.com/xpress
Albrecht, A., Gottschling, P., Naumann, U.: Markowitz-type heuristics for computing Jacobian matrices efficiently. In: Computational Science – ICCS 2003, LNCS, vol. 2658, pp. 575–584. Springer (2003). DOI 10. 1007/3-540-44862-4{ _}61
Forth, S.A., Tadjouddine, M., Pryce, J.D., Reid, J.K.: Jacobian code generated by source transformation and vertex elimination can be as efficient as hand-coding. ACM Transactions on Mathematical Software 30(3), 266–299 (2004). URLhttp://doi.acm.org/10.1145/1024074.1024076
Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd edn. No. 105 in Other Titles in Applied Mathematics. SIAM, Philadelphia, PA (2008). URLhttp://www.ec-securehost.com/SIAM/OT105.html
Miller, C.E., Tucker, A.W., Zemlin, R.A.: Integer programming formulation of traveling salesman problems. J. ACM 7, 326–329 (1960)
Mosenkis, V., Naumann, U.: On optimality preserving eliminations for the minimum edge count and optimal jacobian accumulation problems in linearized dags. Optimization Methods and Software 27, 337–358 (2012)
Naumann, U.: An enhanced Markowitz rule for accumulating Jacobians efficiently. In: K. Mikula (ed.) ALGORITHMY’2000 Conference on Scientific Computing, pp. 320–329 (2000)
Naumann, U.: Optimal Jacobian accumulation is NP-complete. Math. Prog. 112, 427–441 (2006). DOI 10.1007/s10107-006-0042-z
Naumann, U., Gottschling, P.: Simulated annealing for optimal pivot selection in Jacobian accumulation. In: A. Albrecht, K. Steinhöfel (eds.) Stochastic Algorithms: Foundations and Applications, no. 2827 in Lecture Notes in Computer Science, pp. 83–97. Springer (2003). DOI 10.1007/b13596
Naumann, U., Hu, Y.: Optimal vertex elimination in single-expression-use graphs. ACM Transactions on Mathematical Software 35(1), 1–20 (2008). DOI 10.1145/1377603.1377605
Rosenthal, R.E.: GAMS – A User’s Guide (2011)
Wolsey, L.A., Nemhauser, G.L.: Integer and Combinatorial Optimization. Wiley-Interscience (1999)
Acknowledgements
We thank Robert Luce for help on deriving an earlier IP formulation of the vertex elimination problem. This work was supported by the Office of Advanced Scientific Computing Research, Office of Science, U.S. Dept. of Energy, under Contract DE-AC02-06CH11357.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, J., Hovland, P., Munson, T., Utke, J. (2012). An Integer Programming Approach to Optimal Derivative Accumulation. In: Forth, S., Hovland, P., Phipps, E., Utke, J., Walther, A. (eds) Recent Advances in Algorithmic Differentiation. Lecture Notes in Computational Science and Engineering, vol 87. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30023-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-30023-3_20
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30022-6
Online ISBN: 978-3-642-30023-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)