# Using Expression Graphs in Optimization Algorithms

• David M. Gay
Conference paper
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 154)

## Abstract

An expression graph, informally speaking, represents a function in a way that can be manipulated to reveal various kinds of information about the function, such as its value or partial derivatives at specified arguments and bounds thereon in specified regions. (Various representations are possible, and all are equivalent in complexity, in that one can be converted to another in time linear in the expression’s size.) For mathematical programming problems, including the mixed-integer nonlinear programming problems that were the subject of the IMA workshop that led to this paper, there are various advantages to representing problems as collections of expression graphs. “Presolve” deductions can simplify the problem, e.g., by reducing the domains of some variables and proving that some inequality constraints are never or always active. To find global solutions, it is helpful sometimes to solve relaxed problems (e.g., allowing some “integer” variables to vary continuously or introducing convex or concave relaxations of some constraints or objectives), and to introduce “cuts” that exclude some relaxed variable values. There are various ways to compute bounds on an expression within a specified region or to compute relaxed expressions from expression graphs. This paper sketches some of them. As new information becomes available in the course of a branch-and-bound (or -cut) algorithm, some expression-graph manipulations and presolve deductions can be revisited and tightened, so keeping expression graphs around during the solution process can be helpful. Algebraic problem representations are a convenient source of expression graphs. One of my reasons for interest in the AMPL modeling language is that it delivers expression graphs to solvers.

## Key words

Expression graphs automatic differentiation bound computation constraint propagation presolve

## References

1. 1.
Autodi_ Web Site, http://www.autodiff.org.
2. 2.
Roscoe A. Bartlett, David M. Gay, and Eric T. Phipps, Automatic Di_erentiation of C++ Codes for Large-Scale Scienti_c Computing. In Computational Science { ICCS 2006, Vassil N. Alexandrov, Geert Dick van Albada, Peter M.A. Sloot, and Jack Dongarra (eds.), Springer, 2006, pp. 525{532.Google Scholar
3. 3.
Martin Berz, Di_erential Algebraic Description of Beam Dynamics to Very High Orders, Particle Accelerators 24 (1989), p. 109.Google Scholar
4. 4.
Martin Berz, yoko Makino, Khodr Shamseddine, Georg H. Hoffstatter, and Weishi Wan, COSY INFINITY and Its Applications in Nonlinear Dynamics, SIAM, 1996.Google Scholar
5. 5.
Stephen P. Boyd and Jacob Mattingley, CVXMOD | Convex Optimization Software in Python, http://cvxmod.net/, accessed July 2009.
6. 6.
R. Fourer and D.M. Gay, Experience with a Primal Presolve Algorithm. In Large Scale Optimization: State of the Art, W.W. Hager, D.W. Hearn, and P.M. Pardalos (eds.), Kluwer Academic Publishers, 1994, pp. 135{154.Google Scholar
7. 7.
R. Fourer, D.M. Gay, and B.W. Kernighan, A Modeling Language for Mathematical Programming, Management Science 36(5) (1990), pp. 519{554.Google Scholar
8. 8.
Robert Fourer, David M. Gay, and Brian W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, Duxbury Press/Brooks/Cole Publishing Co., 2nd edition, 2003.Google Scholar
9. 9.
Robert Fourer, Jun Ma, and Kipp Martin, An Open Interface for Hooking Solvers to Modeling Systems, slides for DIMACS Workshop on COIN-OR, 2006, http://dimacs.rutgers.edu/Workshops/COIN/slides/osil.pdf.
10. 10.
R. Fourer, C. Maheshwari, A. Neumaier, D. Orban, and H. Schichl, Convexity and Concavity Detection in Computational Graphs, manuscript, 2008, to appear in INFORMS J. Computing.Google Scholar
11. 11.
Edward P. Gatzke, John E. Tolsma, and Paul I. Barton, Construction of Convex Function Relaxations Using Automated Code Generation Techniques Optimization and Engineering 3, 2002, pp. 305{326.Google Scholar
12. 12.
Davd M. Gay, Automatic Di_erentiation of Nonlinear AMPL Models. In Automatic Di_erentiation of Algorithms: Theory, Implementation, and Application, A. Griewank and G. Corliss (eds.), SIAM, 1991, pp. 61{73.Google Scholar
13. 13.
Davd M. Gay, Hooking Your Solver to AMPL, AT&T Bell Laboratories, Numerical Analysis Manuscript 93–10, 1993 (revised 1997). http://www.ampl.com/REFS/hooking2.pdf.
14. 14.
Davd M. Gay, More AD of Nonlinear AMPL Models: Computing Hessian Information and Exploiting Partial Separability. In Computational Di_erentiation : Techniques, Applications, and Tools, Martin Berz, Christian Bischof, George Corliss and Andreas Griewank (eds.), SIAM, 1996, pp. 173{184.Google Scholar
15. 15.
Davd M. Gay, Semiautomatic Di_erentiation for E_cient Gradient Computations. In Automatic Di_erentiation: Applications, Theory, and Implementations, H. Martin Bucker, George F. Corliss, Paul Hovland and Uwe Naumann and Boyana Norris (eds.), Springer, 2005, pp. 147{158.Google Scholar
16. 16.
Davd M. Gay, Bounds from Slopes, report SAND-1010xxxx, to be available as http://www.sandia.gov/~dmgay/bounds10.pdf.
17. 17.
D.M. Gay, T. Head-Gordon, F.H. Stillinger, and M.H. Wright, An Application of Constrained Optimization in Protein Folding: The Poly-L-Alanine Hypothesis, Forefronts 8(2) (1992), pp. 4{6.Google Scholar
18. 18.
Andreas Griewank, On Automatic Di_erentiation. In Mathematical Programming: Recent Developments and Applications, M. Iri and K. Tanabe (eds.), Kluwer, 1989, pp. 83{108.Google Scholar
19. 19.
, Andreas Griewank, Evaluating Derivatives, SIAM, 2000.Google Scholar
20. 20.
A. Griewank, D. Juedes, and J. Utke, Algorithm 755: ADOL-C: A package for the automatic di_erentiation of algorithms written in C/C++, ACM Trans. Math Software 22(2) (1996), pp. 131{167.Google Scholar
21. 21.
R. Baker Kearfott An Overview of the GlobSol Package for Veri_ed Global Optimization, talk slides, 2002, http://www.mat.univie.ac.at/~neum/glopt/mss/Kea02.pdf.
22. 22.
, Padmanaban Kesavan, Russell J. Allgor, Edward P. Gatzke, and Paul I. Barton, Outer Approximation Algorithms for Separable Nonconvex Mixed- Integer Nonlinear Programs, Mathematical Programming 100(3), 2004, pp. 517{535.Google Scholar
23. 23.
R. Krawczyk and A. Neumaier, Interval Slopes for Rational Functions and Associated Centered Forms, SIAM J. Numer. Anal. 22(3) (1985), pp. 604{616.Google Scholar
24. 24.
R.E. Moore, Interval Arithmetic and Automatic Error Analysis in Digital Computing, Ph.D. dissertation, Stanford University, 1962.Google Scholar
25. 25.
Ramon E. Moore, Methods and Applications of Interval Analysis, SIAM, 1979.Google Scholar
26. 26.
Ramon E. Moore, R. Baker Kearfott, and Michael J. Cloud, Introduction to Interval Analysis, SIAM, 2009.Google Scholar
27. 27.
Ivo P. Nenov, Daniel H. Fylstra, and Lubomir V. Kolev, Convexity Determination in the Microsoft Excel Solver Using Automatic Di_erentiation Techniques, extended abstract, 2004, http://www.autodiff.org/ad04/abstracts/Nenov.pdf.
28. 28.
Arnold Neumaier, Interval Methods for Systems of Equations, Cambridge University Press, 1990.Google Scholar
29. 29.
Dominique Orban, Dr. AMPL Web Site, http://www.gerad.ca/~orban/drampl/
30. 30.
31. 31.
Dominique Orban and Robert Fourer, Dr. AMPL, A Meta Solver for Optimization, CORS/INFORMS Joint International Meeting, 2004, http://users.iems.northwestern.edu/~4er/SLIDES/ban0406h.pdf.
32. 32.
Polish notation, http://en.wikipedia.org/wiki/Polish_notation, accessed July 2009.
33. 33.
S.M. Rump, Expansion and Estimation of the Range of Nonlinear Functions, Mathematics of Computation 65(216) (1996), pp. 1503{1512.Google Scholar
34. 34.
35. 35.
Hermann Schichl and Arnold Neumaier, Interval Analysis on Directed Acyclic Graphs for Global Optimization Journal of Global Optimization 33(4) (2005), pp. 541{562.Google Scholar
36. 36.
Marco Schnurr, Steigungen hoeherer Ordnung zur veri_zierten globalen Optimierung, Ph.D. dissertation, Universitat Karlsruhe, 2007.Google Scholar
37. 37.
Marco Schnurr, The Automatic Computation of Second-Order Slope Tuples for Some Nonsmooth Functions, Electronic Transactions on Numerical Analysis 30 (2008), pp. 203{223.Google Scholar
38. 38.
Joseph G. Young Program Analysis and Transformation in Mathematical Programming, Ph.D. dissertation, Rice University, 2008.Google Scholar
39. 39.
Shen Zuhe and M.A. Wolfe, On Interval Enclosures Using Slope Arithmetic, Applied Mathematics and Computation 39 (1990), pp. 89{105.Google Scholar