Lagrangian Models

Abstract

Lagrangian models use calculus to solve multi-variable non-linear constrained optimization models of problems and for identifying the marginal changes (‘shadow prices’) of optimal solutions to changes in constraint bounds. This is especially useful when the constraints represent resource limitations.

11.1 Introduction

Joseph-Louis Lagrange is usually considered to be a French mathematician, but the Italian Encyclopedia refers to him as an Italian mathematician who lived from 1736–1813. Among numerous other honors, a street, Rue Lagrange, in Paris, is named after him (Fig. 11.1).

Fig. 11.1

Joseph-Louis Lagrange and a street in Paris having his name. https://en.wikipedia.org/wiki/Joseph-Louis_LagrangeCreativeCommonsAttribution-ShareAlikeLicense

Joseph-Louis Lagrange is famous for lots of things he did in his life, but for us, he showed how differential calculus could be used to find solutions to constrained non-linear models. But, since it is calculus-based, all the limitations of calculus apply. It is limited to continuous functions. It produces a maximum for objective functions that are concave and a minimum for objectives that are convex. It ignores constants. But it gives us an opportunity to better understand the concept and find the values of shadow prices.

11.2 Constructing Lagrangian Optimization Models

Lagrange’s approach for finding the maximum or minimum value of some objective function F(X) and associated values of all the decision variables X = {x₁, x₂, x₃,.., x_j, … x_n} also determines what economists call ‘shadow prices,’ and operations researchers call ‘dual variables’ or ‘dual prices,’ associated with each constraint, g_i(X) = b_i. Each constraint’s shadow price is the change in the value of the objective function F(X) given a unit change in the constraint’s b_i value. These shadow prices are also called Lagrangian multipliers, typically denoted as λ_i.

$$\lambda_{{\text{i}}} = d{\text{F}}\left( X \right)/d{\text{b}}_{{\text{i}}} \,{\text{for }}\,{\text{each }}\,{\text{constraint }}\,{\text{i}}.$$

The modeling approach involves combining the objective function F(X) and all the constraints expressed as equalities, g_i(X) = b_i, into a single function L(X, λ). The unknown variables are the original ones contained in the vector X and all the Lagrange multiplier variables, λ_i, one for each constraint i.

Each constraint g_i(X) = b_i in L(X, λ) is multiplied by its Lagrangian multiplier λ_i. Their sum is subtracted from F(X). The result is

$${\text{L}}\left( {{\mathbf{X,}}{\varvec{\lambda}}} \right) = {\text{F}}\left( {\varvec{X}} \right) - \sum\limits_{{\text{i}}} {\lambda_{{\text{i}}} } \left( {{\text{g}}_{{\text{i}}} \left( {\varvec{X}} \right) - {\text{b}}_{{\text{i}}} } \right).$$

Setting inequality constraints originally of the form g_i(X) ≤ b_i or g_i(X) ≥ b_i to equalities when one is not sure if they are equalities or not, may involve the addition, or subtraction as appropriate, of the square of an additional unknown slack or surplus variable. For this discussion, assume such variables if needed are included in the vector X. These so-called slack or surplus variables are squared to insure each is non-negative.

Equating to 0 each of the partial derivatives of L(X, λ) with respect to each of the unknown variables, in X and λ results in a set of equations that when simultaneously solved will identify the values of each of the unknown variables in X and shadow prices in λ that maximize or minimize F(X). The procedure is the same whether the objective function F(X) is to be maximized or minimized. Therefore, one should check to see if the solution is a maximum or minimum value. Again, the λ_i values are the shadow prices associated with the b_i values of each constraint i.

$$\begin{gathered} {\text{L}}\left( {{\varvec{X}},{{\varvec{\uplambda}}}} \right) = {\text{F}}\left( {\varvec{X}} \right) \, - \, \sum\limits_{{\text{i}}} {\lambda_{{\text{i}}} \left( {{\text{g}}_{{\text{i}}} \left( {\varvec{X}} \right) \, {-}{\text{ b}}_{{\text{i}}} } \right)} \hfill \\ \partial {\text{L}}/\partial x_{{\text{j}}} = \, 0 \, = \, \partial {\text{F}}/\partial x_{{\text{j}}} - \, \sum\limits_{{\text{i}}} {\lambda_{{\text{i}}} \partial \left( {{\text{g}}_{{\text{i}}} \left( {\varvec{X}} \right)} \right)/\partial x_{{\text{j}}} \,{\text{for }}\,{\text{all }}\,{\text{variables}}\,{\text{x}}_{{\text{j}}} } \hfill \\ \partial {\text{L}}/\partial \lambda_{{\text{i}}} = \, 0 \, = \, \left( {{\text{g}}_{{\text{i}}} \left( \varvec{X} \right) \, {-}{\text{ b}}_{{\text{i}}} } \right)\,{\text{ for}}\,{\text{ all }}\,{\text{constraints}}\,{\text{ i}}. \hfill \\ \end{gathered}$$

Before showing some specific examples, consider the constraints where a surplus or slack variable, x_i², had to be added or subtracted from the left-hand side of a constraint to form an equality. When the partial derivative of L(X, λ) with respect to that variable is set to 0 the result is

$$\partial {\text{L}}/\partial x_{{\text{i}}} = \, 0 \, = - 2\,x_{{\text{i}}} \,\lambda_{{\text{i}}} .$$

Note that either x_i or λ_i or both will equal 0. If the constraint is binding there will be no inequality, and the value of the slack or surplus variable x_i will be 0. If the constraint is not binding (does not affect the values of the other variables x_j) then λ_i will equal 0. There will be no change in F(X) given a small change in b_i.

11.3 Example Lagrangian Models

Consider finding the minimum length of fence needed to enclose a rectangular area of at least A or of finding the maximum rectangular area that can be enclosed by a fence of length P. The area’s perimeter = 2(length) + 2(width). Clearly, the solution is length = width = √A. Solving a Lagrangian model will also identify the Lagrangian multipliers, i.e., the shadow prices, associated with the available resource, i.e., area or length of fencing. Letting l and w be the unknown length and width of the rectangular area A

$$\begin{gathered} {\text{L = 2}}\left( {1 + w} \right) - \lambda \left( {lw - {\text{A}}} \right) \hfill \\ \partial {\text{L/}}\partial l = 0 = 2 - \lambda w \hfill \\ \partial {\text{L/}}\partial w = 0 = 2 - \lambda l \hfill \\ \partial {\text{L/}}\partial \lambda = 0 = lw - {\text{A}}{.} \hfill \\ \end{gathered}$$

One can see from the first two partial differential equations that l = w, thus from the third partial differential equation, l and w = √A. Hence, the shadow price associated with the area, λ, is 2/√A. This is how much more fencing is required for a unit increase in A.

Alternatively, if the total length of fencing is P, we can find the maximum area A (lw) enclosed by P

$$\begin{gathered} {\text{L = }}lw - \lambda \left( {2\left( {l + w} \right) - {\text{P}}} \right) \hfill \\ \partial {\text{L/}}\partial l = 0 = w - 2\lambda \hfill \\ \partial {\text{L/}}\partial w = 0 = l - 2\lambda \hfill \\ \partial {\text{L/}}\partial \lambda = 0 = 2\left( {l + w} \right) - {\text{P}}{.} \hfill \\ \end{gathered}$$

Again, from the first two partial differential equations, l = w, and from the third, l or w equals P/4. Hence the shadow price associated with the perimeter P, λ, equals P/8. This is how much more area is obtained for a unit increase in P.

This model can be extended to one of finding the least-cost dimensions of a storage tank containing a volume of V. Let the average cost per unit of the base area = Cb. Similarly, let the average cost per unit side area = Cs and the average cost per unit top area = Ct. Before we can model this tank, we need to decide on its shape. Here we can consider two different tank shapes, a rectangular and a cylindrical one. Of course, it is possible to pick anything in between these two shapes. The stated objective is to

$$\begin{gathered} {\text{Minimize}}\,Totalcost \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{\text{Subject}}\,{\text{to:}}\,\,\,Totalcost = basecost + sidecost + topcost \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{\text{Volume}}\,{\text{of}}\,{\text{tank}}\, \ge {\text{required}}\,{\text{volume}} \hfill \\ \end{gathered}$$

$$\begin{gathered} {\text{Assuming}}\,{\text{ a}}\,{\text{ rectangular}}\,{\text{ tank }}\,{\text{having }}\,{\text{dimensions}}\,{\text{ of}}\,L,\,W, \, \,{\text{and}}\,H: \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,Basecost = {\text{Cb}}\,L\,W \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,Sidecost = {\text{Cs}}\,{2}\,H\,\left( {L + W} \right) \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,Topcost = {\text{Ct}}\,L\,W \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,L\,W\,H\, \ge {\text{V}}{.} \hfill \\ \end{gathered}$$

$$\begin{gathered} {\text{Assuming }}\,{\text{a }}\,{\text{cylindrical}}\,{\text{ tank }}\,{\text{having }}\,{\text{dimensions}}\,R\,{\text{and}}\,H: \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, Basecost = {\text{ Cb}}\left( {\pi R^{{2}} } \right) \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,Sidecost = {\text{Cs}}\left( {2\pi RH} \right) \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,topcost = {\text{Ct}}\left( {\pi R^{2} } \right) \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\pi \,R^{2} \,H\, \ge {\text{V}}{.} \hfill \\ \end{gathered}$$

Clearly, for each of the two-volume capacity constraints, the least-cost solution will result in; an equality. Constructing a bigger tank than required just costs more.

Using Lagrangian models. For a rectangular tank with dimensions l, w, and h:

$$\begin{gathered} {\text{L = 2}}\,{\text{Cs}}\,h\left( {l + w} \right) + \left( {\text{Cb + Ct}} \right)\,l\,w - \lambda \left( {l\,w\,h - {\text{V}}} \right) \hfill \\ \partial {\text{L/}}\partial l = 0 = 2\,{\text{Cs}}\,h + \left( {\text{Cb + Ct}} \right)w - \lambda \,w\,h \hfill \\ \partial {\text{L/}}\partial w = 0 = 2\,{\text{Cs}}\,h + \left( {\text{Cb + Ct}} \right)l - \lambda \,l\,h \hfill \\ \partial {\text{L/}}\partial h = 0 = 2\,{\text{Cs}}\,\left( {l{\text{ + w}}} \right) - \lambda \,l\,w \hfill \\ \partial {\text{L/}}\partial \lambda = 0 = l\,w\,h - {\text{V}} \hfill \\ \end{gathered}$$

From these first three partial differential equations, one can prove that the width w equals the length l, and that both = 2 Cs h/(Cb + Ct). Since from the last equation, h = V/ww, substituting that into {w = l = 2 Cs h/(Cb+Ct)} yields

$$w = l = \left[ {2\,{\text{Cs}}\,{\text{V/}}\left( {\text{Cb + Ct}} \right)} \right]^{1/3}$$

and

$$h = {\text{Vol/}}\left[ {2\,{\text{Cs}}\,{\text{V/}}\left( {\text{Cb + Ct}} \right)} \right]^{2/3} \,\,{\text{or}}\,\,h{\text{ = V}}^{{1/3}} \,\left[ {\left( {\text{Cb + Ct}} \right)/2\,{\text{Cs}}} \right]^{2/3}$$

The shadow price associated with volume V will denote the change in the total cost per unit change in volume V. The total cost increases if V is increased. What is interesting about all such tank or container problems is that the ratio of base and top cost to total cost will equal 1/3 no matter what the tank shape and unit costs and volumes are. The total side cost will always be 2/3^rds of the total cost of a minimum-cost tank or container.

$$\begin{gathered} {\text{For}}\,{\text{ a }}\,{\text{circular}}\,{\text{ tank }}\,{\text{with }}\,{\text{dimensions'}} \,r\,{\text{and}}\,h: \hfill \\ {\text{L }} = {\text{ Cs}}\,{ 2 }\pi r \, h + \, \left( {{\text{Cb }} + {\text{ Ct}}} \right) \, \pi r^{{2}} {-} \, \lambda \, \, \pi r^{{2}} h{-}{\text{ V}}] \hfill \\ \partial {\text{L/}}\partial r = 0 = 2\,{\text{Cs}}\,h\,\pi + 2\left( {\text{Cb + Ct}} \right)\pi r - 2\lambda \pi rh \hfill \\ \partial {\text{L/}}\partial r = 0 = 2\,{\text{Cs}}\,r\,\pi - \lambda \,\pi r^{2} \hfill \\ \partial {\text{L/}}\partial \lambda = 0 = \pi \,r^{2} h - {\text{V}}{.} \hfill \\ \end{gathered}$$

Use first two partial differential equations to find that r/h = Cs/(Cb+Ct)

Using the third equation

$$\begin{gathered} r = \left[ {{\text{V}}\,{\text{Cs/}}\pi \left( {\text{Cb + Ct}} \right)} \right]^{1/3} \hfill \\ h = \left[ {{\text{V}}\,\left( {\text{Cb + Ct}} \right)^{2} /\pi \,{\text{Cs}}^{{2}} } \right]^{1/3} . \hfill \\ \end{gathered}$$

Now look at cost ratios:

$$\begin{gathered} {\text{Side}}\,{\text{ cost/total}}\,{\text{cost }} = {\text{ Cs}}\,{2}\pi rh{/}\left[ {{\text{Cs }}\,{2}\pi rh + \, \left( {{\text{Cb }} + {\text{ Ct}}} \right) \, \pi r^{{2}} } \right] \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = 2\,{\text{Cs}}\,h/\left[ {2\,{\text{Cs}}\,h + \left( {\text{Cb + Ct}} \right)r} \right] \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = 2\,{\text{Cs}}\,h/\left[ {2\,{\text{Cs}}\,h + \left( {\text{Cb + Ct}} \right)h\,{\text{Cs/}}\left( {\text{Cb + Ct}} \right)} \right] = 2/3. \hfill \\ \end{gathered}$$

This is true regardless of unit costs Cs, Ct, Cb or V!

Finally, consider the resource allocation problem.

Assume the goal is to maximize the total benefits derived from the allocation of resources to three users. Denote the allocations as X, Y, and Z to users 1, 2, and 3, respectively. The benefits obtained from each allocation are 6 X – X², 7Y – 1.5Y², and 8Z – 0.5Z². Assuming that only 6 resources are available, the problem is to find the allocations that

$$\begin{gathered} {\text{Maximize}}\,{\text{F}}\left( {X,Y,Z} \right) = \left( {6X - X^{2} } \right) + \left( {7Y - 1.5Y^{2} } \right) + \left( {8Z - 0.5Z^{2} } \right) \hfill \\ {\text{Subjected}}\,{\text{to}}:X + Y + Z = 6. \hfill \\ \end{gathered}$$

The resource constraint is an equality since more resources are desired, and therefore, all 6 resources will be allocated.

The marginal benefits (slopes of the benefit functions) associated with each respective user are 6 – 2X, 7 – 3Y, and 8 – Z. When these slopes equal each other, the corresponding allocations will maximize the total benefits. This can be shown by just constructing a Lagrangian model.

The Lagrangian equation can be written as

$${\text{L = 6}}X - X^{2} + 7Y - 1.5Y^{2} + 8Z - 0.5Z^{2} - \lambda (X + Y + Z - 6).$$

Differentiating with respect to each unknown (X, Y, Z, λ) and setting the result to 0

$$\begin{gathered} \partial {\text{L/}}\partial X = 0 = 6 - 2X - \lambda \hfill \\ \partial {\text{L/}}\partial Y = 0 = 7 - 3Y - \lambda \hfill \\ \partial {\text{L/}}\partial Z = 0 = 8 - Z - \lambda \hfill \\ \partial {\text{L/}}\partial \lambda = 0 = X + Y + Z - 6 \hfill \\ \end{gathered}$$

The first three partial differential equations show that the slopes of each benefit function, the marginal benefits, at the optimal solution, are all the same, and equal to λ. Using this information in the last equation, X = 1, Y = 1, Z = 4, and thus the shadow price, dF/d6 = λ = 4 and the total benefits are 34.5. If the available resources were 6.1, the total benefits would be 34.9.

Exercises

• 1. Benefit Cost analysis

  Assume a benefit function B = 60*x^0.8 and a cost function C = 4 + 7*x^1.5. The maximum difference between B and C, the maximum net benefits, occurs at x = 8.7686.

  • (a) Would an increase in the fixed cost of 4 affect the value of x?

  • (b) Use a Lagrangian model to find the value of the shadow price, or Lagrangian multiplier, if x cannot exceed 5. What does the multiplier signify?

• 2. Allocating resources

  • (a) Consider the problem of allocating resources to three users. The allocations are X, Y, and Z. User 1’s total revenue is 6X/(1+X). User 2’s total revenue is 7Y/(1 + Y). User 3’s total revenue is 8Z/(1 + Z). Assume 10 resources are available.

    Show how to find the allocations that maximize the total revenue from all three users, and the associated shadow price of the resource constraint, using Lagrange multipliers. Compare that solution with one obtained from solving the model itself, say using Solver in Excel.

  • (b) There are two users of resources, A and B, whose income depends on the resources they are allocated. Let those allocations be A and B, respectively. The income to user A equals 10 A − 0.5 A². The income to user B is 5B – 0.25 B². You wish to know what allocations result in the maximum total income. You only have 14 resources to allocate and are curious what marginal increase in total income could result if you had more resources.