Shadow price of information in discrete time stochastic optimization

The shadow price of information has played a central role in stochastic optimization ever since its introduction by Rockafellar and Wets in the mid-seventies. This article studies the concept in an extended formulation of the problem and gives relaxed sufficient conditions for its existence. We allow for general adapted decision strategies, which enables one to establish the existence of solutions and the absence of a duality gap e.g. in various problems of financial mathematics where the usual boundedness assumptions fail. As applications, we calculate conjugates and subdifferentials of integral functionals and conditional expectations of normal integrands. We also give a dual form of the general dynamic programming recursion that characterizes shadow prices of information.


Introduction
Let (Ω, F , P ) be a probability space with a filtration (F t ) T t=0 and consider the multistage stochastic optimization problem minimize Eh(x) over x ∈ N , (SP) where N = {(x t ) T t=0 | x t ∈ L 0 (Ω, F t , P ; R nt )} denotes the space of decision strategies adapted to the filtration, h is a convex normal integrand on R n × Ω and Eh denotes the associated integral functional on L 0 (Ω, F , P ; R n ).Here and in what follows, n = n 0 + • • • + n T and L 0 (Ω, F , P ; R n ) denotes the linear space of equivalence classes of R n -valued F -measurable functions.As usual, two functions are equivalent if they are equal P -almost surely.Throughout, we define the expectation of a measurable function as +∞ unless its positive part is integrable.
Problems of the form (SP) have been extensively studied since their introduction in the mid 70's; see [16,17,19].Despite its simple appearance, problem (SP) is a very general format of stochastic optimization.Indeed, various pointwise (almost sure) constraints can be incorporated in the objective by assigning f the value +∞ when the constraints are violated.Several examples can be found in the above references.Applications to financial mathematics are given in [8,10,9].Our formulation of problem (SP) extends it's original formulations by allowing for general filtrations (F t ) T t=0 as well as general adapted strategies instead of bounded ones.This somewhat technical extension turns out to be quite convenient e.g. in financial applications.
We will use the short hand notation L ∞ := L ∞ (Ω, F , P ; R n ) and define the function φ : L ∞ → R by We assume throughout that φ(0) is finite and that Eh is proper on L ∞ .Clearly φ(0) is the optimum value of (SP) while in general, φ(z) gives the optimum value that can be achieved in combination with an essentially bounded nonadapted strategy z.Note also that φ(z) = φ(0) for all z ∈ L ∞ ∩ N .
The space L ∞ is in separating duality with L 1 := L 1 (Ω, F , P ; R n ) under the bilinear form z, v := E(z • v).
A v ∈ L 1 is said to be a shadow price of information for problem (SP) if it is a subgradient of φ at the origin, i.e., if The following result, the proof of which is given in the appendix, shows that the shadow price of information has the same fundamental properties here as in Rockafellar and Wets [19] where the primal solutions were restricted to be essentially bounded.Here and in what follows, φ * denotes the conjugate of φ defined for each v ∈ L 1 as The annihilator of N ∞ will be denoted by Theorem 1.We have φ * = Eh * + δ N ⊥ .In particular, v ∈ L 1 is a shadow price of information if and only if it solves the dual problem and the optimum value equals −φ(0).In this case, an x ∈ N is optimal if and only if Eh(x) < 0 and it minimizes the function The notion of a shadow price of information first appeared in a general single period model in Rockafellar [15,Example 6 in Section 10] and Rockafellar and Wets [18,Section 4].Extension to finite discrete time was given in [19].Continuous-time extensions have been studied in Wets [24], Back and Pliska [1], Davis [4] and Davis and Burstein [5] under various structural assumptions.The shadow price of information has been found useful in formulating dual problems and deriving optimality condition in general parametric stochastic optimization problems; see e.g.[20,1,2].The shadow price of information is useful also in subdifferential calculus involving conditional expectations; see [21] and Section 3.2 below.As a further application, we give a dual formulation of the general dynamic programming recursion from [19] and [6]; see Section 3.3.
The main result of this paper gives new generalized sufficient conditions for the existence of a shadow price of information for the discrete time problem (SP).Its proof is obtained by extending the original argument of [19] and by relaxing some of the technical assumptions made there.As already noted, we do not require the decision strategies to be essentially bounded.This allows one to establish the existence of solutions and the absence of a duality gap e.g. in various problems in financial mathematics; see [10,11].We also relax the assumptions made in [19] on the normal integrand h.
We will denote the adapted projection of an x ∈ L ∞ by a x, that is, ( a x) t = E t x t , where E t denotes the conditional expectation with respect to F t .We will also use the notation x t := (x 0 , . . ., x t ).Assumption 1.For every z ∈ dom Eh ∩ L ∞ and every t = 0, . . ., T , there exists ẑ ∈ dom Eh ∩ L ∞ such that E t z t = ẑt .
It was assumed in [19] (conditions C and D, respectively) that the sets dom h(•, ω) are closed, uniformly bounded, and "nonanticipative" and that there exists a µ ∈ L 1 such that |h(x, ω)| ≤ µ(ω) for all x ∈ dom h(•, ω).The nonanticipativity means the projection mappings D t (ω) := {x t | x ∈ dom h(•, ω)} are F t -measurable for all t.These conditions imply Assumption 1. Indeed, if z ∈ dom Eh ∩ L ∞ , then z t ∈ D t (ω) almost surely and, by Jensen's inequality, E t z t ∈ dom h almost surely as well.By the measurable selection theorem (see [22,Corollary 14.6]), there exists a ẑ ∈ L 0 such that ẑ ∈ dom h and ẑt = E t z t almost surely.The uniform boundedness of dom h implies that ẑ ∈ L ∞ while the upper bound µ gives Eh(ẑ) < ∞.
We will also use the following.
Assumption 2. There exists ρ ∈ R such that, for every z ∈ aff dom Eh ∩ L ∞ , there exists Assumption 2 holds, in particular, if a z ∈ aff dom Eh for all z ∈ aff dom Eh∩ L ∞ .In the single-step case where T = 0, this latter condition coincides with Assumption 1. Assumption 2 is also implied by the strict feasibility assumption made in [19,Theorem 2].Indeed, strict feasibility implies that dom Eh contains an open ball so that aff dom Eh = L ∞ .
In order to clarify the structure and the logic of its proof, we have split our main result in two statements of independent interest, Theorems 4 and 5 below.Combining them gives the following extension of [19,Theorem 2].
Theorem 2. Let Assumption 1 and 2 hold, and assume that Eh is strongly continuous at a point of N ∞ relative to aff dom Eh ∩ L ∞ .Then a shadow price of information exists.
A sufficient condition for the relative continuity will be given in Theorem 6 below.It is obtained by extending the argument in the proof of [14,Theorem 2].

Existence of a shadow price of information
Our main results are derived by analyzing the auxiliary value function φ : Here decision strategies are restricted to be essentially bounded like in [19].Clearly φ ≥ φ.Our strategy is to establish the existence of a subgradient of φ at the origin much like in [19].By the following simple lemma, this will then serve as a shadow price of information for the general problem (SP).Following [13], we denote the biconjugate of a function f by cl f := f * * .The general idea in [19] was first to prove the existence of a subgradient for φ with respect to the pairing of L ∞ with its Banach dual (L ∞ ) * .This was then modified to get a subgradient with respect to the pairing of In order to control the singular component v s , we have introduced Assumption 1.
Below, the strong topology will refer to the norm topology of L ∞ .
Theorem 4. Let Assumption 1 hold.If φ is proper and strongly closed at the origin, then φ is closed at the origin and φ(0) = (cl φ)(0).If φ is strongly subdifferentiable at the origin, then ∂φ(0 Proof.By Lemma 3, the first claim holds as soon as φ(0) = cl φ(0), while the second holds if ∂ φ(0) = ∅.Strong closedness of φ at the origin means that for every ǫ > 0 there is a Similarly, φ is strongly subdifferentiable at the origin iff v ⊥ N ∞ and (1) holds with ǫ = 0. We will prove the existence of a v ⊥ N ∞ which has v s = 0 and satisfies (1) with ǫ multiplied with 2 T +1 .Similarly to the above, this means that φ is closed (if (1) holds with all ǫ > 0) or subdifferentiable (if ǫ = 0) at the origin with respect to the weak topology.The existence will be proved recursively by showing that if v ⊥ N ∞ satisfies (1) and v s s = 0 for s > t (this holds for t = T as noted above), then there exists a ṽ ⊥ N ∞ which satisfies (1) with ǫ multiplied by 2 and ṽs t = 0 for s ≥ t.Thus, assume that v s s = 0 for s > t and let ǭ > 0 and x ∈ N ∞ be such that φ(0) ≥ Eh(x) − ǫ.Combined with (1) and noting that x, v = 0, we get Let z ∈ dom Eh ∩ L ∞ and let ẑ be as in Assumption 1.By Theorem 14 in the appendix, Since ẑt = E t z t and v s s = 0 for s > t by assumption, (3) means that Each term in the sum can be written as It is easily checked that we still have ṽ ∈ N ⊥ but now ṽs s = 0 for every s ≥ t as desired.Since ǭ > 0 was arbitrary and x, ṽ = 0, we see that ṽ satisfies (1) with ǫ multiplied by 2. This completes the proof since z ∈ dom Eh ∩ L ∞ was arbitrary.
The general idea of the above proof is from [19,Theorem 2] where the imposed assumptions guarantee the strong continuity of φ at the origin, which in turn guarantees subdifferentiability.The following two results give more general conditions under which the subdifferentiability holds.
Theorem 5. Let Assumption 2 hold.If Eh is strongly continuous at a point of N ∞ relative to aff dom Eh ∩ L ∞ , then φ is strongly subdifferentiable at the origin.
Proof.We may assume without loss of generality that there exist M, ǫ > 0 such that Eh(z) ≤ M for all z ∈ aff dom Eh with z ≤ ǫ.It is straightforward to check that dom φ = N ∞ + dom Eh and aff dom φ = N ∞ + aff dom Eh.Assumption 2 implies that if z ∈ aff dom φ, then z − x z ∈ aff dom Eh for some x z ∈ N ∞ with z−x z ≤ ρ a z−z .Indeed, each z ∈ aff dom φ can be expressed as z = x + w, where x ∈ N ∞ and w ∈ aff dom Eh, while Assumption 2 gives the existence of a xz 0) is finite by assumption, this implies that φ is strongly continuous and thus subdifferentiable on aff dom φ; see [15,Theorem 11].By the Hahn-Banach theorem, relative subgradients on aff dom φ can be extended to subgradients on L ∞ .
If Eh is a closed proper and convex with aff dom Eh closed, then Eh is continuous on rint s dom Eh, the relative strong interior of dom Eh (recall that the relative interior of a set is defined as its interior with respect to its affine hull).Indeed, aff dom Eh is a Banach space whenever it is closed, and then Eh is strongly continuous relative to rint s dom Eh; see e.g.[15,Corollary 8B].
The following result gives sufficient conditions for aff dom Eh to be strongly closed and rint s dom Eh to be nonempty.Its proof, contained in the appendix, is obtained by modifying the proof of [14, Theorem 2] which required that aff dom h = R n almost surely.Recall that the set-valued mappings ω → dom h and ω → aff dom h are measurable; see [22, Proposition 14.8 and Exercise 14.12].Theorem 6. Assume that the set is nonempty and contained in dom Eh.Then Eh : L ∞ → R is closed proper and convex, aff dom Eh is closed and rint s dom Eh = D.In particular, Eh is strongly continuous throughout D relative to aff dom Eh ∩ L ∞ .Remark 1.Under the assumptions Theorem 6, Eh is subdifferentiable throughout D. Indeed, the construction of y in the proof shows that y ∈ ∂Eh(x), since y ∈ ∂h(x) almost surely.
Example 1.The extension of the integrability condition of [14,Theorem 2] in Theorem 6 is needed, for example, in problems of the form minimize Eh 0 (x) over x ∈ N ∞ subject to Ax = b P -a.s., where h 0 is a convex normal integrand such that h 0 (x, •) ∈ L 1 for every x ∈ R n , A is a measurable matrix and b is a measurable vector of appropriate dimensions such that the problem is feasible.Indeed, this fits the general format of (SP) with

Calculating conjugates and subgradients
This section applies the results of the previous sections to calculate subdifferentials and conjugates of certain integral functionals and conditional expectations of normal integrands.

Integral functionals on N ∞
Let f be a normal integrand and consider the associated integral functional Ef with respect to the pairing N ∞ , N 1 .We assume throughout this section that dom The following theorem gives sufficient conditions for this to hold as an equality.
We will use the convention that the subdifferential of a function at a point is nonempty unless the function is finite at the point.
Combining the previous theorem with the results of Section 2, we get global conditions when the subdifferential of Ef coincides with the optional projection of the subdifferential of Ef with respect to the pairing L ∞ , L 1 .Corollary 8. Let f satisfy Assumptions 1 and 2. If Ef is strongly continuous at a point of N ∞ relative to aff dom f ∩ L ∞ , then where the infimum is attained, and ∂Ef (x) = a L 1 (∂f (x)).
Without the assumptions of Corollary 8, inclusion (4) may be strict.A simple example is given on page 176 of [21].
Remark 2. By Theorem 6, the continuity assumption in Corollary 8 holds, in particular, if is nonempty and contained in dom Ef .

Conditional expectation of a normal integrand
Results of the previous section allow for a simple proof of the interchange rule for subdifferentiation and conditional expectation of a normal integrand.Commutation of the two operations has been extensively studied ever since the introduction of the notion of a conditional expectation of a normal integrand in Bismut [3]; see Rockafellar and Wets [21], Truffert [23] and the references there in.The results of the previous section allow us to relax some of the continuity assumption made in earlier works.
Given a sub-sigma-algebra The G-conditional expectation of an F -measurable set-valued mapping S : The conditional expectation is well-defined and unique as soon as S admits at least one integrable selection; see Hiai and Umegaki [7,Theorem 5.1].
The general form of "Jensen's inequality" in the following lemma is from [23,Corollary 2.1.2].We give a direct proof for completeness.
almost surely for all v ∈ L 1 (F ) and To prove the first claim, assume, for contradiction, that there is a v ∈ L 1 (F ) and a set A ∈ G with P (A) > 0 on which the inequality is violated.Passing to a subset of A if necessary, we may assume that This cannot happen since, by Fenchel's inequality where the equality follows by applying the interchange rule in L ∞ (A, G, P ; R n ).
Given v ∈ L 1 (F , ∂f (x)), we have so by the first part, almost surely on A ν .This finishes the proof since ν was arbitrary.
Remark 3. If in Lemma 9, f is normal G-integrand, then the inequality can be written in the more familiar form The following gives conditions for the equalities in Lemma 9 to hold.
is subdifferentiable at the origin, then there is a If x ∈ dom Ef ∩ L 0 (G) and the above holds for every Proof.Applying Theorem 7 with T = 0 and On the other hand, By the first part, there is a This far reaching generalization of the classical dynamic programming recursion for control systems was introduced in [19] and [6].The following result from [10] relaxes the compactness assumptions made in [19] and [6].In the context of financial mathematics, this allows for various extensions of certain fundamental results in financial mathematics; see [10] for details.
Theorem 12 ([10]).Assume that h ≥ m for an m ∈ L 1 and that is a linear space.The functions h t are then well-defined normal integrands and we have for every x ∈ N that Eh t (x t ) ≥ φ(0) t = 0, . . ., T.
Optimal solutions x ∈ N exist and they are characterized by the condition which is equivalent to having equalities in (7).
Consider now the dual problem minimize Eh * (v) over v ∈ N ⊥ from Theorem 1.We know that the optimum dual value is at least −φ(0) and that if the values are equal, the shadow prices of information are exactly the dual solutions.Note also that when the functions h t and ht in the dynamic programming equations are well-defined, their conjugates solve the dual dynamic programming equations gT = h * , Much like Theorem 12 characterizes optimal primal solutions in terms of the dynamic programming equations ( 6), the following result characterizes optimal dual solutions in terms of the dual recursion (8).
Theorem 13.Assume that the dual problem is proper and that there is a feasible x ∈ N ∞ for the primal problem.Then the dual dynamic programming equations are well-defined and we have for every v ∈ N ⊥ that Eg t (E t v t ) ≥ −φ(0) t = 0, . . ., T.
In the absence of a duality gap, optimal dual solutions are characterized by having equalities in (9) while x ∈ N and v ∈ N ⊥ are primal and dual optimal, respectively, if and only if Eg(x) < ∞, Eg * (v) < ∞ and which is equivalent to having E t v t ∈ ∂g t (x t ) P -a.s.t = 0, . . ., T.
Proof.Let v ∈ N ⊥ be feasible for the dual problem.We first show inductively that E t+1 vt ∈ dom Eg t and xt ∈ dom Eg * t which implies, in particular, that each g t = Ft gt is well-defined.For t = T , this is trivial.Assume that the claim holds for some t ≤ T .Then, for every v ∈ N ⊥ , we have where the inequality follows from the induction hypotheses xt ∈ dom Eg * t and Lemma 9.
. Thus, for every x ∈ N ∩ dom Eg, we have Thus xt−1 ∈ dom Eg * t−1 which finishes the induction proof.Let x ∈ dom Eg ∩ N , and v ∈ dom Eg * ∩ N ⊥ .Combining (10) and (11) with the fact that g * 0 (x 0 ) ≥ −g 0 (0) gives for all t.In particular, (9) holds.In the absence of duality gap, ( 12) also imply that optimal dual solutions are characterized by having inequalities in (9).Likewise, we get from (12)

Appendix
This appendix contains the proofs of Theorems 1 and 6 as well as Theorem 14 below which was used in the proof of Theorem 4. Both Theorem 14 and 6 are simple refinements of well-known results on convex integral functionals, both originally due to Terry Rockafellar.
Theorem 14.Let h be a convex normal integrand and z then Proof.Let z ∈ dom Eh ∩ L ∞ and define z ν := ½ A ν z + ½ Ω\A ν z where A ν are the sets in the characterization of the singular component v s .We have h(z ν ) → h(z) almost surely and z ν → z both weakly and almost surely.Thus, since h(z ν ) ≤ max{h(z), h(z)}, Fatou's lemma and ( 13) give, where the equality holds since z ν − z = ½ Ω\A ν (z − z), so that We have that h(z ν ) → h(z) almost surely and z ν → z both weakly and almost surely.Since h(z ν ) ≤ max{h(z), h(z)}, Fatou's lemma and (13) give, where the equality holds since z ν − z = ½ A ν (z − z) so that which completes the proof.
Proof of Theorem 6. Translating, if necessary, we may assume 0 ∈ D so that L ∞ (aff dom h) ⊆ ∪ λ>0 λD ⊆ aff D. By assumption, D ⊆ dom Eh ∩ L ∞ ⊆ L ∞ (dom h) ⊆ L ∞ (aff dom h).Thus, aff D = aff(dom Eh ∩ L ∞ ) = aff L ∞ (dom h) = L ∞ (aff dom h) which is a closed set.The above also implies rint s D ⊆ rint s dom Eh ⊆ rint s L ∞ (dom h).Clearly rint s L ∞ (dom h) ⊆ D while rint s D = D.It remains to prove that Eh is closed and proper.
Let r > 0 be such that B(0, r) ∩ aff dom h ⊆ rint s dom h almost surely and let π(ω) be the projection from R d to aff dom h(•, ω).There exist x i ∈ R d , i = 0, . . ., d and r > 0 such that |x i | < r and B(0, r) belongs to the interior of the convex hull of {x i | i = 0, . . ., d}.By [22, Exercise 14.17], πx is measurable for every measurable x, so each πx i belongs to D and thus, α := max
then the expectation exists and is unique in the sense that if f is another function with the above property, then f (•, ω) = (E G f )(•,ω) almost surely; see e.g.[23, Theorem 2.1.2].
E[x t• (E t v t )] = 0 whenever the left side is integrable.Thus Eg * t (x t ) + Eg t (E t v t ) = 0 is equivalent to having g * that x and v are primal and dual optimal, respectively, if and only if Eg * t (x t ) + Eg t (E t v t ) = 0 t = 0, . . ., T. By Fenchel's inequality, g * t (x t ) + g t (E t v t ) ≥ x t • (E t v t ), so, by [11, Lemma 1], t (x t ) + g t (E t v t ) = x t • (E t v t ) almost surely, which means that E t v t ∈ ∂g t (x t ) almost surely.