Optimal dynamic multi-keyword bidding policy of an advertiser in search-based advertising

Abstract

Sponsored search advertisement allows advertisers to target their messages to appropriate customer segments at low costs. While search engines are interested in auction mechanisms that boost their revenues, advertisers seek optimal bidding strategies to increase their net sale revenues for multiple keywords under strict daily budget constraints in an environment where keyword query arrivals, competitor bid amounts, and user purchases are random. We focus on the advertiser’s question and formulate her optimal intraday dynamic multi-keyword bidding problem as a continuous-time stochastic optimization problem. We solve the problem, characterize an optimal policy, and bring a numerical algorithm for implementation. We also illustrate our optimal bidding policy and its benefits over heuristic solutions on numerical examples.

Appendix: Supplementary proofs

Proof of the equality in (7)

For a given admissible bidding policy, since $\sum _{n\ge 1} b_n R_n Z_n 1_{\{\tau _n \le T\}}$ is bounded by B, we have

$$\begin{aligned} \begin{aligned} \mathbb {E}\Big [ \sum _{n\ge 1} (W_n-b_n R_n ) Z_n 1_{\{\tau _n \le T\}} \Big ]&= \mathbb {E}\Big [ \sum _{n\ge 1} W_n Z_n 1_{\{\tau _n \le T\}} \Big ] - \mathbb {E}\Big [ \sum _{n\ge 1} b_n R_n Z_n 1_{\{\tau _n \le T\}} \Big ] \\&= \sum _{n\ge 1} \mathbb {E}[ W_n Z_n 1_{\{\tau _n \le T\}} ] - \sum _{n\ge 1} \mathbb {E}[ b_n R_n Z_n 1_{\{\tau _n \le T\}} ], \end{aligned} \end{aligned}$$

(36)

where the second line follows by the monotone convergence theorem applied to each expected random sum separately. Note that

$$\begin{aligned} \mathbb {E}[ W_n Z_n 1_{\{\tau _n \le T\}} ] = \mathbb {E}[ 1_{\{\tau _n \le T\}} \mathbb {E}[ W_n Z_n \mid {\mathcal {F}}_{\tau _n} ] ] = \mathbb {E}[ 1_{\{\tau _n \le T\}} \mu _{\kappa _n} G_{\kappa _n} (b_n) ], \end{aligned}$$

(37)

due to the conditional independence of $W_n$, the conditional expectations in (3), and the conditional distributions in (1). Similarly, we write

$$\begin{aligned} \mathbb {E}[ b_n R_n Z_n 1_{\{\tau _n \le T\}} ] = \mathbb {E}[ 1_{\{\tau _n \le T\}} b_n \mathbb {E}[ R_n Z_n \mid {\mathcal {F}}_{\tau _n} ] ] = \mathbb {E}[ 1_{\{\tau _n \le T\}} b_n \rho _{\kappa _n} G_{\kappa _n} (b_n) ], \end{aligned}$$

(38)

thanks to (1) and (2). Using now (37-38) in (36) yields

$$\begin{aligned} \begin{aligned}&\mathbb {E}\Big [ \sum _{n\ge 1} (W_n-b_n R_n ) Z_n 1_{\{\tau _n \le T\}} \Big ] \\&\quad = \sum _{n\ge 1} \mathbb {E}[ 1_{\{\tau _n \le T\}} \mu _{\kappa _n} G_{\kappa _n} (b_n) ] - \sum _{n\ge 1} \mathbb {E}[ 1_{\{\tau _n \le T\}} b_n \rho _{\kappa _n} G_{\kappa _n} (b_n) ] \\&\quad = \mathbb {E}\Big [ \sum _{n\ge 1} 1_{\{\tau _n \le T\}} \mu _{\kappa _n} G_{\kappa _n} (b_n) \Big ] - \mathbb {E}\Big [ \sum _{n\ge 1} 1_{\{\tau _n \le T\}} b_n \rho _{\kappa _n} G_{\kappa _n} (b_n) \Big ] \\&\quad = \mathbb {E}\Big [ \sum _{n\ge 1} 1_{\{\tau _n \le T\}} ( \mu _{\kappa _n} - b_n \rho _{\kappa _n}) G_{\kappa _n} (b_n) \Big ] \end{aligned} \end{aligned}$$

(39)

establishing (7). In (39), the second equality is by the monotone convergence theorem (applied to each expectation), and the last equality follows simply by the boundedness of $\sum _{n\ge 1} 1_{\{\tau _n \le T\}} b_n \rho _{\kappa _n} G_{\kappa _n} (b_n)$ (by B). $\square$

Proof of Proposition 1

Monotonicity of $\mathcal {D}[f]$ in f is obvious, and the non-negativity of $D[f_1]$ follows after taking $b_1= 0$ in (10). To prove the upper bound $\mathcal {D}[f_2] (\cdot , T) \le \gamma \lambda T$, we observe that

$$\begin{aligned} \mathcal {D}[f](B,&T) \le \sup _{b_1 \in \mathcal {F}_{\tau _1}} \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [ (W_1-b_1 R_1)Z_1 + \gamma \lambda (T-\tau _1) \Big ] \\&= \sup _{b_1 \in \mathcal {F}_{\tau _1}} \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [ (\mu _{\kappa _1}-b_1 \rho _{\kappa _1} ) G_{\kappa _1}(b_1) + \gamma \lambda (T-\tau _1) \Big ] \\&\le \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [ \gamma + \gamma \lambda (T-\tau _1) \Big ] = \gamma \int _0^T \lambda e^{-\lambda t } [1+ \lambda (T-t) ] dt = \gamma \lambda T, \end{aligned}$$

in which the second line follows by conditioning on $\mathcal {F}_{\tau _1}$, the inequality in the third line is by related arguments on the mapping $b \mapsto (\mu _k - b \rho _k)G_k(b)$ and the definition of $\gamma$ in Remark 3, and the very last equality is simply by integration (by parts). $\square$

Proof of Proposition 2

For notational convenience, let us define for $1 \le k \le K$

$$\begin{aligned}&L_k[f] (b,B,T) := f(B,T) + M_k [f](b,B,T) \quad \text {and}\quad \\&\quad L^*_k[f] (B,T) := f(B,T) + M^*_k [f](B,T), \end{aligned}$$

with $M_k$ and $M^*_k$ defined in (12-13). It is easy to verify that the mappings $B \mapsto L^*_k[f] (B,\cdot )$ and $T \mapsto L^*_k[f] (\cdot ,T)$ are non-decreasing under the given assumptions (on the monotonicity of f in its arguments). Since

$$\begin{aligned} \mathcal {D}[f] (B,T) = \int _0^T \sum _{k=1}^K \lambda _k e^{-\lambda u } L^*_k[f] (B,T-u) du, \end{aligned}$$

it follows that $B \mapsto \mathcal {D}[f] (B,\cdot )$ and $T \mapsto \mathcal {D}[f] (\cdot ,T)$ are again non-decreasing. This proves the non-negativity of the differences in (16) and (17).

The prove the upper bound in (16), we note that, for $T_1 < T_2$, we have

$$\begin{aligned} \mathcal {D}[f] (B,T_2) - \mathcal {D}[f] (B,T_1)= & {} ( e^{- \lambda T_2} - e^{- \lambda T_1}) \int _0^{T_1} \sum _{k = 1}^K \lambda _k e^{ \lambda u } L^*_k[f](B,u) du \nonumber \\{} & {} + e^{- \lambda T_2} \int _{T_1}^{T_2} \sum _{k = 1}^K \lambda _k e^{ \lambda u } L^*_k[f](B,u) du \nonumber \\\le & {} \int _{T_1}^{T_2} \sum _{k = 1}^K \lambda _k L^*_k[f](B,u) du. \end{aligned}$$

(40)

It is easy to verify that $L^*_k[f] (B,T) \le \mu _k + \Vert f\Vert \le {\bar{\mu }} + \Vert f\Vert$. Hence, we obtain $\mathcal {D}[f] (B,T_2) - \mathcal {D}[f] (B,T_1) \le \lambda ({\bar{\mu }} + \Vert f\Vert ) (T_2 - T_1)$.

To establish the second claim, let $B_1 < B_2$ be two budget levels, and for fixed T, let $b_2 = b^*_k [f] (B_2,T)$ denote the maximum bid in (14) for a fixed k with the budget level $B_2$. Note that $b_1 = \frac{B_1}{B_2} b_2 < b_2$ is a feasible bid for the budget $B_1$, and we have $b_2 - b_1 = \frac{(B_2-B_1)}{B_2} b_2 \le B_2 - B_1$. Then, we write

$$\begin{aligned} L^*_k[f](B_2,T)= & {} (1-G_k(b_2)) f(B_2 , T) + G_k(b_2) \Big [ \mu _k - \rho _k b_2 \nonumber \\{} & {} + \int _0^1 f(B_2- r b_2, T) H_k(dr)\Big ] \nonumber \\\le & {} (1-G_k(b_2)) ( f(B_1 , T) + \alpha _f (B_2-B_1) ) \nonumber \\{} & {} + G_k(b_2) \Big [ \mu _k - \rho _k b_1 + \int _0^1 f(B_1- r b_1, T) H_k(dr) + \alpha _f (B_2-B_1) \Big ] \nonumber \\= & {} \alpha _f (B_2-B_1) + (1-G_k(b_2)) f(B_1 , T) \nonumber \\{} & {} + G_k(b_2) \Big [ \mu _k - \rho _k b_1 + \int _0^1 f(B_1- r b_1, T) H_k(dr) \Big ] \nonumber \\\le & {} \alpha _f (B_2-B_1) + L_k[f](b_1,B_1,T) + \alpha _G(b_2 - b_1) ({\bar{\mu }} + \Vert f\Vert ) \nonumber \\\le & {} \alpha _f (B_2-B_1) + L^*_k[f](B_1,T) + \alpha _G(B_2 - B_1) ({\bar{\mu }} + \Vert f\Vert ) \nonumber \\= & {} L^*_k[f](B_1,T) + (B_2-B_1) \big [ \alpha _f + \alpha _G ({\bar{\mu }} + \Vert f\Vert ) \big ]. \end{aligned}$$

(41)

Next, we consider the difference

$$\begin{aligned} \mathcal {D}[f](B_2,T) - \mathcal {D}[f](B_1,T)&= e^{- \lambda T } \int _{0}^{T } \sum _{k = 1}^K \lambda _k e^{ \lambda u } \big [ L^*_k[f](B_2,u)\\&\quad -L^*_k[f](B_1,u) \big ] du, \end{aligned}$$

and using (41), we obtain $\mathcal {D}[f](B_2,T) - \mathcal {D}[f](B_1,T) \le$

$$\begin{aligned}&e^{- \lambda T } \int _{0}^{T } \sum _{k = 1}^K \lambda _k e^{ \lambda u } (B_2 - B_1) \big [ \alpha _f + \alpha _G ({\bar{\mu }} + \Vert f\Vert ) \big ] du \\&\quad \le (B_2 - B_1) (1- e^{ - \lambda T_{\max } } ) \big [ \alpha _f + \alpha _G ({\bar{\mu }} + \Vert f\Vert ) \big ] \end{aligned}$$

giving us the upper bound in (17). $\square$

Proof of Proposition 3

Because ${\underline{U}}_0 (\cdot ,\cdot ) = 0 \le {\overline{U}}_0(B,T) = \gamma \lambda T$, Proposition 1 implies that $0 \le {\underline{U}}_1 \le {\overline{U}}_1 \le \gamma \lambda T$. Let us now assume that ${\underline{U}}_{n-1} \le {\underline{U}}_{n} \le {\overline{U}}_{n} \le {\overline{U}}_{n-1}$ for some $n\ge 1$. Then, again by Proposition 1, we have $\mathcal {D}[{\underline{U}}_{n-1}] \le \mathcal {D}[{\underline{U}}_{n}] \le \mathcal {D}[ {\overline{U}}_{n}] \le \mathcal {D}[{\overline{U}}_{n-1}]$, and together with the induction hypothesis, this yields ${\underline{U}}_{n-1} \le {\underline{U}}_{n} \le {\underline{U}}_{n+1} \le {\overline{U}}_{n+1} \le {\overline{U}}_{n} \le {\overline{U}}_{n-1}$. Hence, by induction we conclude that $( {\underline{U}}_n (\cdot ,\cdot ) )_{n \ge 0}$ and $( {\overline{U}}_n (\cdot ,\cdot ) )_{n \ge 0}$ are non-decreasing and non-increasing, respectively, and the collection $( {\underline{U}}_n (\cdot ,\cdot ) )_{n \ge 0}$ is bounded from above by $( {\overline{U}}_n (\cdot ,\cdot ) )_{n \ge 0}$.

Let ${\underline{U}}_{\infty }$ denote the pointwise limit of the monotone sequence $({\underline{U}}_{n})_{n \ge 0}$. Because the operator $\mathcal {D}$ is a contraction mapping, $\Vert {\underline{U}}_{\ell +1} - {\underline{U}}_{\ell } \Vert = \Vert \mathcal {D}[{\underline{U}}_{\ell }] - \mathcal {D}[{\underline{U}}_{\ell -1}] \Vert \le (1-e^{-\lambda T_{\text {max}}}) \Vert {\underline{U}}_{\ell } - {\underline{U}}_{\ell -1} \Vert \le \ldots \le (1-e^{-\lambda T_{\text {max}}})^\ell \; \Vert {\underline{U}}_1 \Vert ,$ using which we obtain

$$\begin{aligned} \Vert {\underline{U}}_{m+n}\!\!-\!\!{\underline{U}}_{n} \Vert \!\!\le \!\!\sum _{\ell =n}^{m+n-1} \Vert {\underline{U}}_{\ell +1} \!\!-\!\! {\underline{U}}_{\ell } \Vert \le \Vert {\underline{U}}_1 \Vert \!\!\sum _{\ell = n}^{m+n-1} (1-e^{-\lambda T_{\text {max}}})^\ell \text {for every }m,n \!\!\ge \!\! 0, \end{aligned}$$

and this gives $0 \le {\underline{U}}_{\infty } (B,T) - {\underline{U}}_{n} (B,T) = \lim _{m \rightarrow \infty } {\underline{U}}_{m+n} (B,T) - {\underline{U}}_{n} (B,T) \le \Vert {\underline{U}}_1 \Vert \lim _{m \rightarrow \infty } \sum _{\ell = n}^{m+n-1} (1-e^{-\lambda T_{\text {max}}})^\ell$ $\le \gamma \lambda T_{\text {max}} \sum _{\ell = n}^{\infty } (1-e^{-\lambda T_{\text {max}}})^\ell = \gamma \lambda T_{\text {max}} (1-e^{-\lambda T_{\text {max}}})^n e^{\lambda T_{\text {max}}}$. Since this is true for all $(B,T) \in \Delta$, we have $\Vert {\underline{U}}_{\infty } - {\underline{U}}_{n} \Vert \le \gamma \lambda T_{\text {max}} (1-e^{-\lambda T_{\text {max}}})^n e^{\lambda T_{\text {max}}}$, which proves that the convergence of ${\underline{U}}_{n}$ to ${\underline{U}}_{\infty }$ is uniform on $\Delta$ as $n \rightarrow \infty$. In the arguments above, if we replace ${\underline{U}}$ with ${\overline{U}}$, then we obtain the same upper bound for ${\overline{U}}_{n} (B,T) - {\overline{U}}_{\infty } (B,T) \ge 0$. This shows that the ${\overline{U}}_{n}$’s converge to ${\overline{U}}_{\infty } (B,T)$ also uniformly with the same error bound. Because ${\overline{U}}_{n}$’s and ${\underline{U}}_{n}$’s are continuous, their uniform limits ${\overline{U}}_{\infty }$ and ${\underline{U}}_{\infty }$, respectively, are also continuous on $\Delta$.

The uniform convergence of $({\underline{U}}_n)_{n \ge 0}$ and $({\overline{U}}_n)_{n \ge 0}$ also imply that as $n \rightarrow \infty$, we have $M^*_{k} [{\overline{U}}_n] (\cdot ,\cdot )$ $\rightarrow M^*_{k} [{\overline{U}}_\infty ] (\cdot ,\cdot )$ and $M^*_{k} [{\underline{U}}_n] (\cdot ,\cdot ) \rightarrow M^*_{k} [{\underline{U}}_\infty ] (\cdot ,\cdot )$ for each $1 \le k \le K$, and they are bounded from above by $\max _k M^*_{k} [{\overline{U}}_0] (B_{\text {max}},T_{\text {max}}) < \infty$. Then, by bounded convergence theorem we obtain

$$\begin{aligned} {\underline{U}}_\infty (B,T)= & {} \lim _n {\underline{U}}_{n+1} (B,T) = \lim _n \mathcal {D}[ {\underline{U}}_{n}] (B,T) = \lim _n \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [ {\underline{U}}_{n}(B, T-\tau _1) \\{} & {} + M^*_{\kappa _1} [{\underline{U}}_{n}] (B,T-\tau _1) \Big ] \\= & {} \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [ {\underline{U}}_{\infty }(B, T-\tau _1) + M^*_{\kappa _1} [{\underline{U}}_{\infty }] (B,T-\tau _1) \Big ] = \mathcal {D}[{\underline{U}}_{\infty }] (B,T), \end{aligned}$$

which shows that ${\underline{U}}_{\infty }$ is a fixed point of $\mathcal {D}$. Replicating the arguments above with ${\overline{U}}_{\infty }= \lim _n {\overline{U}}_{n}$, we observe that ${\overline{U}}_{\infty }$ is also a fixed point of the operator $\mathcal {D}$. Finally, the uniqueness of the fixed point (see Lemma 1) implies that ${\underline{U}}_{\infty } = {\overline{U}}_{\infty }$. $\square$

Proof of Lemma 2

The inequality in (26) becomes an equality for $j=1$. Assume that it holds for some $1 \le j \le n$, and let us prove it for $j+1$. Note that the right hand side of (26) can be decomposed as

$$\begin{aligned}{} & {} \mathbb {E}\sum _{i=1}^{n-j} (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} \nonumber \\{} & {} +\! \mathbb {E}\Big [1_{\{\tau _{n-j+1} \!\le \! T\}} \!\Big ( (W_{n-j\!+\!1}-b_{n-j\!+\!1} R_{n-j+1}) Z_{n-j+1} \!\!+\!\! {\underline{U}}_{j-1} (B_{\tau _{n+j+1}}\!, T-\tau _{n-j+1}) \!\Big )\!\Big ].\nonumber \\ \end{aligned}$$

(42)

Conditioning on $\mathcal {F}_{\tau _{n-j+1}}$ and using the conditional distributions of $W_{n-j+1}, R_{n-j+1}$, and $Z_{n-j+1}$, the second expectation in (42) above becomes

$$\begin{aligned} \begin{aligned}&\mathbb {E}\Big [1_{\{\tau _{n-j+1} \le T\}} \Big ( (\mu _{\kappa _{n-j+1}} - b_{n-j+1} \cdot \rho _{n-j+1} )G_{\kappa _{n-j+1}} (b_{n-j+1}) \\&\quad + {\underline{U}}_{j-1} (B_{\tau _{n-j}}, T-\tau _{n-j+1}) (1- G_{\kappa _{n-j+1}} (b_{n-j+1})) \\&\quad + G_{\kappa _{n-j+1}} (b_{n-j+1}) \int _0^1 {\underline{U}}_{j-1} (B_{\tau _{n-j}}- r b_{n-j+1}, T-\tau _{n-j+1}) h_{ \kappa _{n-j+1} } (r) dr \Big )\Big ] \\&= \mathbb {E}\Big [1_{\{\tau _{n-j+1} \le T\}} \Big ( {\underline{U}}_{j-1} (B_{\tau _{n-j}}, T-\tau _{n-j+1}) \\&\quad + M_{\kappa _{n-j+1}}[{\underline{U}}_{j-1}] (b_{n-j+1}, B_{\tau _{n-j}} , T-\tau _{n-j+1}) \Big )\Big ] \\&\!\le \! \mathbb {E}\Big [1_{\{\tau _{n-j+1} \!\le \! T\}} \!\Big ( {\underline{U}}_{j-1} (B_{\tau _{n-j}}, T-\tau _{n-j+1}) \!+\! M^*_{\kappa _{n-j+1}}[{\underline{U}}_{j-1}] (B_{\tau _{n-j}}\!, T\!-\!\tau _{n-j+1}) \!\Big ) \!\Big ] \\&= \mathbb {E}\Big [1_{\{\tau _{n-j} \le T\}} \mathbb {E}\Big [ 1_{\{\tau _{n-j+1} \le T\}} \Big ( {\underline{U}}_{j-1} (B_{\tau _{n-j}}, T-\tau _{n-j+1}) \\&\quad + M^*_{\kappa _{n-j+1}}[{\underline{U}}_{j-1}] (B_{\tau _{n-j}} , T-\tau _{n-j+1}) \Big ) \Big | \mathcal {F}_{\tau _{n-j}} \vee \sigma (B_{\tau _{n-j}}) \Big ]\Big ] \\&= \mathbb {E}\Big [ 1_{ \{ \tau _{n-j} \le T \} } \; D[{\underline{U}}_{j-1}] (B_{\tau _{n-j}} , T - \tau _{n-j}) \Big ] \end{aligned} \end{aligned}$$

(43)

where the last line is due to strong Markov property. Because $\mathcal {D}[{\underline{U}}_{j-1}] = {\underline{U}}_j$, combining (42-43) with the induction hypothesis yields

$$\begin{aligned} \mathbb {E}\Big [ \sum _{i=1}^n (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} \big ]\le & {} \mathbb {E}\big [ \sum _{i=1}^{n-j+1} (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} \nonumber \\{} & {} + 1_{ \{ \tau _{n-j+1} \le T \} } \, {\underline{U}}_{j-1} (B_{\tau _{n+j+1}}, T-\tau _{n-j+1}) \Big ]\nonumber \\\le & {} \mathbb {E}\big [ \sum _{i=1}^{n-j} (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} + 1_{ \{ \tau _{n-j} \le T \} } \nonumber \\{} & {} {\underline{U}}_{j} (B_{\tau _{n-j}}, T-\tau _{n-j}) \Big ], \end{aligned}$$

(44)

and this proves the inequality (26) for $j+1$. Hence, by induction it holds for all $1\le j \le n+1$. $\square$

Proof of Proposition 5

For $n =0$, we have $V_0 = {\underline{U}}_0 = 0$ by construction, and the summation in (28) equals zero. Hence, the claim of the proposition is obvious. Therefore, we state the proposition and prove the equalities in (28) for $n \ge 1$.

By Lemma 2, we have for every admissible policy $(b_i)_{i \ge 1}$

$$\begin{aligned}{} & {} \mathbb {E}\Big [ \sum _{i=1}^n (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} \big ]\\{} & {} \quad \le \mathbb {E}\big [ \sum _{i=1}^{n-j+1} (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} + 1_{ \{ \tau _{n-j+1} \le T \} } \, {\underline{U}}_{j-1} (B_{\tau _{n-j+1}}, T-\tau _{n-j+1}) \Big ] \end{aligned}$$

for $1 \le j \le n+1$. Evaluating this inequality with $k = n+1$ gives $\mathbb {E}\Big [ \sum _{i=1}^n (W_i-b_i R_i) Z_i 1_{\{\tau _i \le T\}} \big ] \le {\underline{U}}_{n} (B,T)$ and this implies $V_n (B,T) \le {\underline{U}}_n (B,T)$ for all $n \ge 1$.

We next establish the second equality in (28) for all $n \ge 1$. Because ${\underline{U}}_0 \equiv 0$, we have

$$\begin{aligned} {\underline{U}}_1(B,T)= & {} \mathcal {D}[{\underline{U}}_0] (B,T) = \sup _{b_1 \in \mathcal {F}_{\tau _1} } \mathbb {E}\left[ 1_{\{ \tau _1 \le T\}} \left( W_1-b_1 R_1 \right) Z_1 \right] \\= & {} \sup _{b_1 \in \mathcal {F}_{\tau _1} } \mathbb {E}\, 1_{\{ \tau _1 \le T\}} G_{\kappa _1}(b_1) \left( \mu _{\kappa _1} - b_1 \rho _{\kappa _1} \right) \\= & {} \mathbb {E}\Big [1_{\{ \tau _1 \le T\}} \left( W_1-b^*_{\kappa _1} [{\underline{U}}_{0}] (B,T- \tau _1) \cdot R_1 \right) Z_1 \Big ]\\\equiv & {} \mathbb {E}\Big [1_{\{ \tau _1 \le T\}} \left( W_1-b^{(1)}_1 R_1 \right) Z_1 \Big ], \end{aligned}$$

and this gives the second equality in (28) for $n = 1$. Assume now that the second equality in (28) holds for some $n\ge 1$. Then we have

$$\begin{aligned} \begin{aligned} {\underline{U}}_{n+1}(B,T)&= \mathcal {D}[{\underline{U}}_n] (B,T)\\&= \sup _{b_1 \in \mathcal {F}_{\tau _1}} \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [(W_1-b_1 R_1)Z_1 + {\underline{U}}_n(B_{\tau _1}, T-\tau _1) \Big ] \\&= \mathbb {E}\, 1_{\{\tau _1 \le T\}} \Big [(W_1- \underbrace{b^*_{\kappa _1}[{\underline{U}}_{n}](B,T- \tau _1)}_{b^{(n+1)}_1} \cdot R_1)Z_1 + {\underline{U}}_n(B_{\tau _1}, T-\tau _1) \Big ]. \end{aligned} \end{aligned}$$

(45)

Using the induction hypothesis and the strong Markov property, we obtain

$$\begin{aligned} \begin{aligned}&\mathbb {E}\big ( 1_{\{\tau _1 \le T\}} \, {\underline{U}}_n(B_{\tau _1}, T-\tau _1) \big ) \\&\quad = \mathbb {E}\bigg ( 1_{\{\tau _1 \le T\}} \mathbb {E}\Big [ \sum _{i =1}^{n} \left( W_{i+1} -b^*_{\kappa _{i+1}} [{\underline{U}}_{n-i}]\big (B^{(n+1)}_{T_{i}}, T- T_{i+1} \big )\right. \\&\qquad \left. \cdot R_{i+1} \right) Z_{i+1} 1_{\{T_{i+1} \le T \}} \mid \mathcal {F}_{\tau _1} \vee \sigma (B_{\tau _1}) \Big ] \bigg ) \\&\quad = \mathbb {E}\bigg [ \sum _{i =2}^{n+1} \left( W_i -b^*_{\kappa _i} [{\underline{U}}_{n+1-i}] (B^{(n+1)}_{\tau _{i-1}} ,T- \tau _i) \cdot R_i \right) Z_i \, 1_{\{\tau _i \le T \}} \bigg ]\\&\quad \equiv \mathbb {E}\bigg [ \sum _{i =2}^{n+1} \left( W_i -b^{(n+1)}_i R_i \right) Z_i \, 1_{\{\tau _i \le T \}} \bigg ] . \end{aligned} \end{aligned}$$

(46)

Substituting (46) into (45) gives

$$\begin{aligned} {\underline{U}}_{n+1}(B,T)&= \mathbb {E}\bigg [ \sum _{i =1}^{n+1} \left( W_i -b^{(n+1)}_i R_i \right) Z_i \, 1_{\{\tau _i \le T \}} \bigg ] . \end{aligned}$$

This proves the second equality in (28) for $n+1$. Hence, it holds for all $n \ge 1$ by induction.

Clearly, $V_n$ is an upper bound for the expected net revenue of any n-bid policy. Hence, combining all the arguments above, we now have, for any $n \ge 1$,

$$\begin{aligned} V_n (B,T) \le {\underline{U}}_{n}(B,T) = \mathbb {E}\bigg [ \sum _{i =1}^{n} \left( W_i -b^{(n+1)}_i R_i \right) Z_i \, 1_{\{\tau _i \le T \}} \bigg ] \le V_{n}(B,T), \end{aligned}$$

and this establishes the equalities in (28). $\square$

Proof of Proposition 6

To prove the claim, it is sufficient to establish the identity

$$\begin{aligned} V(B,T) = \mathbb {E}\bigg [ \sum _{i=1}^n \left( W_i- b^{(\infty )}_i R_i \right) Z_i 1_{\{\tau _i \le T\}} + 1_{ \{ \tau _n \le T \} } \, V\big (B^{(\infty )}_{\tau _n}, T-\tau _n\big ) \bigg ], \end{aligned}$$

(47)

inductively for all $n \ge 1$. When we let $n \rightarrow \infty$, the expectation of the second term converges to zero because V is bounded (see Remark 3) and $\mathbb {P} (\tau _n \le T) \rightarrow 0$. Also both $\mathbb {E}\big [ \sum _{i=1}^n W_i \cdot R_i Z_i 1_{\{\tau _i \le T\}} \big ]$ and $\mathbb {E}\big [ \sum _{i=1}^n b^{(\infty )}_i R_i Z_i 1_{\{\tau _i \le T\}} \big ]$ convergence by monotone convergence theorem to the expectations of the corresponding infinite sums, and $\sum _{i=1}^\infty b^{(\infty )}_i R_i Z_i 1_{\{\tau _i \le T\}} \le B$. Hence, as $n \rightarrow \infty$, the expectation of the summation in (47) converges to the right hand side in (31).

For $n= 1$, the expectation in (47) becomes $\mathcal {D}[V] (B,T)$, and the equality holds since V is a fixed point of the operator $\mathcal {D}$. Assume now that the equality holds for some $n \ge 1$. Then

$$\begin{aligned}{} & {} \mathbb {E}\bigg [ \sum _{i=1}^{n+1} \left( W_i- b^{(\infty )}_i R_i \right) Z_i 1_{\{\tau _i \le T\}} + 1_{ \{ T_{n+1} \le T \} }\; V \big ( B^{(\infty )}_{T_{n+1}}, T-T_{n+1} \big ) \bigg ] \nonumber \\{} & {} \quad =\mathbb {E}\bigg [ \left( W_1- b^{(\infty )}_1 R_1 \right) Z_1 1_{\{\tau _1 \le T\}} \nonumber \\{} & {} \qquad + 1_{\{\tau _1 \le T\}} \, \mathbb {E}\Big [ \sum _{i=2}^{n+1} \left( W_i- b^{(\infty )}_i R_i \right) Z_i 1_{\{\tau _i \le T\}} \nonumber \\{} & {} \qquad + 1_{ \{ T_{n+1} \le T \} } V\big ( B^{(\infty )}_{T_{n+1}} , T-T_{n+1} \big ) \; \Big | \; \mathcal {F}_{\tau _1} \vee \sigma ( B^{(\infty )}_{T_{1}} )\Big ] \bigg ]. \end{aligned}$$

(48)

On the event $\{\tau _1 \le T\}$, the conditional expectation above is equal to $V\big (B^{(\infty )}_{\tau _1}, T-\tau _1 \big )$ by the strong Markov property and the induction hypothesis. Hence, the right hand side of the equality in (48) becomes

$$\begin{aligned} \mathbb {E}1_{\{\tau _1 \le T\}} \bigg [ \left( W_1- b^{(\infty )}_1 R_1 \right) Z_1 + V\big (B^{(\infty )}_{\tau _1}, T-\tau _1 \big ) \bigg ] = \mathcal {D}[V](B,T) = V(B,T) , \end{aligned}$$

which proves (47) for $n+1$. Hence, (47) holds for all $n \ge 1$ by induction, and this completes the proof. $\square$

Proof of Proposition 7

The identity $f(B,0) = V(B,0)$ for $T = 0$ is obvious. Therefore, we only give the proof for $T> 0$. Note that for a given admissible policy $(b_i)_{i \ge 1}$ and the corresponding budget process $\{ B_t \}_{t \in [0,T]}$, the chain rule gives

$$\begin{aligned}{} & {} \mathbb {E}\underbrace{f(B_T , 0 )}_{0} - f(B, T) \nonumber \\{} & {} \quad = \mathbb {E}\bigg [- \int _0^T f_T (B_{t-}, T-t ) dt + \sum _{i \ge 1} \big [ f(B_{\tau _i}, T-\tau _i) - f(B_{\tau _i-}, T-\tau _i) \big ] 1_{ \{ \tau _i \le T \} } \bigg ] \nonumber \\{} & {} \quad = \mathbb {E}\bigg [- \int _0^T \bigg ( \sum _{k = 1}^K \frac{\lambda _k}{\lambda } \; M^*_{k} [f] (B_{t-}, T-t) \bigg ) \lambda dt \nonumber \\{} & {} \qquad + \sum _{i \ge 1} \left[ f(B_{\tau _i}, T-\tau _i) - f(B_{\tau _i-}, T-\tau _i) \right] 1_{ \{ \tau _i \le T \} } \bigg ]. \end{aligned}$$

(49)

Because $M^*_{k} [f] (\cdot ,\cdot )$ is continuous on $\Delta$ and the budget process is an $\mathbb {F}$-adapted càdlàg process, it follows that $\{ M^*_{k} [f] (B_{t-} ,T-t) \}_{t \in [0,T]}$ is bounded and $\mathbb {F}$-predictable. Therefore, in terms of the counting process $N = \{ N_t\}_{t \ge 0}$ with $N_t = \sum _{i \ge 1} 1_{ \{ \tau _i \le T \} }$, for $t \ge 0$, we have

$$\begin{aligned} \begin{aligned}&\mathbb {E}\bigg [\int _0^T \bigg ( \sum _{k = 1}^K \frac{\lambda _k}{\lambda } \; M^*_{k} [f] (B_{t-}, T-t) \bigg ) \lambda dt \bigg ] \\&\quad = \mathbb {E}\bigg [ \int _{(0,T]} \bigg ( \sum _{k = 1}^K \frac{\lambda _k}{\lambda } \; M^*_{k} [f] (B_{t-}, T-t) \bigg ) d N_t \bigg ] \\&\quad = \mathbb {E}\bigg [ \sum _{i \ge 1} M^*_{\kappa _i} [f] \big ( B_{\tau _i-}, T-\tau _i \big ) \cdot 1_{ \{ \tau _i \le T \} } \bigg ] \\&\quad \ge \mathbb {E}\bigg [ \sum _{i \ge 1} M_{\kappa _i} [f] \big ( b_i,B_{\tau _i-}, T-\tau _i \big ) \cdot 1_{ \{ \tau _i \le T \} } \bigg ] \\&\quad \equiv \mathbb {E}\bigg [ \sum _{i \ge 1} G_{\kappa _i}(b_i) \Big ( \mu _{\kappa _i} - b_i \rho _{\kappa _i} + \int _0^1 f\big (B_{\tau _i-}- r b_i, T-\tau _i \big ) \, H_{\kappa _i} (\textrm{d}r) \\&\qquad - f \big ( B_{\tau _i-}, T-\tau _i \big ) \Big ) 1_{ \{ \tau _i \le T \} } \bigg ]. \end{aligned} \end{aligned}$$

(50)

Using the conditional distribution of $(Z_i,R_i)$ (with the help of the dominated convergence theorem to interchange the summation and the expectation), we obtain

$$\begin{aligned}{} & {} \mathbb {E}\Big [ \sum _{i \ge 1} \left( f\big (B_{\tau _i}, T-\tau _i \big ) - f \big (B_{\tau _i-}, T-\tau _i \big ) \right) 1_{ \{ \tau _i \le T \} } \Big ] \nonumber \\{} & {} =\mathbb {E}\bigg [\sum _{i \ge 1} G_{\kappa _i}(b_i) \!\bigg ( \int _0^1 f\!\big ( B_{\tau _i-}- r b_i, T-\tau _i \!\big ) H_{\kappa _i} (\textrm{d}r) \!-\! f \!\big ( B_{\tau _i-}, T-\tau _i \!\big ) \!\bigg ) 1_{ \{ \tau _i \!\le \! T \} } \!\bigg ].\nonumber \\ \end{aligned}$$

(51)

Inserting (50-51) into (49) yields

$$\begin{aligned} f(B,T)&\ge \mathbb {E}\bigg [ \sum _{i \ge 1} G_{\kappa _i}(b_i) \big ( \mu _{\kappa _i} - b_i \rho _{\kappa _i} \big ) \cdot 1_{ \{ \tau _i \le T \} } \bigg ]\nonumber \\&= \mathbb {E}\bigg [ \sum _{i \ge 1} \big ( W_i - b_i R_i \big ) Z_i \cdot 1_{ \{ \tau _i \le T \} } \bigg ], \end{aligned}$$

(52)

and this implies that $f(B,T) \ge V(B,T)$ because $(b_i)_{i \ge 1}$ was an arbitrary admissible policy. In particular, with the optimal policy $b^{(\infty )}$ (and its budget process $B^{(\infty )}$) given in Lemma 6, the inequalities in (50) and (52) become equalities, and this proves that $f(B,T) = V(B,T)$. $\square$