Data-driven inverse optimization with imperfect information

Abstract

In data-driven inverse optimization an observer aims to learn the preferences of an agent who solves a parametric optimization problem depending on an exogenous signal. Thus, the observer seeks the agent’s objective function that best explains a historical sequence of signals and corresponding optimal actions. We focus here on situations where the observer has imperfect information, that is, where the agent’s true objective function is not contained in the search space of candidate objectives, where the agent suffers from bounded rationality or implementation errors, or where the observed signal-response pairs are corrupted by measurement noise. We formalize this inverse optimization problem as a distributionally robust program minimizing the worst-case risk that the predicted decision (i.e., the decision implied by a particular candidate objective) differs from the agent’s actual response to a random signal. We show that our framework offers rigorous out-of-sample guarantees for different loss functions used to measure prediction errors and that the emerging inverse optimization problems can be exactly reformulated as (or safely approximated by) tractable convex programs when a new suboptimality loss function is used. We show through extensive numerical tests that the proposed distributionally robust approach to inverse optimization attains often better out-of-sample performance than the state-of-the-art approaches.

Appendix A. Convex hypotheses

Consider the class $\mathcal {F}$ of hypotheses of the form $F_\theta (s,x):=\big \langle \theta , \Psi (x) \big \rangle$, where each component function of the feature map $\Psi :{\mathbb {R}}^n\rightarrow {\mathbb {R}}^d$ is convex, and where the weight vector $\theta$ ranges over a convex closed search space $\Theta \subseteq {\mathbb {R}}^d_+$. Thus, by construction, $F_\theta (s,x)$ is convex in x for every fixed $\theta \in \Theta$. In the remainder we will assume without much loss of generality that the transformation from the signal-response pair (s, x) to the signal-feature pair $(s,\Psi (x))$ is Lipschitz continuous with Lipschitz modulus 1, that is, we require

$$\begin{aligned} \Vert (s_1,\Psi (x_1)) - (s_2,\Psi (x_2)) \Vert \le \Vert (s_1,x_1) - (s_2,x_2) \Vert \quad \forall (s_1,x_1), (s_2,x_2)\in {\mathbb {R}}^m\times {\mathbb {R}}^n.\nonumber \\ \end{aligned}$$

(34)

Before devising a safe conic approximation for the inverse optimization problem (13) with convex hypotheses, we recall that the conjugate of a function $f:{\mathbb {R}}^n\rightarrow {\mathbb {R}}$ is defined through $f^*(z)=\sup _{x\in {\mathbb {R}}^n} \big \langle z, x \big \rangle -f(x)$.

Theorem A.1

(Convex hypotheses and suboptimality loss) Assume that $\mathcal {F}$ represents the class of convex hypotheses induced by the feature map $\Psi$ and with a convex closed search space $\Theta \subseteq {\mathbb {R}}_+^d$ and that Assumption 5.1 holds. If the observer uses the suboptimality loss (4b) and measures risk using the $\mathrm{CVaR}$ at level $\alpha \in (0,1]$, then the following convex program provides a safe approximation for the distributionally robust inverse optimization problem (13) over the 1-Wasserstein ball:

$$\begin{aligned} \begin{array}{llll} \text {minimize} &{}\quad \displaystyle \tau +\frac{1}{\alpha }\left( {\varepsilon } \lambda + {1 \over N}\sum \limits _{i = 1}^{N} r_i\right) \\ \text {subject to} &{}\quad { \theta \in \Theta ,\;\; \lambda \in {\mathbb {R}}_+,\;\;\tau , r_i\in \mathbb R,\;\; \phi _{i1},\phi _{i2}\in {\mathcal {C}}^*, \;\;\gamma _{i} \in {\mathcal {K}}^* ,\;\; z_{ij}\in {\mathbb {R}}^n } &{}\quad \forall i \le N,\forall j\le d\\ &{}\quad \displaystyle \sum _{j=1}^d \theta _j \Psi ^*_j(z_{ij}/\theta _j) + \big \langle \theta , \Psi (\widehat{x}_i) \big \rangle + \big \langle \phi _{i1}, C \widehat{s}_i - d \big \rangle - \big \langle \gamma _i, H \widehat{s}_i + h \big \rangle \le r_i + \tau &{}\quad \forall i \le N \\ &{}\quad \displaystyle \sum _{j=1}^d z_{ij} = W^\top \gamma _i &{}\quad \forall i \le N\\ &{}\quad \big \langle C \widehat{s}_i-d, \phi _{i2} \big \rangle \le r_i &{}\quad \forall i \le N \\ &{}\quad \left\| \begin{pmatrix} H^\top \gamma _i - C^\top \phi _{i1} \\ \theta \end{pmatrix} \right\| _* \le \lambda ,\;\; \left\| \begin{pmatrix} C^\top \phi _{i2} \\ 0 \end{pmatrix} \right\| _* \le \lambda&\quad \forall i \le N. \end{array} \end{aligned}$$

(35)

Note that Theorem A.1 remains valid if $(\widehat{s}_i, \widehat{x}_i)\notin \Xi$ for some $i\le N$.

Proof of Theorem A.1

As in the proof of Theorem 5.2 one can show that the objective function of the inverse optimization problem (13) is equivalent to (19). In the remainder, we derive a safe conic approximation for the (intractable) subordinate worst-case expectation problem

$$\begin{aligned} \sup _{\mathbb {Q}\in \mathbb {B}^{1}_{\varepsilon }({\widehat{\mathbb {P}}_N})} \mathbb {E}^\mathbb {Q}\big [\max \{\ell _\theta (s,x) - \tau , 0\} \big ]. \end{aligned}$$

(36)

To this end, note that the suboptimality loss $\ell _\theta (s,x) = \big \langle \theta , \Psi (x) \big \rangle - \min _{y \in {\mathbb {X}}(s)} \big \langle \theta , \Psi (y) \big \rangle$ depends on x only through $\Psi (x)$. This motivates us to define a lifted suboptimality loss $\ell ^\Psi _\theta (s,\psi ) = \big \langle \theta , \psi \big \rangle - \min _{y \in {\mathbb {X}}(s)} \big \langle \theta , \Psi (y) \big \rangle$, where $\psi$ represents an element of the feature space ${\mathbb {R}}^d$, and the empirical distribution $\widehat{\mathbb {P}}_N^\Psi = \frac{1}{N}\sum _{i=1}^N \delta _{(\widehat{s}_i, \Psi (\widehat{x}_i))}$ of the signal-feature pairs. Moreover, we denote by ${\mathbb {B}}^1_{\varepsilon } (\widehat{\mathbb {P}}_N^\Psi )$ the 1-Wasserstein ball of all distributions on ${\mathbb {S}} \times {\mathbb {R}}^d$ that have a distance of at most $\varepsilon$ from $\widehat{\mathbb {P}}_N^\Psi$. In the following we will show that the worst-case expectation

$$\begin{aligned} \sup _{{\mathbb {Q}} \in {\mathbb {B}}^1_{\varepsilon } (\widehat{\mathbb {P}}_N^\Psi )} \mathbb {E}^\mathbb {Q}\big [\max \{\ell _\theta (s,\psi ) - \tau , 0\} \big ] \end{aligned}$$

(37)

on the signal-feature space ${\mathbb {S}} \times {\mathbb {R}}^d$ provides a tractable upper bound on the worst-case expectation (36). By Definition 4.3, each distribution $\mathbb {Q}\in {\mathbb {B}}^1_{\varepsilon } (\widehat{\mathbb {P}}_N)$ corresponds to a transportation plan $\Pi$, that is, a joint distribution of two signal-response pairs (s, x) and $(s',x')$ under which (s, x) has marginal distribution $\mathbb {Q}$ and $(s',x')$ has marginal distribution $\widehat{\mathbb {P}}_N$. By the law of total probability, the transportation plan can be expressed as $\Pi =\frac{1}{N}\sum _{i=1}^N \delta _{(\widehat{s}_i,\widehat{x}_i)}\otimes \mathbb {Q}_i$, where $\mathbb {Q}_i$ denotes the conditional distribution of (s, x) given $(s',x')=(\widehat{s}_i,\widehat{x}_i)$, $i\le N$, see also [30, Theorem 4.2]. Thus, we the worst-case expectation (36) satisfies

$$\begin{aligned}&\sup _{\mathbb {Q}\in \mathbb {B}^{1}_{\varepsilon }({\widehat{\mathbb {P}}_N})} \mathbb {E}^\mathbb {Q}\big [\max \{\ell _\theta (s,x) - \tau , 0\} \big ] \\&\quad =\sup \limits _{\mathbb {Q}^i}\frac{1}{N} \sum \limits _{i=1}^N \int _{\Xi } \max \{\ell _\theta (s,\Psi (x)) - \tau , 0\} \, \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}x) \\&\quad \text {s.t.}\frac{1}{N} \sum \limits _{i=1}^N \int _{\Xi } \Vert (s,x) - (\widehat{s}_i, \widehat{x}_i ) \Vert \, \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}x) \le \varepsilon \\&\quad \int _{\Xi } \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}x) = 1 \qquad \forall i \le N \\&\quad \le \sup \limits _{\mathbb {Q}^i}\frac{1}{N} \sum \limits _{i=1}^N \int _{\Xi } \max \{\ell _\theta (s,\Psi (x)) - \tau , 0\} \, \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}x) \\&\quad \text {s.t.}\frac{1}{N} \sum \limits _{i=1}^N \int _{\Xi } \Vert (s,\Psi (x)) - (\widehat{s}_i, \Psi (\widehat{x}_i) ) \Vert \, \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}x) \le \varepsilon \\&\quad \int _{\Xi } \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}x) = 1 \qquad \forall i \le N \\&\quad \le \sup \limits _{\mathbb {Q}^i}\frac{1}{N} \sum \limits _{i=1}^N \int _{{\mathbb {S}} \times {\mathbb {R}}^d} \max \{\ell _\theta (s,\psi ) - \tau , 0\} \, \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}\psi ) \\&\quad \text {s.t.}\frac{1}{N} \sum \limits _{i=1}^N \int _{{\mathbb {S}} \times {\mathbb {R}}^d} \Vert (s,\psi ) - (\widehat{s}_i, \Psi (\widehat{x}_i) ) \Vert \, \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}\psi ) \le \varepsilon \\&\quad \int _{{\mathbb {S}} \times {\mathbb {R}}^d} \mathbb {Q}^i(\mathrm {d}s, \mathrm {d}\psi ) = 1 \qquad \forall i \le N, \end{aligned}$$

where the first inequality follows from (34), while the second inequality follows from relaxing the implicit condition that the signal-feature pair $(s,\psi )$ must be supported on $\{(s,\Psi (x)):(s,x)\in \Xi \}\subseteq \mathbb {S}\times {\mathbb {R}}^d$. Using a similar reasoning as before, the last expression is readily recognized as the worst-case expectation (37). Thus, (37) provides indeed an upper bound on (36). Duality arguments borrowed from [30, Theorem 4.2] imply that the infinite-dimensional linear program (37) admits a strong dual of the form

$$\begin{aligned} \begin{array}{cll} \inf \limits _{\lambda \ge 0, r_i} &{} \displaystyle \lambda \varepsilon + \frac{1}{N} \sum _{i=1}^N r_i &{} \\ \text {s.t.} &{} \sup \limits _{(s,y) \in \Xi , \psi \in {\mathbb {R}}^d} \big \langle \theta , \psi - \Psi (y) \big \rangle - \tau - \lambda \Vert (s,\psi ) - (\widehat{s}_i, \Psi (\widehat{x}_i)) \Vert \le r_i &{} \forall i \le N \\ &{}\sup \limits _{s \in {\mathbb {S}}, \psi \in {\mathbb {R}}^d} -\lambda \Vert (s,\psi ) - (\widehat{s}_i, \Psi (\widehat{x}_i)) \Vert \le r_i &{} \forall i \le N. \end{array} \end{aligned}$$

(38)

By using the definitions of $\mathbb S$ and $\mathbb {X}(s)$ put forth in Assumption 5.1, the i-th member of the first constraint group in (38) is satisfied if and only if the optimal value of the maximization problem

$$\begin{aligned} \begin{array}{cl} \sup \limits _{s,y,\psi } &{} \big \langle \theta , \psi - \Psi (y) \big \rangle - \tau - \lambda \Vert (s,\psi ) - (\widehat{s}_i, \Psi (\widehat{x}_i)) \Vert \\ \text {s.t.}&{} C s \succeq _{\mathcal {C}} d,\; Wy \succeq _{\mathcal {K}} Hs + h \end{array} \end{aligned}$$

(39)

does not exceed $r_i$. A tedious but routine calculation shows that the dual of (39) can be represented as

$$\begin{aligned} \begin{array}{cll} \inf \limits _{p_i, q_i, \gamma _i, \phi _{i1}, z_{ij}} &{} \displaystyle \sum _{j=1}^d \theta _j \Psi ^*_j(z_{ij}/\theta _j) - \tau + \big \langle \theta , \Psi (\widehat{x}_i) \big \rangle + \big \langle \phi _{i1}, C \widehat{s}_i - d \big \rangle - \big \langle \gamma _i, H \widehat{s}_i + h \big \rangle \\ \text {s.t.} &{} \sum _{j=1}^d z_{ij} = W^\top \gamma _i \\ &{} \Vert (H^\top \gamma _i - C^\top \phi _{i1}, \theta ) \Vert _* \le \lambda \\ &{} \gamma _i \in \mathcal K^*, \phi _{i1} \in \mathcal C^*. \end{array} \end{aligned}$$

(40)

Note that the perspective functions $\theta _j \Psi ^*_j(z_{ij}/\theta _j)$ in the objective of (40) emerge from taking the conjugate of $\theta _j \Psi _j(y)$. Thus, for $\theta _j=0$ we must interpret $\theta _j \Psi ^*_j(z_{ij}/\theta _j)$ as an indicator function in $z_{ij}$ which vanishes for $z_{ij}=0$ and equals $\infty$ otherwise. By weak duality, (40) provides an upper bound on (39). We conclude that the i-th member of the first constraint group in (38) is satisfied whenever the dual problem (40) has a feasible solution whose objective value does not exceed $r_i$. A similar reasoning shows that the i-th member of the second constraint group in (38) holds if and only if there exists $\phi _{i2} \in \mathcal {C}^*$ such that

$$\begin{aligned}&\begin{array}{clll} &{} \big \langle C\widehat{s}_i-d, \phi _{i2} \big \rangle \le r_i \quad \text {and} \quad \left\| \begin{pmatrix}C^\intercal \phi _{i2} \\ 0\end{pmatrix} \right\| _* \le \lambda . \end{array} \end{aligned}$$

Thus, the worst-case expectation (36) is bounded above by the optimal value of the finite convex program

$$\begin{aligned} \begin{array}{clll} \inf \limits _{} &{}\quad \displaystyle {\varepsilon } \lambda + {1 \over N}\sum \limits _{i = 1}^{N} r_i \\ \text {s.t.}&{}\quad { \lambda \in {\mathbb {R}}_+,\;r_i\in {\mathbb {R}},\; \phi _{i1},\phi _{i2}\in \mathcal {C}^*, \;\gamma _{i}\in \mathcal {K}^* } &{}\quad \forall i \le N \\ &{}\quad \sum _{j=1}^d \theta _j \Psi ^*_j(z_{ij}/\theta _j) + \big \langle \theta , \Psi (\widehat{x}_i) \big \rangle + \big \langle \phi _{i1}, C \widehat{s}_i - d \big \rangle - \big \langle \gamma _i, H \widehat{s}_i + h \big \rangle \le r_i + \tau &{}\quad \forall i \le N \\ &{}\quad \sum _{j=1}^d z_{ij} = W^\top \gamma _i &{}\quad \forall i \le N \\ &{}\quad \big \langle C \widehat{s}_i-d, \phi _{i2} \big \rangle \le r_i\\ &{}\quad \left\| \begin{pmatrix} H^\top \gamma _i - C^\top \phi _{i1} \\ \theta \end{pmatrix} \right\| _* \le \lambda ,\;\; \left\| \begin{pmatrix} C^\top \phi _{i2} \\ 0 \end{pmatrix} \right\| _* \le \lambda&\quad \forall i \le N. \end{array} \end{aligned}$$

The claim then follows by substituting this convex program into (19).