CVaR distance between univariate probability distributions and approximation problems

Abstract

The paper defines new distances between univariate probability distributions, based on the concept of the CVaR norm. We consider the problem of approximation of a discrete distribution by some other discrete distribution. The approximating distribution has a smaller number of atoms than the original one. Such problems, for instance, must be solved for generation of scenarios in stochastic programming. The quality of the approximation is evaluated with new distances suggested in this paper. We use CVaR constraints to assure that the approximating distribution has tail characteristics similar to the target distribution. The numerical algorithm is based on two main steps: (i) optimal placement of positions of atoms of the approximating distribution with fixed probabilities; (ii) optimization of probabilities with fixed positions of atoms. These two steps are iterated to find both optimal atom positions and probabilities. Numerical experiments show high efficiency of the proposed algorithms, solved with convex and linear programming.

Appendices

Appendix A: Risk-measure-based distance between maximal monotone relations

This section defines the notion of a risk-measure-based distance between two probability distributions in a formal way. It is convenient to formally define risk-measure-based distances based on maximal monotone relations, Rockafellar and Royset (2014).

A general cumulative distribution function can have points of discontinuity, however, if the corresponding jumps of such function are filled with vertical segments, we obtain an example of the maximal monotone relation. In a similar fashion, the quantile function of a probability distribution can generate a maximal monotone relation. We will consider further distances based on both cumulative distribution functions and quantile functions, so it is convenient to define the notion of distance in a more general fashion. First, we define the notion of a monotone relation on a set $\mathcal A \subseteq \mathbb R$.

Definition 9

(Rockafellar and Royset 2014). Let $\mathcal A = [a, b] \subseteq \mathbb {R}$, possibly unbounded closed interval. A set $\Gamma = \{(x,\, p) \subset \mathcal A \times \mathbb R \}$ is called a monotone relation on $\mathcal {A}$ if $\, \forall \; (x_1,\, p_1),\, (x_2,\, p_2) \in \Gamma$

$$\begin{aligned} (x_1 - x_2)(p_1 - p_2) \ge 0. \end{aligned}$$

(A.1)

Set $\Gamma$ is called a maximal monotone relation on $\mathcal A$ if there exists no monotone relation $\Gamma ^{'} \ne \Gamma$ on $\mathcal A$, such that $\Gamma \subset \Gamma ^{'}$.

Associated with a maximal monotone relation $\Gamma$ on $\mathcal A$, the function $\Gamma (x),\, x \in \mathcal A$, is defined. An arbitrary monotone relation can clearly contain a vertical segment, therefore, let $\Gamma (x)$ be defined as

$$\begin{aligned} \Gamma (x) = {\left\{ \begin{array}{ll} \displaystyle \inf _{(x,\, p) \in \Gamma } \; p , &{} \text { if } \quad x=b<+\infty , \\ \displaystyle \sup _{(x,\, p) \in \Gamma } \; p, &{} \text { otherwise}, \end{array}\right. } \end{aligned}$$

(A.2)

where b is the right point of the closed interval $\mathcal {A}$. Clearly, $\Gamma (x)$ is a nondecreasing function on $\mathcal {A}$.

Suppose F and G are two maximal monotone relations on $\mathcal A$ and we randomly pick a point $\xi \in \mathcal {A}$, so that the absolute difference between F and G becomes a random variable taking the value $|F(\xi ) - G(\xi )|$. Specifically, we suppose there is an underlying probability space $(\Omega , \mathcal {F}, \mathbb {P})$ and the random variable $\xi$ is a $\mathcal {F}-$measurable function from $\Omega$ to $\mathcal A$, $\xi : \Omega \rightarrow \mathcal {A}$. Let $\mathcal A$ be equipped with a Borel $\sigma -$algebra $\mathcal B$. Moreover, the auxiliary random variable $\xi$ is supposed to have a probability distribution H, such that (i) it has a density function $h(x), \, x \in \mathcal A$ and (ii) $h(x) > 0$ for any $x \in int(\mathcal A)$. The distance (discrepancy metric) between F and G will be defined using a risk measure to the random variable $|F(\xi ) - G(\xi )|$.

A risk measure $\mathcal R$ is a map from a space of random variables into $\mathbb R$. In our study $\mathcal R$ belongs to a special class of risk measures, called coherent risk measures and defined in Artzner et al. (1999). To be coherent, a risk measure has to satisfy the following axioms (the axioms are in a slightly different from Artzner et al. (1999) form, due to Rockafellar and Uryasev (2013)):

• A1. $\mathcal R(\xi ) = C$ for constant random variables $\xi = C$ a.s.,

• A2. $\mathcal R(\xi _1) \le \mathcal R(\xi _2)$ for $\xi _1 \le \xi _2$ a.s.,

• A3. $\mathcal R(\xi _1 + \xi _2) \le \mathcal R(\xi _1) + \mathcal R(\xi _2)$,

• A4. $\mathcal R(\lambda \xi _1) = \lambda \mathcal R(\xi _1)$, for any $\lambda \in \mathbb (0, +\infty )$.

Definition 10

The risk-measure-based distance between maximal monotone relations on $\mathcal {A}$, F and G, is defined through the corresponding functions $F(\cdot )$ and $G(\cdot )$, as follows:

$$\begin{aligned}&d^H(F, G) = \mathcal R \big (|F(\xi )-G(\xi )|\big ). \end{aligned}$$

(A.3)

The function $d^H(F, G)$ satisfies the usual properties of a probability metric, discussed in, for instance, Rachev et al. (2008), Chapter 3.

Proposition A.1

Let F, Z, G be maximal monotone relations on $\mathcal A = [a,b]$. If H is a distribution with a density function $h(x) > 0,\, \forall x \in int(\mathcal A)$, the following properties hold:

• 1. $d^H(F, G) \ge 0$

• 2. $d^H(F, G) = 0 \iff \mu \big (\{x : F(x) \ne G(x)\}\big ) = 0$ where $\mu$ denotes Lebesgue measure

• 3. $d^H(F, G) = d^H(G, F)$

• 4. $d^H(F, Z) \le d^H(F, G) + d^H(G, Z)$

Proof

First, correctness of definition $|F(\xi ) - G(\xi )|$ needs to be shown, i.e., that $|F(\xi ) - G(\xi )|$ has to be a $\mathcal F-$ measurable function. It is sufficient to show that $F(\xi )$ is measurable, i.e., that the preimage of an open set in $\mathcal A$ is in $\mathcal {F}$. In order to see this sufficiency, note first that the sum or the difference of two measurable functions is measurable, see for instance McDonald and Weiss (1999), Chapter 3. Then, it is well-known that a preimage (with respect to a continuous function) of an open set is another open set, see McDonald and Weiss (1999), Chapter 2, therefore the absolute value function also preserves measurability. Thus, consider the measurability of the function $F(\cdot )$. The function $F(\cdot )$ is associated with a maximal monotone relation F, therefore it is nondecreasing and has at most countable number of points of discontinuity, cf. Rudin (1964) for example. Therefore, $F(\cdot )$ can be approximated by a sequence of continuous nondecreasing functions $F_n(\cdot )$ such that the sequence converges pointwise to F: $\displaystyle \forall x \in \mathcal A,\, \lim _{n\rightarrow \infty } F_n(x) = F(x)$. By Theorem 4.5 in McDonald and Weiss (1999), the function $F(\cdot )$ is measurable.

Properties 1, 3, 4 are trivial and direct consequences of the axioms of coherent risk measures. The proof of Property 2 follows next.

1. We start from the $\implies$ implication. Suppose $d^H(F,G) = 0$, in other words, $0 = \mathcal R(|F(\xi ) - G(\xi )|) \le \mathcal R(0)$, which implies by the property A2 of coherent risk measures that $\mathbb P(\omega : |F(\xi (\omega ) ) - G(\xi (\omega ))| \le 0) = \mathbb P(\omega : F(\xi (\omega ) ) = G(\xi (\omega ))) = 1$. Thus, we obtain the following

$$\begin{aligned}&\mathbb P\big (\omega : F(\xi (\omega )) \ne G(\xi (\omega ))\big ) = 0. \end{aligned}$$

(A.4)

Let $\mathcal {A}^{'} \subset \mathcal A$ denote the image of $\xi$. Clearly, $\mu \left( \mathcal A^{'}\right) = \mu (\mathcal A)$ because of the absolute continuity of the distribution of $\xi$. Then,

$$\begin{aligned} \mu \{x \in \mathcal A: F(x) \ne G(x)\} = \mu \left\{ x \in \mathcal A^{'}: F(x) \ne G(x)\right\} . \end{aligned}$$

(A.5)

Let $E = \left\{ x \in \mathcal A^{'}: F(x) \ne G(x)\right\}$. Consider a sequence of $\{\epsilon _k > 0\}, \, \epsilon _k \rightarrow 0,\, k \rightarrow +\infty$ and let $E_k = \left\{ x \in \mathcal A^{'}: F(x) \ne G(x),\, h(x) \ge \epsilon _k\right\}$. Clearly, $\displaystyle \bigcup ^{\infty }_{k=1} E_k = E$. Also,

$$\begin{aligned} 0&= \mathbb {P} \big (\omega : F(\xi (\omega )) \ne G(\xi (\omega ))\big ) \ge \int _{E_k} h d\mu \ge \epsilon _k \int _{E_k} d\mu = \epsilon _k \mu (E_k) \implies \end{aligned}$$

(A.6)

$$\begin{aligned} \mu \{E_k\}&= 0, \quad k = 1,\ldots ,+\infty . \end{aligned}$$

(A.7)

Thus

$$\begin{aligned} \mu (E) = \mu \left( \bigcup ^{\infty }_{k=1}E_k\right) \le \sum ^{\infty }_{k=1} \mu (E_k) = 0. \end{aligned}$$

(A.8)

2. The implication is more obvious. Let $E = \big \{x \in \mathcal A: F(x) \ne G(x)\big \}$. Then, due to $\xi$ having the density function h,

$$\begin{aligned} \mathbb {P}\big (\omega : F(\xi (\omega )) \ne G(\xi (\omega ))\big ) = \int _{E} h d\mu = 0, \end{aligned}$$

(A.9)

as an integral of the nonnegative function over the set of measure 0. $\square$

Appendix B

Formulation 1

The problem (18)–(20) for $0< \alpha < 1$ can be reformulated as the following linear problem:

$$\begin{aligned}&\min _{c,\, p_{i},\, z_k} \left( c + \frac{1}{1-\alpha }\sum _{k=1}^{s-1}\mathbb {P}(d_{k})z_{k}\right) \end{aligned}$$

(B.1)

$$\begin{aligned}&\text{ subject } \text{ to } \nonumber \\&z_{k} \ge \;\;\, F(t_{k}) - G(t_{k}) - c,\quad k = 1,\ldots ,s-1, \end{aligned}$$

(B.2)

$$\begin{aligned}&z_{k} \ge -F(t_{k}) + G(t_{k}) - c,\quad k = 1,\ldots ,s-1, \end{aligned}$$

(B.3)

$$\begin{aligned}&F(t_{k}) = \sum ^{n}_{i=1} p_{i} \mathbb {1}_{t_{k}\ge x_{i}},\quad k = 1,\ldots ,s-1, \end{aligned}$$

(B.4)

$$\begin{aligned}&\sum ^{n}_{i=1} p_{i} = 1, \end{aligned}$$

(B.5)

$$\begin{aligned}&z_{k} \ge 0,\quad k = 1,\ldots ,s-1, \end{aligned}$$

(B.6)

$$\begin{aligned}&p_{i} \ge 0,\quad i = 1,\ldots ,n. \end{aligned}$$

(B.7)

Formulation 2

The problem (18)–(20) with $\alpha = 1$ can be reformulated as the following linear problem:

$$\begin{aligned}&\min _{a, \, p_{i}} \quad a \end{aligned}$$

(B.8)

$$\begin{aligned}&\text{ subject } \text{ to } \nonumber \\&a \ge \;\;\, F(t_{k}) - G(t_{k}),\quad k=1,\ldots ,s-1, \end{aligned}$$

(B.9)

$$\begin{aligned}&a \ge -F(t_{k}) + G(t_{k}),\quad k=1,\ldots ,s-1, \end{aligned}$$

(B.10)

$$\begin{aligned}&F(t_{k}) = \sum ^{n}_{i=1} p_{i} \mathbb {1}_{t_{k} \, \ge \, x_{i}},\quad k = 1,\ldots ,s-1,\nonumber \\&(B.5),\, (B.7). \end{aligned}$$

(B.11)

Formulation 3

The problem (18)–(20) with $\alpha = 0$ can be reformulated as the following linear problem:

$$\begin{aligned}&\min _{p_{i},\, d_k}\quad \sum ^{s-1}_{k=1}d_{k} \mathbb {P}(d_{k}) \end{aligned}$$

(B.12)

$$\begin{aligned}&\text{ subject } \text{ to } \nonumber \\&d_{k} \ge \;\;\, F(t_{k}) - G(t_{k}),\quad k=1,\ldots ,s-1, \end{aligned}$$

(B.13)

$$\begin{aligned}&d_{k} \ge -F(t_{k}) + G(t_{k}),\quad k=1,\ldots ,s-1, \end{aligned}$$

(B.14)

$$\begin{aligned}&F(t_{k}) = \sum ^{n}_{i=1} p_{i} \mathbb {1}_{t_{k}\,\ge \, x_{i}},\quad k = 1,\ldots ,s-1,\nonumber \\&(B.5),\, (B.7). \end{aligned}$$

(B.15)

Formulation 4

The problem (29)–(33) with $\alpha \in (0,1)$ can be reformulated as the following linear problem:

$$\begin{aligned}&\min _{c,\, p_i,\, r_i,\, z_{k}}\quad \left( c + \frac{1}{1-\alpha }\sum _{k=1}^{m-1}\mathbb {P}(d_{k})z_{k}\right) \end{aligned}$$

(B.16)

$$\begin{aligned}&\text{ subject } \text{ to } \nonumber \\&z_{k} \ge \;\;\,F(y_{k}) - G(y_{k}) - c,\quad k = 1,\ldots ,m-1, \end{aligned}$$

(B.17)

$$\begin{aligned}&z_{k} \ge -F(y_{k}) + G(y_{k}) - c,\quad k = 1,\ldots ,m-1, \end{aligned}$$

(B.18)

$$\begin{aligned}&F(y_{k}) = \sum ^{m}_{i=1} p_{i} \mathbb {1}_{y_{k}\,\ge \, y_{i}},\quad k = 1,\ldots ,m-1, \end{aligned}$$

(B.19)

$$\begin{aligned}&p_{i} \le r_{i}, \quad i=1,\ldots ,m, \end{aligned}$$

(B.20)

$$\begin{aligned}&\sum ^{m}_{i=1} r_{i} \le r, \end{aligned}$$

(B.21)

$$\begin{aligned}&z_{k} \ge 0,\quad k = 1,\ldots ,m-1, \end{aligned}$$

(B.22)

$$\begin{aligned}&r_{i} \in \{0,1\},\quad i=1,\ldots ,m,\nonumber \\&(B.5),\, (B.7). \end{aligned}$$

(B.23)

Corresponding reformulations for problem (29)–(33) with $\alpha = 1$ and $\alpha = 0$ can be obtained similar to Formulations 2 and 3.

Formulation 5

The problem (61)–(62) with $\alpha \in (0,1)$ for the minimization of CVaR distance between quantile functions can be reformulated as the following linear problem:

$$\begin{aligned}&\min _{c,\, x_{i},\, z_k} \left( c + \frac{1}{1-\alpha }\sum _{k=1}^{s-1}\mathbb {P}(d_{k})z_{k}\right) \end{aligned}$$

(B.24)

$$\begin{aligned}&\text{ subject } \text{ to } \nonumber \\&z_{k} \ge \;\;\, x_{i_{\gamma _k}} - y_{j_{\gamma _k}} - c,\quad k=1,\ldots ,s-1, \end{aligned}$$

(B.25)

$$\begin{aligned}&z_{k} \ge - x_{i_{\gamma _k}} + y_{j_{\gamma _k}} - c,\quad k=1,\ldots ,s-1, \end{aligned}$$

(B.26)

$$\begin{aligned}&z_{k} \ge 0, \quad k = 1,\ldots ,s-1, \end{aligned}$$

(B.27)

$$\begin{aligned}&x_{1} \le \ldots \le x_{n}. \end{aligned}$$

(B.28)

Corresponding reformulations for problem (61)–(62) with $\alpha = 1$ and $\alpha = 0$ can be obtained similar to Formulations 2 and 3.