Computational and Analytical Bounds for Multivariate Bernoulli Distributions

Building on a new but simple method to characterize multivariate Bernoulli variables with given means, we investigate their dependence structure. We evaluate on some computational examples whether the assumption of exchangeability is binding. This is useful in applications where exchangeability is a standard assumption, such as credit risk.


Introduction
Multivariate Bernoulli variables and their dependence structure are widely studied in the statistical literature, see e.g., [1].A part of the literature focuses on exchangeable Bernoulli variables for their importance in applications, as credit risk modeling [6], and for the De Finetti representation theorem, which describes the dependence structure in a very simple way.Nevertheless, the De Finetti theorem holds only for infinite sequences of exchangeable variables (see, e.g., [3]).
A novel representation of the class of multivariate Bernoulli variables with some given moments is provided in [4].If we consider the Fréchet class of ddimensional Bernoulli variables with given one-dimensional means ( p 1 , . . ., p d ), i.e., F( p 1 , . . ., p d ), we can use this representation to investigate the dependence structure of the class.In fact, in [4], the probability mass functions belonging to F( p 1 , . . ., p d ) are represented as points in a convex hull whose generators are mass functions which belong to the same class.The generators of the class can be explicitly found, although we do not have a general analytical expression for them.This representation is general and allows us to easily generate a sample of mass functions in the class and to find bounds for the other moments of the distribution.It is worth noting that this method puts no restriction on the number of variables.The range of applications is limited only by the amount of computational effort required, because the number of generators increases very quickly as the dimension of the multivariate Bernoulli variables increases.This limitation is overcome if we consider the class of exchangeable Bernoulli variables.Fontana et al. [5] analytically finds the convex hull generators for the class of exchangeable Bernoulli variables with given mean.This analytical representation holds for any finite sequence of exchangeable Bernoulli variables, thus in a more general framework than the De Finetti representation theorem.The analytical solution allows us to work in any dimension.
The aim of this paper is to investigate some Fréchet classes and to compare the entire Frechét class with the subclass of exchangeable random variables.For this reason, we choose to assume that the vector has identically distributed Bernoulli margins, i.e., they all have the same mean.This analysis is computational because we have the analytical solution only under the assumption of exchangeability.As a consequence, it cannot be developed in very high dimension because of the computational effort required.Nevertheless, we work in a truly multidimensional setting, since we reach dimension six.Our comparison between the whole Fréchet class and the set of exchangeable variables makes it possible to draw some conclusions about the limitation that can derive from the assumption of exchangeability in applications.
The paper is organized as follows.After a preliminary section, we restate the theoretical results in [4], but using the same approach that is used in [5] to focus on the exchangeable case.We also recall the analytical construction in the exchangeable case.In Sect.4, we investigate two special cases.For each case, we find the generators of the class and we provide bounds for the first four cross-moments, since they drive dependence.In particular, we exhibit the correlation bounds, to underline the admissible strength of linear dependence in the class.We also consider the value at risk (VaR) of the sums of Bernoulli variables.This measure is important in credit risk, where Bernoulli variables are indicators of default.In fact, VaR is an indicator for the possible loss of a portfolio with dependent obligors.Interestingly, we find that the bounds for the VaR remain the same if we consider the subclass of exchangeable variables and therefore they can be computed on this subclass and obtained analytically.1) by q i and p i , respectively, i ∈ {1, . . ., d}.
We observe that q i = 1 − p i and that the expected value of and #{x j : x j = 1} = i.Therefore, we identify a mass function f X d in E d ( p) with the corresponding vector f := ( f 0 , . . ., f d ).The simplest example of exchangeable distribution is the case of independence that is linked to the Binomial distribution.
Furthermore, the moments depend only on their order, we therefore use μ α to denote a moment of order α = ord(α) = d i=1 α i , where α ∈ X d .For example, we have μ 1 = p.We also observe that the correlation ρ between two Bernoulli variables X i ∼ B( p) and X j ∼ B( p) is related to the second-order moment μ 2 = E[X i X j ] as follows:

Theoretical Background
Building on the results in [4,5] in this section, we represent the Fréchet class of multivariate d-dimensional Bernoulli distributions with given margins, d ≥ 2 as the points of a convex polytope.We recall that a polytope (or more specifically a dpolytope) is the convex hull of a finite set of points in R d called the extremal points of the polytope.We say that a set of k points is affinely independent if no one point can be expressed as a linear convex combination of the others.For example, three points are affinely independent if they are not on the same line, four points are affinely independent if they are not on the same plane, and so on.The convex hull of k + 1 affinely independent points is called a simplex or k-simplex.For example, the line segment joining two points is a 1-simplex, the triangle defined by three points is a 2-simplex, and the tetrahedron defined by four points is a 3-simplex.A complete reference on computational geometry is [2].The representation of F( p 1 , . . ., p d ) as a convex polytope holds for any p, with the drawback that the search of the generators, is computationally challenging for high dimension.This limitation is not present in the class of exchangeable d-dimensional Bernoulli distributions with given margins, where we have an analytical expression for the convex polytope generators.
Let f X d be a multivariate d-dimensional Bernoulli distribution with margins p, i.e., Using the conditions on the mean values, we can write any vector density f X d in F( p 1 , . . ., p d ) as the solution of a linear system.Formally, since ( The d equations in Eq. ( 2) provide a linear system.Let H be its coefficients matrix.The rows of H are (γ i (1 − x i ) − x i ), i ∈ {1, . . ., d}, where 1 is the vector with all the elements equal to 1 and x i is the projection vector which contains only the i-th element of x ∈ X d , i ∈ {1, . . ., d}, e.g., for the bivariate case x 1 = (0, 1, 0, 1) and x 2 = (0, 0, 1, 1).
The densities f X d in F( p 1 , . . ., p d ) are the positive solutions of the system H z = 0, whose components sum up to one.
All the positive, normalized, solutions of H z = 0 are elements of the convex polytope , where I is the 2 d × 2 d identity matrix.Each point in the polytope is a convex combinations of a set of generators which are referred to as extremal densities of the linear system.We denote them as R (i)  X d , i = 1, . . ., n F and n F is the number of generators that depends on d and p.
Using the above arguments, [4] proved the following theorem. where and n P is the number of the extremal points of P.
To find the extremal densities, i.e., the generators of F( p 1 , . . ., p d ), we have to find the extremal solutions of an homogeneous system.If the dimension of the system increases, the number of extremal solutions becomes huge, leading to computational difficulties.These difficulties disappear when we consider the class E d ( p) of exchangeable Bernoulli variables, where we have the analytical expression of the extremal densities.If The map,  6), this is also equivalent to find a set of conditions that a pmf of a multivariate Bernoulli has to satisfy for being in E d ( p).Following the approach developed in the proof of Theorem 1, the set of conditions are homogeneous equations, whose unknown are the values of a pmf in D d ( pd).
Proposition 1 Let Y be a discrete random variable defined over {0, . . ., d} and let p Y be its pmf.Then,

123
Using Proposition 1, we can find all generators of S d ( p).Thanks to the map E, that is equivalent to finding all the generators of E d ( p).
We have to find the normalized extremal points of the convex cone where a j = j − pd and I is the (d + 1) × (d + 1) identity matrix.The following proposition, proved in [5], provides the analytical expression of the extremal points in S d ( p).

Proposition 2
The extremal points of the convex cone C p in (7) are

the largest integer less than pd and j m
2 is the smallest integer greater than pd.If pd is integer, the extremal points contain also A corollary of the above proposition is the number of ray densities.

Corollary 1
If pd is not integer there are n p = ( j M 1 + 1)(d − j M 1 ) extremal densities.If pd is integer there are n p = d 2 p(1 − p) + 1 extremal densities.

Moments, Quantiles and their Bounds
This section focuses on the problem of finding bounds for the moments of multivariate Bernoulli variables in F( p 1 , . . ., p d ) and in We denote by A X d the matrix whose columns contain all the moments of the extremal mass functions, where M ⊗d k is the sub-matrix of M ⊗d obtained by selecting the rows corresponding to the k-order moments and R X d is the ray matrix.We observe that the columns of the matrix A kX d contain the moments of the extremal mass functions, i.e., the bounds for the k-th order moment are reached on the extremal densities.
where A (α) 2 p λ and {i, j} = {k : As we observed, for a given p, the class of exchangeable multivariate pmfs E d ( p) is a subclass of the Frechét class F( p, . . ., p) where all margins are equal to p.For the sake of simplicity, we denote F d ( p, . . ., p) by F d ( p).
If we consider the class E d ( p) of exchangeable distributions, the moments depend only on their order.Therefore, as said in the preliminaries, we use μ α to denote a moment of order α.Being E d ( p) ⊂ F d ( p), the above bounds are still true; thus, the minimum and the maximum moments are reached on the ray densities of E d ( p).We expect that the bounds for the moments of the variables in E d ( p) are more binding that the bounds for the moments in F d ( p).We computationally investigate this aspect on some cases in Sect. 4.There is no particular relation between the extremal densities of the Fréchet class F d ( p) and the extremal densities of the exchangeable class E d ( p) apart the fact that the extremal density f U (the upper Fréchet bound) defined as belongs to both sets of extremal densities.An example of extremal densities is given provided in Sect. 4.
The class E d ( p) is of interest in several fields including finance, where exchangeable Bernoulli variables are used to model indicators of default of the obligors in a credit risk portfolio.In this framework, the distribution of the number of defaults, i.e., the sum of the components of an exchangeable multivariate Bernoulli variable, is studied.One of the quantities of interest are the quantiles of the distribution, q α .For some levels of α, the quantiles are measures of risk and often referred to as Value at risk (VaR α ).
Definition 1 Let Y be a random variable with finite mean.Then, the VaR α at level α is defined by In [5], the authors prove that the bounds of the quantiles of a distribution p S ∈ S d ( p) are reached on the ray densities and they analytically find them.In particular, they prove the following.

Proposition 5 Let us consider the class S d ( p). Let j
1 be the largest integer less than pd and j m 2 be the smallest integer greater than pd. 1.If j p 1 < 0, min q α (R ( j 1 , j 2 ) ) = 0 and max q α (R ( j 1 , j 2 ) ) = j * 2 , where j * 2 is the largest integer smaller than pd where j * 1 is the smallest integer greater or equal to j p The proof of the above propositions relies on the analytical expression of the extremal densities of the convex polytope S d ( p).For this reason, the assumption of exchangeability does not affect these bounds.Precisely, let Therefore, the quantile of S X is the quantile of a distribution in the class S d ( p) and satisfies the bounds in Proposition 5.This fact is of interest in credit risk, since it states that the assumption of exchangeability does not effect the bounds of the value at risk.

Computational Results for Some Frechét Classes
This section explores some Fréchet classes for given one-dimensional marginal probabilities.To make comparisons between the general case and the exchangeable case, we choose two Fréchet classes of d-dimensional Bernoulli variables with identically distributed one-dimensional margins.We consider the classes F d  , 5. Table 1 provides the number of extremal points for each class and exhibits the computational effort necessary to work in the general case and high dimension.
Case d = 2 is analytical.We know that the extremal densities are the upper and lower Fréchet bound, as proved in [4].The same extremal densities generate E 2 ( 12 ); in fact, in the bi-dimensional case, the condition to have the same margins implies exchangeability.
As a simple example, for case d = 3 and p = 1/2, we provide the extremal densities of the Fréchet class F 3 1 2 (Table 2) and the extremal densities of the exchangeable class E 3 1 2 (Table 3).As we already pointed out, the upper Fréchet bound (R (5) F in Table 2 and R (2) E in Table 3) belongs to both classes.As can be seen, the number of the generators of the whole Fréchet class increases very quickly, while the number of generators of the subclass of exchangeable variables is much smaller.This means that working under the assumption of exchangeability is far easier.The following two sections explore how much it could be binding to assume exchangeability in terms of dependence flexibility.To do this, we find the bounds for the cross-moments of the entire Fréchet class and of the exchangeable subclass to  consider both linear and nonlinear dependence.We also consider the VaR of the sums, whose bounds-as discussed in Sect.3.1-are not affected by the assumption of exchangeability.

The Class F d 1 2
In this section, we consider the case p = 1 2 and d = 2, . . ., 6.We conclude this section with the bounds for the value at risk VaR 0.95 of the sums, i.e., the quantile q 0.95 of the distribution of S X = X 1 + . . .+ X d , where X has pmf in F d ( 12 ).The bounds, in Table 5, remain the same if we assume that X has pmf in E d ( 12 ).Notice that the maximum VaR is always the dimension d; this is probably due to the fact that marginal probability p = 1  2 is quite large.The results in [5], where marginal default probabilities are small, support this interpretation.In this section, we consider the case p = 1 5 and d = 2, . . ., 6. Table 6 reports the bounds for moments of order 2, . . ., 6 both for the Fréchet class We conclude this section with the bounds for the value at risk VaR 0.95 of the sums, i.e., the quantile q 0.95 of the distribution of S X , where X has pmf in F d 1 5 .The bounds are in Table 7.Also in this case, the bounds remain the same if we assume that X has pmf in E d and #{x j : x j = 1} = i.Using this fact, we can define a one-to-one correspondence between E d ( p) and the class of the distributions of their sums.Let S d ( p) be the class of distributions p S on {0, . . ., d} such that S d = d i=0 X i with X ∈ E d ( p).Let p S ( j) = p j = P(S d = j) and p S = ( p 0 , . . ., p d ).

Proposition 3 Proposition 4
For each α ∈ X d , α 0 = k, the k-order moment μ the row of the matrix A kX d such that μ Important special cases are the second-order moments which allow us to find bounds for correlations: The correlations ρ i j must satisfy the following bounds:min A (α) d = 2, . . .

1 5
and the maximum VaR is always the dimension d.

2 Preliminaries
Let F d be the set of d-dimensional distributions which have Bernoulli univariate marginal distributions.Let us consider the Fréchet classF( p 1 , . .., p d ) ⊆ F d of distribution functions in F d which have Bernoulli marginal distributions B( p i ), 0 < p i < 1, i ∈ {1, . .., d}.If X = (X 1 , . ..,X d ) is a random vector with joint distribution in F( p 1 , . . ., p d ), we denote -Its cumulative distribution function by F X d and its probability mass function (pmf) by f X d , where X d = {0, 1} d ; -The column vector which contains the values of F and f over X d , by F X d = (F p (x) : x ∈ X d ) and f X d = ( f p (x) : x ∈ X d ) respectively; we make the non-restrictive hypothesis that the set X d of 2 d binary vectors is ordered according d}.Let now E d ( p) be the class of d -dimensional exchangeable Bernoulli distributions with mean p.If X = (X 1 , . .., X d ) is a random vector with joint distribution in E d ( p), it holds f X d (x) = f X d (σ (x))for any σ ∈ P d , where P d is the set of permutations on {1, . . ., d}.Thus, any mass function f in E d ( p) is given by f [5]s simplifies the search.The generators we find are in one-to-one relationship with the generators of E d ( p).Using the equivalence S d ( p) ≡ D d ( pd) stated in[5], a pmf in S d ( p) is a pmf on {0, . . ., d} with mean pd.Thanks to the map E in Eq. ( [5]a one-to-one correspondence between E d ( p) and S d ( p).Notice that the pmf f I of independent Bernoulli variables is exchangeable, i.e., f I ∈ E d ( p) and the map E sends f I in the Binomial distribution.Therefore, we haveE d ( p) ↔ S d ( p). (5)Fontana et al.[5]proved that the class of distributions S d ( p) coincides with the entire class of discrete distributions with mean dp, say D d (dp).This fact is useful to simplify the search of the generators of E d ( p).Therefore, the three classes E d ( p), S d ( p) and D d (dp) are essentially the same class, i.e., E d ( p) ↔ S d ( p) ≡ D d (dp) (6) Thanks to the above correspondence to find the generators of S d ( p), we can look for the generators of D d (dp).

Table 1
Number of extremal densities for each class: # R F : number of extremal densities of F d ( p) and # R E : number of extremal densities of E d ( p)

Table 2
Extremal densities of the Fréchet class F 3

Table 4
Bounds for moments of order 2, . . ., 6: m F and M F are the minimum and maximum moments for F d 1 2 and m E and M E are the minimum and maximum moments for E d

Table 5
Bounds for the VaR 0.95 -case p = 1

Table 6
Bounds for moments of order 2, . . ., 6: m F and M F are the minimum and maximum moments for F d 1 5 and m E and M E are the minimum and maximum moments for E d

Table 4
reports the bounds for moments of order 2, . . ., 6 both for the Fréchet class F d

Table 7
Bounds for the VaR 0.95 -case p = 1