Diversity, dependence and independence

We propose a very general, unifying framework for the concepts of dependence and independence. For this purpose, we introduce the notion of diversity rank. By means of this diversity rank we identify total determination with the inability to create more diversity, and independence with the presence of maximum diversity. We show that our theory of dependence and independence covers a variety of dependence concepts, for example the seemingly unrelated concepts of linear dependence in algebra and dependence of variables in logic.


Introduction
The concepts of dependence and independence occur widely in science.The exact study of these concepts has taken place at least in four different contexts: • Mathematics: Dependence and independence are fundamental concepts in algebra: linear dependence in linear algebra and algebraic dependence in field theory.In both cases independence is defined as the lack of dependence: elements {x 1 , . . ., x n } are independent if no x i is dependent on the rest.Whitney [1] and van der Waerden [2] pointed out the similarity between these two notions of dependence and proposed axioms that cover both cases.Whitney suggested the name matroid for the general dependence structure inherent in algebra, giving rise to matroid theory, nowadays a branch of discrete mathematics.
• Computer science: Functional dependence [3] is a fundamental concept of database theory.The design and analysis of so called relational databases is often based on a careful study of the functional dependencies between attributes of various parts of the database.The more general multivalued dependencies are analogous to what we call independence relations between attributes.
• Statistics and probability theory: Dependence and independence of events (or random variables) is the basis of probability theory and statistical analysis of data.
• Logic: Dependence of a variable on another is the basic concept in quantification theory.In Dependence Logic [4] this concept is separated from quantification, making it possible, as in Independence-Friendly Logic [5], to write formulas with more complicated dependence relations between variables than what first order logic allows.Likewise Independence Logic [6] extends First Order Logic by an atom x ⊥ y that states that the tuples of quantified variables x and y are chosen independently, in the sense that every possible choice of x and of y may occur together.These logics -and the generalization of Tarski's Semantics used for their analysis, commonly called Team Semantics -have lead in the last decade to a considerable amount of research regarding logics augmented by various notions of dependence and independence, in the first order case but also in the propositional case [7], in the modal case [8,9], in the temporal case [10], and recently even in probabilistic cases [11,12,13].
In this paper, we show that these seemingly very diverse notions of dependence and independence can be captured by a simple, very general, unifying framework.
Our starting point is very general.Suppose we have a set M of objects.We want to make sense of the concept that a finite subset 1 x ⊆ f M depends on another subset y ⊆ f M , or that a subset x ⊆ f M is independent of another subset y ⊆ f M .To accomplish this in the most general sense, we define the concept of the diversity of a set x ⊆ f M .A small set has less diversity than a bigger set, hence our diversity function is monotone.Also, the diversity of x arises from properties of the individual elements, hence our diversity function satisfies certain further conditions.The connection between diversity and dependence arises from the idea that dependence reduces diversity, and respectively independence preserves diversity.If y is totally determined by x, then adding y to x does not increase the diversity of x at all.On the other hand, if x and y are independent, then putting them together means simply adding the diversities together: nothing is lost, because there is no interaction between x and y.
Because of the generality of our approach, according to which M is just a set of objects about which we a priori know nothing, we do not define the diversity function explicitly, but rather give a few conditions it ought to satisfy.The point is that on the basis of these conditions we can introduce natural notions of dependence and independence with a variety of applications, in particular the ones mentioned above.
We will now give an overview of our results.In Section 2 we introduce the concept of diversity rank x , a non-negative real number, that the whole paper is about.It is defined in a completely general setting by means of four axioms for x .We show that another general and closely related concept, that of a matroid, is a special case.Thus our approach generalizes the matroid approach.We use our diversity rank to define two binary relations, namely =(x, y) (the dependence relation "y is totally determined on x") and x ⊥ y (the independence relation "x and y are independent").Previously dependence and independence relations were studied under various guises in particular contexts such as team semantics, databases, algebraic structures, and statistics.In our approach these concepts are defined in a general setting which covers the special cases mentioned.
In Section 3 we give examples of diversity rank functions.The most interesting examples are relational diversity, algebraic diversity and entropy.We show that relational diversity does not satisfy submodularity, a property that instead matroids (such as the ones corresponding to algebraic diversity) necessarily satisfy.The entropy of a set of random variables, as we will see, also satisfies the submodularity property; however, it is not necessarily integer, and thus it does not correspond to a matroid either.
The main benefit of our approach as compared to the matroid approach is that we can develop a theory of diversity which covers matroids, the relational case, and the probabilistic (entropy) case.The relational case is important because it arises naturally in database theory, where our dependence is called functional dependence and our independence is called embedded multivalued dependence.Likewise, the entropy diversity rank function gives rise to the known probability-theoretic notions of independence and functional dependence between (sets of) random variables.
As was observed above, our diversity rank x makes it possible to define a dependence relation =(x, y).In section 4 we use the diversity rank axioms to prove the so-called Armstrong Axioms for this relation.In Theorem 1 we prove the completeness of Armstrong Axioms in our more general context.Thus the well-known completeness of Armstrong Axioms in team semantics or in the theory of database dependencies is a more general phenomenon.
In Section 5 we use diversity rank axioms to prove (a simple extension of) the Geiger-Paz-Pearl axioms for the independence relation x ⊥ y derived from a diversity rank x .In Theorem 2 we prove the completeness of these axioms in our general context.
In Section 6 we address the "reverse" question, whether every binary relation = (x, y) satisfying the Armstrong Axioms of dependence arises from a diversity rank function x .In Theorem 3 we give a positive answer in the case of countable domains.The corresponding question for the Geiger-Paz-Pearl axioms for the independence relation remains open.

Diversity rank in a general setting
We now define the concept of diversity rank in an entirely general setting.We use the notation xy to denote the union x∪y of subsets x and y of a fixed set M .The following is the key definition of this paper: Definition 1. Suppose M is an arbitrary set.A function x → x from the finite subsets of M to R + ∪ {0} is called a diversity rank function on M if it satisfies the following conditions for all x, y, z ⊆ f M : Note that the set M may be infinite, but the subsets x, y, . . . in the domain of the diversity rank function must be finite.This is intentional: in the case of relational diversity (Section 3.7), for example, we can have potentially infinitely many variables, but we are only talking about dependence/independence between finite sets of variables.
Intuitively, the diversity rank of x is the amount of "diversity" or "variation" that x contains.For example, if x is a sequence of vectors in a vector space, the amount of diversity in x is revealed by the dimension of the subspace spanned by x.If x is the set of attributes in a relation schema in a given database schema, the amount of diversity in x is revealed by the maximum number of different tuples (records) that may exist in the corresponding relation in a given database instance that can be considered as valid for that schema.Finally, if x is simply a word in a finite alphabet, a possible measure of the amount of diversity in x is the number of different letters in x, so that for example the diversity of "abbab" is 2 and the diversity of "abcdda" is 4.
It is obvious that we have to require R1 and R2.The empty set cannot manifest any diversity, more elements means more diversity, and the amount of diversity manifested by two sets taken together is at most the sum of the amounts of diversity occurring in each of them separately.
The axioms R3 and R4 are less intuitive.In brief, Axiom R3 states that if adding y to x does not increase the amount of diversity of x (that is, y is "trivial" given x), then adding it to xz does not increase the amount of diversity of xz (that is, y is also "trivial" given xz) either; and Axiom R4 states that if adding yz to x increases the amount of diversity of the maximum amount possible (that is, yz is "maximally non-trivial" given x) then adding y to x also increases the amount of diversity of the maximum amount possible (that is, y is also "maximally non-trivial" given x).
To better understand the roles of these axioms, let us briefly consider toy examples that would violate them.
For R3 let us consider the function, defined over the two-element set M = {a, b}, such that ∅ = a = b = 0 but ab = 1.According to this candidate diversity rank function, {a} is "trivial" with respect to the empty set (i.e., it adds no further diversity if added to it), but it is not so with respect to the bigger set {b}: indeed, ∅a = ∅ = 0 but ∅ab = 1 > ∅b = 0.The purpose of R3 is to prevent this kind of scenario, in which a contributes more diversity in the presence of some other element b than in its absence.
For R4, instead, let us consider a function over three elements a, b and c such that a = b = c = 1, ab = 1.5 , ac = 2 and abc = 3.Here b is not maximally diverse with respect to a, in the sense that ab < a + b ; however, b is maximally diverse with respect to ac, in the sense that abc = ac + b .This type of scenario, in some of the diversity in b would appear to be subsumed by the diversity in a but not by the diverstiy in ac, is what we aim to prevent by this rule.
R3 and R4 (as well as the right part of R2) would follow immediately if we assumed that our diversity rank function is submodular, in the sense that it satisfies the condition SUBM: xyz + z ≤ xz + yz : Proposition 1.Every function from finite subsets of some set M to non-negative real numbers satisfying R1, the left part of R2 and SUBM is a diversity rank function in the sense of Definition 1.
Proof.Let x satisfy R1, the left part of R2 and SUBM.We need to show that x also satisfies the right part of R2 as well as R3 and R4.Choosing z = ∅, we obtain immediately from submodularity that xy + ∅ ≤ x + y .But by R1 we know ∅ = 0, and subadditivity (that is, the right part of R2) follows.As for R3, suppose The converse of the above result, however, is not true: there exist diversity rank functions that do not satisfy submodularity.As a toy example, consider the diversity rank function over the set {a, b, c} defined as Here abc + c = 3 + 1 = 4, but ac + bc = 1.6 + 1.6 = 3.2; therefore, SUBM fails.
Of course, one needs to verify that this is indeed a diversity rank function.R1 holds, because ∅ is indeed 0; R3 holds, because its premise is never satisfied non-trivially (it is never the case that xy = x for y = ∅); R4 likewise holds because its premise is never satisfied non-trivially; the left side of R2 is easily verified by inspection; and as for the right part of R2 we need to verify the following six cases: (The other possible cases follow from these because of the left side of rule R2, e.g. ab + ac ≥ ab + c ≥ abc ).
It could appear tempting at this point of the work to exclude counterexamples such as this one from consideration by adding submodularity to the definition of diversity rank function.But as Proposition 3 will show, there exist diversity rank functions of practical interest that fail to satisfy SUBM.
A direct consequence of Proposition 1 is that our diversity rank functions generalize matroids: Definition 2 (Matroid).A matroid r over some finite set E is a function from subsets of E to non-negative integers satisfying the following conditions for all subsets x and y of E.
The above is just one of several equivalent definitions used in the literature.It is equally possible to define a matroid in terms of its independent sets (that is, the x such that r(x) = |x|), in terms of bases (maximal independent sets), in terms of circuits (minimal non-independent sets), or in terms of closure operations.All these definitions can be shown to give rise to the same class of mathematical objects: we refer the reader to [14] for more details.
Corollary 2. Let r be a matroid over some finite set E. Then r is a diversity rank function over E.
Proof.R1 holds for r.Indeed, by M1, r(∅) ≤ |∅| = 0, and therefore the only possibility is that r(∅) = 0.The left part of R2 holds immediately because of M3 (this can be shown by easy induction on the the number of elements in y\x).Moreover, SUBM also holds of r.Indeed, since xyz = xzyz, we have by M2, r(xyz) + r(xz ∩ yz) ≤ r(xz) + r(yz).But z ⊆ xz ∩ yz, and so by the left part of R2 (which we already proved), r(z) ≤ r(xz ∩ yz) and hence r(xyz) + r(z) ≤ r(xz) + r(yz), as required.The conclusion then follows, since because of Proposition 1 the right part of R2 as well as R3 and R4 are also true of r.
Given a notion of diversity, it is easy to define dependence and independence in terms of minimal and maximal diversity contributions: Definition 3. Suppose M is a set and • a diversity rank function on M .We can now define dependence relations between finite subsets of M (with respect to • ) as follows: • Dependence: y is totally determined by (or depends on) x, in symbols =(x, y), if and only if xy = x .
• Constancy: x is constant, in symbols =(x), if and only if x = 0.
• Independence: x and y are independent, in symbols x ⊥ y, if and only if x + y = xy .
The idea is that =(x, y) holds under a diversity rank function if the amount of diversity inherent in x in terms of the rank function does not increase when y is added.Simply put, x determines y, so no new diversity occurs.=(x), on the other hand, holds if x has no diversity at all; and x ⊥ y holds if the diversity inherent in x is so unrelated to the diversity inherent in y that when the two are put together into xy, the diversity is the sum of the diversity of x and the diversity of y: no loss of diversity occurs because there is-intuitively-no connection between x and y.

Examples
Let us now consider some examples of our definitions, in order to get a better feel of their applicability and consequences.As we will see, aside from the example of Subsection 3.7 (relational diversity), all our examples also satisfy the submodularity condition.Nonetheless, it is worth emphasizing that, when the ranks are not necessarily integer, these diversity rank functions are not matroids.
There are various diversity ranks, also called diversity measures or diversity indexes, in biology, such as Shannon index, Simpson index, Renyi index, α-diversity, βdiversity or γ-diversity, each attempting to describe in terms of a single real number the number of different species in an area, the number of individuals of each species, and other parameters relevant in characterizing diversity of a biological system [15,16,17].Similar diversity ranks have been introduced in other areas, as well, and usually take the concept of entropy (see subsection 3.6 below) as a starting point.

Constant diversity
One extreme case is the constant rank for which ∅ = 0 and, for some number c, x = c for all x ⊆ f M with x = ∅.If c = 0, there is no diversity: every set depends on every other set and is also independent of every other set.If c = 0, then every set y is still dependent on any non-empty set x, because xy = c = x , and every set x is still independent from the empty set ∅, because x∅ = x = x + ∅ ; but two non-empty sets x and y are not independent, because Constant diversity is trivially submodular: if z is empty then xyz + z = 2c = xz + yz , and if instead z is empty then submodularity reduces to xy ≤ x + y (which is part of rule R2, and which is also easily verified for this diversity function).

Singular diversity
Let a 0 ∈ M be fixed.Let In this case a selected element a 0 is the only source of "diversity" there is.A set has diversity 1 if and only if it contains a 0 .In this case y depends on x if and two sets x and y are independent if at most one of them contains a 0 .So dependence reduces in this case to implication and independence to the Sheffer stroke (also known as NAND).
Again, this diversity rank function is submodular: Note that the notion of dependence arising from singular diversity is not symmetric: in particular, the empty set depends on {a 0 } but {a 0 } does not depend on the empty set.

Two-valued diversity
Suppose {a} is either 0 or 1 for all a ∈ M .Then, by Rule R2, it follows that x = max{ {a} : a ∈ x} for all x ⊆ f M .Hence, it suffices in this case to declare which elements have diversity 1; a set has diversity 1 if and only one if it contains one of those elements. 4n this case diversity is an on/off phenomenon, either it exists (1) or it does not (0), and a set has diversity if it includes some singleton that has it.In an extreme case {a} = 0 for all singletons {a}, a ∈ M , and we have zero constant diversity: every set has diversity 0. In another extreme case {a} = 1 for all singletons {a}, a ∈ M , and we are again in constant diversity: every non-empty set has diversity 1.
According to this diversity notion, =(x, y) if and only if ∃a ∈ x such that {a} = 1 ⇒ ∃b ∈ y such that {b} = 1; and x is independent from y if and only if at most one of x and y contain an element c with {c} = 1.
Submodularity is, once more, straightforwardly verified.xyz + z is 2 if and only if z = 1, and in that case it is easy to see that xz + yz is also Remark: Uniform diversity shows that if M has at least three elements a, b, c then independence is not necessarily equivalent to the failure of dependence both ways.Indeed, {a, b} is not dependent on {b, c} or vice versa, since neither set is contained in the other, but {a, b} and {b, c} are not independent either since they are not disjoint.This is not surprising: in our framework, y depends on x if adding y to x contributes no diversity whatsoever to it, while x and y are independent if adding y to x contributes the maximal amount y of diversity to it.But it is certainly possible for neither extreme to be the case.

Coverage diversity
Suppose U is a finite set and choose We can think of each A a as "data", about the element a of M .The more data we have the more diversity we give to the element, and the diversity of a set is obtained by simply putting together all the data we have.In this simple example the data is not thought to be specific to the elements of M , so the data about different elements is just lumped together.For example, if a and b are two botanic genera in the bean family, the diversity of {a, b} in a set U of data about species (e.g. in some location) is obtained by counting how many different species of the two genera there are in U .
According to this diversity notion, y is dependent on x if and only if {A a : a ∈ y} ⊆ {A b : b ∈ x}, that is, every data point corresponding to some element of y also corresponds to some element of x; and y is independent from x if and only if that is, if no data point corresponds to some element of x and to some element of y.

Submodularity can be verified much as in the previous
and it is easy to see that |C| ≥ z .

Entropy
Let us think of the elements of M as discrete random variables v 1 , v 2 , . . .over some probability space and with outcomes in some finite set A. 5 Then for any x = {v 1 . . .v k } ⊆ f M we can define x as the joint entropy [18] This definition clearly satisfies rule R0, since the entropy of the only possible distribution over the empty space is zero; moreover, it is not hard to convince oneself that it is monotone and submodular.In brief, this can be shown by considering the conditional entropy Indeed, it can be proved (see any Information Theory textbook, for instance Theorem 2.2.1 of [19]) that the conditional entropy H(y | x) is always non-negative 7 , from which we have the left part of R2; furthermore (see e.g.Theorem 2.6.5 of [19] From Proposition 1, we can immediately conclude that entropy is an example of a (submodular) diversity rank function.Here, y depends on x according to the entropy diversity rank if and only if H(xy) = H(x), that is, if and only if the relative entropy of y given x is 0, or in other words if the value of y is completely determined by the value of x; and x and y are independent according to this rank if and only if they are independent sets of random variables, that is, P (x = m, y = m ) = P (x = m)P (y = m ) for all possible choices of values a and b for x and y.We observe that this notion of probabilistic independence of random variables is exactly the one axiomatized by Geiger, Paz and Pearl in [20].Their axiomatization will be the base for our axiomatization of independence in our more general setting in Section 5.

Relational diversity
Suppose X is a nonempty, finite set of variable assignments s from a set V of variables to a set A of elements (in the language of Dependence and Independence Logic, X is said to be a team over A with domain V ; and it is not hard to see that it is equivalent to a relational table in which every variable indicates a different column). 9Given some is the number of different values that x = v 1 . . .v n takes in X. 10We can think of each s ∈ X as an "observation", or "data", about the possible values that the variables in V can take.The more different observations we have the more diversity we give to the element.Note the difference with coverage diversity, where the data was not specific to the element of A.Here what matters is the relationships of the different observations to each other.Thus that is, the diversity rank of a single element v of V is the (logarithm of the) number of different observations about v.The diversity of a pair {v, w} is the (logarithm of the) number of different combinations of observations of v and w.For example, if v and w are two genera, the diversity of {v, w} in a set X of observations is calculated by counting how many different pairs of observations of a specimen of v and a specimen of w there are in X.
The presence of the logarithm operator in this definition may appear at first sight somewhat outlandish, but it is in fact quite natural.Indeed, one may recall that, in information theory, the information content of an event is defined as the negative logarithm of its probability, and the information content of a random variable (i.e. its entropy) is the expectation of the information content of it taking all possible values [18,21].Therefore, if we associate a relation R with the probability distribution selecting any tuple in R with equal probability 1/|R|, the information content of this distribution (and, hence, of the relation) is precisely − log(1/|R|) = log(|R|).
The dependence relation arising from the relational diversity rank is the usual functional dependence relation of database theory and dependence logic [3,22,4].Why?By definition, =(x, y) if and only if log(#rows X (xy)) = log(#rows X (x)), that is, if and only if #rows X (xy) = #rows X (x).This can be the case if and only if any two s, s ∈ X which differ with respect to xy differ already on x alone, or, by contraposition, if and only if any two s, s ∈ X which are the same with respect to x are also the same with respect to y.This is precisely the usual notion of database-theoretic functional dependence [3], which is arguably the most important (although by no means the only) notion of dependence studied in the context of database theory.
The independence relation arising from the relational diversity rank is also the independence relation of Independence Logic [6].Indeed, by definition, x ⊥ y if and only if #rows X (xy) = #rows X (x) • #rows X (y).Then, enumerating the elements of x as (v 1 . . .v k ) and the elements of y as (w 1 . . .w t ), it is always the case that #rows • #rows X (y), since every possible value for (v 1 . . .v k , w 1 . . .w t ) in X corresponds to one possible value for (v 1 . . .v k ) and one possible value for (w 1 . . .w k ) in X. Equality holds if and only if the converse also holds, i.e., if and only if for any two s, s ∈ X there exists some s ∈ X such that s (v 1 . . .v k ) = s(v 1 . . .v k ) and s (w 1 . . .w t ) = s (w 1 . . .w t ).In particular, this implies at once that if x ⊥ y the variables occurring in both x and y must take only one value in X, and that if x and y are disjoint and x ⊥ y then the projection of X onto xy must be the Cartesian product of its projections onto x and y. 11It may be instructive to verify that the relational diversity notion of rank satisfies our axioms: R1: Since #rows(∅) = |{()}| = 1 for any choice of X, where () represents the empty tuple, we have ∅ = 0, as required.In contrast to our other examples, this notion of relational diversity is not submodular, as the following counterexample, which we owe to Tong Wang 12 shows: Proposition 3. Relational diversity fails to satisfy SUBM.

Algebraic diversity
Suppose that V is a vector space and that h maps M into V .We obtain a diversity rank function by letting for finite x ⊆ M : 13x = the dimension of the subspace generated by {h(a) : a ∈ x}.
Submodularity SUBM follows from the well known fact that if U and V are vector subspaces, In this context it is not hard to verify that V is dependent on U if and only if dim(U ∪ V ) = dim(U ), that is, if and only if every vector of V is a linear combination of vectors in U ; and that, on the other hand, U and V are independent if and only if dim(U ∪ V ) = dim(U ) + dim(V ), that is, if and only if the subspace generated by U and the subspace generated by V do not share a nonzero vector.
Likewise if F is a field, we get a diversity rank function by letting for finite x ⊆ M and letting h map M into F instead: x = the transcendence degree of the subfield generated by {h(a) : a ∈ x}.
This gives rise to the concepts of algebraic dependence and independence.
As mentioned in the Introduction, this notion of rank defines a matroid (in fact, it was one of the original motivations for the development of Matroid Theory); and thus, by Corollary 2, it is also a diversity rank function according to our definition.

From diversity to dependence
Given a diversity function • , we have defined dependence =(x, y) of y on x by xy = x .
It is easy to verify that dependence satisfies the following axioms: Proposition 4. Dependence satisfies the following properties: 1. Reflexivity: =(xy, x).
2. Augmentation: =(x, y) implies =(xz, yz).We can use the above rules as the axioms of a proof system for inferring the consequences of a set of dependence assertions.More precisely, given a set Σ of dependence assertions and a dependence assertion = (x, y) for x, y ⊆ f M , we will write that Σ D =(x, y) if it is possible to derive =(x, y) from Σ through applications of the rules of Reflexivity, Augmentation and Transitivity.
It follows from these axioms that a dependency notion is entirely defined even if we only consider singletons on the right-hand side of it: Proposition 5. Let Σ be a set of assertions of the form =(z, w) for z, w ⊆ f M , and let also x, y ⊆ f M .Then Σ D =(x, y) if and only if Σ D =(x, {a}) for all a ∈ y.
Proof.By Reflexivity, if a ∈ y then it is always the case that Σ D =(y, {a}).If Σ D =(x, y), by Transitivity it is thus the case that Σ D =(x, {a}) for all such a.Conversely, suppose Σ D =(x, {a}) for all a ∈ y.Then, in order to reach our conclusion that Σ D =(x, y), it suffices to verify that whenever Σ D =(x, y 1 ) and Σ D =(x, y 2 ) it is also the case that Σ D =(x, y 1 y 2 ).This is easily shown: if The following is essentially proved in [3], albeit in the special case of relations and functional dependencies: Theorem 6 (Completeness of the Dependence Axioms).Let Σ be a set of assertions of the form =(z, w), where all z and w are finite subsets of some set M and let also x, y ⊆ f M .The following properties are equivalent: Let W be all the remaining elements of M .Since y ⊆ V , W = ∅.Let us define a diversity rank function on M by letting for a ∈ M : Let us then assume (3).We proceed as above.Let X consist of the two functions {s 1 , s 2 }, where s 1 (a) = 0 for all a ∈ M , s 2 (a) = 0 for a ∈ V and s 2 (a) = 1 for a ∈ W .We obtain the same rank as above, so we are done.
Since -as we saw -functional dependence is exactly the dependency notion generated by the relational diversity rank function, we obtain: Corollary 7 (Armstrong).A functional dependence follows semantically, for all relations, from a given set of functional dependencies if and only if it follows by the rules of Proposition 4.
Theorem 6 shows that Armstrong's completeness theorem for functional dependence is actually a more general completeness theorem of dependence relations arising from diversity ranks.

From diversity to independence
We shall now study the properties of the notions of independence arising from our diversity ranks.Let us recall that, according to our definition, x and y are independent (x ⊥ y) if and only if xy = x + y .
Let us begin by observing that, by our definition, x ⊥ x if and only if x = x + x , that is, if and only if x = 0: x is independent of itself if and only if x contains no diversity whatsoever according to our diversity rank function, that is, if and only if x is constant in the sense of Definition 3.
Note that in the probabilistic case, a random variable v is independent of itself, in the sense that P (v = a and v = b) = P (v = a)P (v = b) for all possible values a, b of X, if and only if v may take only one value with probability 1, and hence it is constant in the ordinary sense of the word; and similarly, in the logical/relational case a variable v is independent on itself if and only if it takes only possible value for all variable assignments in the set being considered.Thus, our use of the term "constancy" is not unmotivated.

Symmetry:
3. Decomposition: If x ⊥ yz, then x ⊥ y. 144.Mixing: If x ⊥ y and xy ⊥ z, then x ⊥ yz. 15roof.Let us prove that these axioms follow from our notion of independence: Given a set Σ of independence assertions and an independence assertion x ⊥ y for x, y ⊆ f M , we will write that Σ I x ⊥ y if it is possible to derive x ⊥ y from Σ through applications of the rules of Empty Set, Symmetry, Decomposition, Mixing and Constancy.

Constancy:
The following derived rule will be useful: In [20], a sound and complete axiomatization for independence of tuples of random variables (as derived from the definition of entropy in Section 3.6) was found, with the additional requirement that the left-and right-hand sides of the independence assertion are disjoint.
Theorem 11 below is a generalization of that result to the case of general diversity rank functions, and without that additional requirement.
First, we will show that that the axioms given above are complete for assertions of the form {a} ⊥ {a}: Lemma 10 (Completeness of Independence Axioms wrt Constancy Assertions).Given a set M , let Σ be a set of independence assertions over M , and let a ∈ M .Then the following properties are equivalent: 1. {a} ⊥ {a} holds under every diversity rank function on M under which Σ holds.
2. {a} ⊥ {a} holds under every relational diversity rank function under which Σ holds.
3. {a} ⊥ {a} follows from Σ by the rules of Proposition 8.
Proof.Trivially (1) implies ( 2) and ( 3) implies (1).Let us verify that (2) implies (3).Suppose that {a} ⊥ {a} does not follow from Σ by the above rules.Then let V contain all b ∈ M such that Σ I {b} ⊥ {b} and let S be a team with domain M over {0, 1} (that is, a set of functions from M to {0, 1}) that contains all s : M → {0, 1} such that s(b) = 0 for all b ∈ V .Now let • = log(#rows S (•)) be the relational diversity rank function induced by S: as already discussed, such a diversity rank function satisfies an independence assertion z ⊥ w if and only if any possible values of z and w in S may occur together, or, in other words, if and only if for all s, s ∈ S there exists some s ∈ S that agrees with s on z and with s on w.In particular, for the S given, this means that z ⊥ w is satisfied if and only if z ∩ w ⊆ V .Thus, S does not satisfy {a} ⊥ {a}, since by assumption a ∈ V .
On the other hand, S satisfies all assertions of Σ.Indeed, consider any z ⊥ w ∈ Σ.By Decomposition and Symmetry, every element of c ∈ M which is in both z and w is such that Σ I {c} ⊥ {c}, that is, such that c ∈ V .Thus, z ∩ w ⊆ V and therefore z ⊥ w is satisfied by (the relational diversity rank function corresponding to) S, as required.
In conclusion, from the assumption that {a} ⊥ {a} does not follow from Σ according to the rules we were able to find a relational diversity rank function that satisfies Σ but not {a} ⊥ {a}.Thus (2) implies (3), and this concludes the proof.Now we can generalize the completeness result of Lemma 10 to arbitrary independence assertions.The proof is an adaptation of the proof of [20] to our framework.
Theorem 11 (Completeness of the Independence Axioms).Let M be a set and let Σ be a set of independence assertions z ⊥ w for z, w ⊆ f M .Then the following conditions are equivalent for any x, y ⊆ f M : Thus, we saw that the diversity rank function induced by S does not satisfy x ⊥ y but satisfies all independence statements of Σ, as required.
Corollary 12 ([20]).An independence assertion follows semantically, in all databases, from a given set of independence assertions if and only if it follows by the rules of Proposition 8.

At this point it would be natural to ask the following
Open Problem: What are the rules that govern the interaction between dependence and independence in our framework?
In general, inference problems for dependence/independence assertions are not necessarily decidable in a relational setting ([24, 25, 26]); but in the setting of general diversity rank functions, the decision problem for sets of dependence and independence assertions is necessarily decidable.Indeed, the axioms of diversity rank functions as well as dependence and independence assertions may be translated into the first-order arithmetic of the reals, replacing each x expression with a variable r x that ranges over non-negative real numbers and -for example -writing a dependence atom =(x, y) as r xy = r x , an independence atom x ⊥ y as r xy = r x + r y , and Axiom R3 as an axiom schema of the form (r xy = r x ) → (r xyz = r xz ).But the arithmetic of the reals is decidable [27], and therefore it is decidable whether some dependence or independence assertion follows from a set Σ of dependence and independence assertions in our setting.
We leave the problem of searching for an axiom system for dependence and independence assertions combined to future work.Here we only point out two simple axioms that govern the interaction between dependence and independence in our setting: • Constancy Equivalence: x ⊥ x if and only if =(∅, x);

A Representation Theorem for Dependence Atoms
As we saw, every dependence notion induced by a diversity rank function satisfies Armstrong's Axioms, which furthermore are complete for diversity rank functions.However, a question that remains open is whether every dependency notion that satisfies Armstrong's Axioms is induced by a diversity rank function.More formally, let M be a set, and let Σ be a set of dependence assertions over M which is closed under Armstrong's Axioms.Is there a diversity rank function • for which Σ = {=(x, y) : x, y ⊆ f M, xy = x }? 16An example may be helpful here.Suppose that M = {a, b, c} and Σ is the closure under Armstrong's Axioms of {=(ab, c)}, i.e.Σ = {=(x, y) : x, y ⊆ M, y ⊆ x} ∪ {=(x, y) : ab ⊆ x and y ⊆ M }.
It is not hard to verify that this set is indeed closed under Reflexivity, Augmentation and Transitivity; but can we find a diversity rank function over {a, b, c} such that, for x, y ⊆ {a, b, c}, y depends on x according to it if and only if =(x, y) ∈ Σ?
By Theorem 13 below, such a diversity function does exist.
Theorem 13.Let M be a countable set and let Σ be a set of dependency assertions of the form =(x, y) (for x, y ⊆ f M ) that is closed under Armstrong's Axioms.Then there exists a diversity rank function • over M such that Σ • = Σ.
Proof.We first define the relation ≡ on the finite subsets of M , as follows.For x, y ⊆ f M , x ≡ y if both =(x, y) ∈ Σ and =(y, x) ∈ Σ.Using that Σ satisfies Armstrong's axioms, it is straightforwardly seen that ≡ is an equivalence relation.Next, we observe that the set of all finite subsets of M is countable, because M is.Hence E is also countable.Let E = {E 0 , E 1 , E 2 , E 3 , ...} be an enumeration of E with E 0 = E ∅ .We now construct an order-preserving injection f : E → {0}∪]1, 2[ recursively as follows: 2. Assume that, for i = 0, ..., n, f (E i ) has been defined.Let Notice that, for all x ⊆ f M , E 0 = E ∅ ≤ E x , since =(x, ∅) ∈ Σ by Reflexivity.Hence, the construction of f guarantees that it is indeed order-preserving.
Finally, we define a diversity rank function • on M as follows: for x ⊆ f M , x = f (E x ).We prove that • satisfies R1-R4.As an example, let us apply the procedure described here to M = {a, b, c} and to the set Σ we mentioned before, i.e.Σ = {=(x, y) : x, y ⊆ M, y ⊆ x} ∪ {=(x, y) : ab ⊆ x and y ⊆ M }.
It is readily seen that E ab = E abc = {ab, abc}.For all other x ⊆ f M , E x = {x}.Let us now consider the following enumeration of the finite subsets of We now construct the function f in the proof of Theorem 3 recursively according to this enumeration: 1. f (∅) = 0, by construction.
2. Since E a , E b , and E c are mutually incomparable, we may assign to them arbitrary mutually different values between 1 and 2, e.g., f (E a ) = 1.5, f (E b ) = 1.6, and f (E c ) = 1.1.
3. Since, for every x ⊆ f M, =(ab, x) ∈ Σ, we have in particular that E a < E ab , E b < E ab , and E c < E ab .Therefore, we must assign to E ab a value between 1.6 and 2, say, f (E ab ) = 1.8.∈ Σ and =(ac, bc) / ∈ Σ, E ac and E bc are incomparable.Therefore, we must assign to E bc an unused value between 1.6 and 1.8, say, f (E bc ) = 1.65.
According to the proof of Theorem 13, Σ = {=(x, y) : x, y ⊆ f M, xy = x }, as can be easily verified.It is also easily verified that this is indeed a diversity rank function.
We leave for future work the question whether this representation theorem may be generalized to independence atoms, that is to say, whether every set of dependence and independence assertions satisfying the axioms for dependence, the ones for independence, and some additional axioms for dependence/independence interactions arises from some diversity rank function.

Conclusions
In this work, we showed how many distinct notions of dependence and independence, originating in different branches of mathematics and computer science, may be treated as instances of the same framework: one which can be seen as a generalization of matroid theory allowing for non-integer ranks and weakening the submodularity condition.In this framework, y is said to be dependent from x if adding it to x does not increase the amount of diversity, while y is said to be independent from x if adding it to x increases maximally the amount of diversity.
Despite its generality, this framework is nonetheless powerful enough to prove nontrivial results -including, in particular, completeness theorems for the corresponding dependence and independence notions.These results generalize to our entire setting the completeness theorems by Armstrong and by Geiger-Paz-Pearl for database-theoretic functional dependence and for probabilistic independence respectively.In addition, we have obtained a representation theorem showing that every set of dependence assertions that satisfies Armstrong's Axioms arises from some diversity rank function.
One natural next step would be to investigate further the properties of this formalism, in particular with respect to the interaction between independence and dependence assertions.Combinatorial properties of this system would also be worth investigating, as would the study of possible operations that combine different diversity rank functions.This could also contribute to the logical study of notions of dependence and independence in the context of Team Semantics, in particular providing a unifying approach for its different variants (e.g.probabilistic, modal, propositional, . . .).
Another important next step would be to extend our representation result to dependence and independence assertions combined, as well as investigate the connections between our approach and other approaches, such as the study of measure-based constraints in the context of Database Theory [28], and the lattice-theoretic study of conditional independence of [29].
Finally, it would be interesting to investigate potential applications of our framework.Our approach in this work has been one of synthesis, motivated by the search of a framework that captures a wide array of notions of dependence and independence studied in different areas.Such a framework had to be more general than matroids, which -despite their success and undeniable importance -are limited by their commitment to submodularity and to integer values; had to be formally simple and widely applicable; and it had to be specific enough to highlight true commonalities and analogies between notions of dependence and independence developed in different areas.We think that our notion of diversity rank function meets these objectives, and that this has the potential to lead to interesting connections and fruitful cross-topic fertilization between the study of notions of dependence and independence in these different areas.
xy = x .Then by SUBM, xyz ≤ xy + xz − x = x + xz − x = xz ; but on the other hand xz ≤ xyz by the left part of R2 and so xyz = xz as required.Finally, R4 holds.Indeed, by SUBM we know xyz + y ≤ xy + yz .Thus, if xyz = x + yz we have immediately that x + yz + y ≤ xy + yz , that is, x + y ≤ xy .But xy ≤ x + y by the right part of R2, which we already proved, and hence xy = x + y as required.

R2:
Since #rows(x) ≤ #rows(xy) and the logarithm is a monotone function, we have immediately that x ≤ xy ; and since #rows(xy) ≤ #rows(x) • #rows(y), we have immediately that xy ≤ x + y .R3: If xy = x , #rows(xy) = #rows(x) and hence every possible value of x occurs together with only one possible value of y.But then every possible value of xz occurs together with only one possible value of y, and hence #rows(xyz) = #rows(xz) and xyz = xz ; R4: If xyz = x + yz , it must be the case that #rows(xyz) = #rows(x) • #rows(yz), and hence that every possible value of x occurs together with every possible value for yz.But then in particular every possible value for x occurs together with every possible value for y, and so #rows(xy) = #rows(x) • #rows(y) and xy = x + y .
y 1 ), by Augmentation we have Σ D =(x, xy 1 ) (remember that in our notation xx = x ∪ x = x), and if Σ D =(x, y 2 ) again by Augmentation we have Σ D =(xy 1 , y 1 y 2 ), and an application of Transitivity gives us Σ D =(x, y 1 y 2 ).The conclusion follows immediately.

Symmetry:
This follows easily from the commutativity of sum and union.If x + y = xy then y + x = xy = yx .Decomposition: Suppose x ⊥ yz, that is, x + yz = xyz .By R4, we then have xy = x + y and x ⊥ y.Mixing: Suppose xy = x + y and xyz = xy + z .We need to prove xyz = x + yz .We begin by observing that x + y + z = xy + z = xyz .But by R2 yz ≤ y + z , and therefore x + yz ≤ x + y + = xyz .On the other hand, again by R2, xyz ≤ x + yz , and so in conclusion xyz = x + yz , as required.Constancy: If z ⊥ z then z = z + z , and hence z = 0.But then by R2 x ≤ xz ≤ x + z = x , and thus xz = x + z and z ⊥ x.

Proposition 9 (
Constancy Augmentation).Given a set M , let Σ be a set of independence assertions over M of the form z ⊥ w for z, w ⊆ f M , and suppose Σ I u ⊥ u and Σ I x ⊥ y.Then Σ I xu ⊥ y Proof.By Constancy, if Σ I u ⊥ u then Σ I u ⊥ xy, and so by Symmetry Σ I xy ⊥ u.If furthermore Σ I x ⊥ y, by Symmetry Σ I y ⊥ x; and thus, by Mixing, Σ I y ⊥ xu, and by Symmetry once more Σ I xu ⊥ y as required.

•
Propagation: If x ⊥ y and =(y, z) then x ⊥ yz.Both of these can be shown to follow easily from our notion of rank: • Constancy Equivalence: Suppose x ⊥ x.Then, by definition, x + x = xx .But on the other hand, xx = x and therefore, x = 0 and x = ∅x = ∅ = 0. Conversely, suppose =(∅, x).Then x = ∅x = ∅ = 0, and therefore x + x = 0 = xx .• Propagation: Suppose that x + y = xy and that yz = y .From the second hypothesis, by R3, we can show that xy = xyz and therefore in the first hypothesis we can replace y with yz and xy with xyz , thus obtaining x + yz = xyz .Therefore x ⊥ yz, as required.

5 .
Since =(bc, b) ∈ Σ, =(bc, c) ∈ Σ, and =(ab, bc) ∈ Σ, E b < E bc , E c < E bc , and E bc < E ab .Since =(bc, a) / ∈ Σ and =(a, bc) / ∈ Σ, E a and E bc are incomparable.Since =(bc, ac) / in which case at least one of x and y is 1 and hence xz + yz ≥ 1; and finally, if xyz + z = 0 then it is trivially true that xyz + z ≥ xz + yz .This is the choice of taking the cardinality of the (finite) set x ⊆ f M as the measure of its diversity.Dependence means inclusion: y is totally determined by x if and only if |xy| = |x|, that is, if and only if y ⊆ x.Independence is disjointness: x and y are independent if and only if |xy| = |x| + |y|, that is, if and only if x ∩ y = ∅.Once more, submodularity is easily verified by observing that |xyz| = |xz| + |yz| − |(xz) ∩ (yz)| and that |(xz) ∩ (yz)| > |z|.