Association, agreement, and equity

Abstract

The paper attempts to make a clear distinction between three broad families of statistical indices: association, agreement, and what one may call equity.

The need for this distinction arises in social research, for example, where reliability (accuracy, reproducibility, and stability) is assessed by measures of association rather than agreement. In this application, the assumptions built into an association measure conflict with the reality that gives rise to reliability data. A second motivation for this distinction is that association measures tend to express chance as the product of two potentially very different frequency distributions, agreement as the product of two identical distributions, and equity ignores such distributions altogether. A third motivation for this distinction is that the probability distribution of such measures does not depend on whether they are linear or non-linear, symmetrical or asymmetrical, or whether they express predictability or the extremality of a frequency distribution, but on their family membership.

Notions of association, agreement, and equity have inherently nothing to do with the (nominal, ordinal, interval, and ratio) ordering in data. The 2-by-2 case is therefore chosen as the basis of the proposed distinction. All statistical indices, whether they are designed to characterise multivariate data or to identify complex orderings, ought to be applicable to this most reduced case of two variables, making one distinction in each. To test a coefficient's membership in one of the three families, nothing more complex is needed.