Information Measures of Frequency Distributions with an Application to Labeled Graphs

Abstract

The problem of describing the distribution of labels over a set of objects is common in many domains. Cyber security, social media, and protein interactions all care about the manner in which labels are distributed among different objects. In this paper we present three interacting statistical measures on label distributions, thought of as integer partitions, inspired by entropy and information theory. Of central concern to us is how the open- versus closed-world semantics of one’s problem leads to different ways that information about the support of a distribution is accounted for. In particular, we can consider the number of labels seen in a particular data set in relation to both the number of items and the number of labels available, if known. This will lead us to consider both two alternate entropy normalizations, and a new measure specifically of support size, based not on entropy but on nonspecificity measures as used in nontraditional information theory. The entropy- and nonspecificity-based measures are related in their ability to index integer partitions within Young’s lattice. Labeled graphs are discussed as a specific case of labels distributed over a set of edges. We describe a use case in cyber security using a labeled directed multigraph of IPFLOW. Finally, we show how these measures respond when labels are updated in certain ways corresponding to particular changes of the Young’s diagram of an integer partition.