On Some Properties of Information Theoretical Measures for the Study of Complex Systems

Abstract

The identification of emergent structures in dynamical systems is a major challenge in complex systems science. In particular, the formation of intermediate-level dynamical structures is of particular interest for what concerns biological as well as artificial systems. In this work, we present a set of measures aimed at identifying groups of elements that behave in a coherent and coordinated way and that loosely interact with the rest of the system (the so-called “relevant sets”). These measures are based on Shannon entropy, and they are an extension of a measure introduced for detecting clusters in biological neural networks. Even if our results are still preliminary, we have evidence for showing that our approach is able to identify and partially characterise the relevant sets in some artificial systems, and that this way is more powerful than usual measures based on statistical correlation. In this work, the two measures that contribute to the cluster index, previously adopted in the analysis of neural networks, i.e. integration and mutual information, are analysed separately in order to enhance the overall performance of the so-called dynamical cluster index. Although this latter variable already provides useful information about highly integrated subsystems, the analysis of the different parts of the index are extremely useful to better characterise the nature of the sub-systems.