# Structure versus function: a topological perspective on immune networks

## Abstract

Many recent advances have been made in understanding the functional implications of the global topological properties of biological networks through the application of complex network theory, particularly in the area of small-world and scale-free topologies. Computational studies which attempt to understand the structure–function relationship usually proceed by defining a representation of cells and an affinity measure to describe their interactions. We show that this necessarily restricts the topology of the networks that can arise—furthermore, we show that although simple topologies can be produced via representation and affinity measures common in the literature, it is unclear how to select measures which result in complex topologies, for example, exhibiting scale-free functionality. In this paper, we introduce the concept of the *potential network* as a method in which abstract network topologies can be directly studied, bypassing any definition of shape-space and affinity function. We illustrate the benefit of the approach by studying the evolution of idiotypic networks on a selection of scale-free and regular topologies, finding that a key immunological property—tolerance—is promoted by bi-partite and heterogeneous topologies. The approach, however, is applicable to the study of any network and thus has implications for both immunology and artificial immune systems.

### Keywords

Topology Complex networks Artificial immune systems Tolerance## 1 Introduction

In recent years, it has been experimentally observed that real-world biological, social and technological networks are not structured in a random way (Albert and Barabási 2002). Instead, most of these networks are organized in such a way that a few nodes are able to interact with many others, whereas many others only interact with a few. The extreme case is often referred to as a scale-free network in which the degree distribution follows a power-law (Barabási and Albert 1999). However, other configurations showing lower levels of heterogeneity are also common (Amaral et al. 2000). Dissections of real world networks have produced evidence for single-scale networks, characterized by a fast Gaussian decaying tail in the degree distribution, broad-scale distributions, defined by a power-law with an abrupt truncation for large connectivities (Tanaka et al. 2005) and, finally, the previously referred to scale-free class. The ubiquity of such classes of networks raises many questions of which one is on the origin of these topological properties. Moreover, in the context of understanding complex social and biological phenomena, it is useful, and in many cases necessary, to understand the topology of the underlying networks of interactions (Albert and Barabási 2002; Barabási and Oltvai 2004 ; Dorogotsev and Mendes 2003 ).

The global properties of a system rely extensively upon the underlying network and different dynamical outcomes emerge from different topologies (Guimerá and Amaral 2005; May and Lloyd 2001; Pastor-Satorras and Vespignani 2001; Santos et al. 2006). This relationship between the *structure* and the *function* of a network is attracting much interest today, and the study of it has been greatly facilitated by the development of complex network theory and formalisms. Many studies can be found in the immunological literature which develop this theme. For example, scale-free topologies have been postulated to explain the robustness of biological networks to mutation and environmental stress (Barabási and Oltvai 2004). In particular, they are shown to be resistant to random failure but vulnerable to attack at their most connected points (the ‘hubs’ of the network.) The ability to identify high-degree proteins in a network might lead to new strategies for therapeutic mediation of signalling pathways in cancer. Jeong et al. link topological centrality to functional essentiality in (Jeong et al. 2001). Frankenstein et al. (2006) discuss the topological implications of cytokine networks. Tieri et al. (2004) note that immune function arises as a result of interactions between a number of cell types each playing a distinctive role (e.g. cytokines, chemokines and hormones) but that a quantitative analysis of the topological properties of an immunological network involving this complex interchange of mediators among immune cells is still lacking, and attempts to address this problem.

The relationship between structure and function is also central to any future success of AIS. In the past, a rather ad-hoc approach to trying to achieve *function* has been taken, paying little regard to the type of network *structure* that might facilitate such function. Recent theoretical work has attempted to untangle the relationship between the function of idiotypic networks and the affinity measure used within the network, e.g. (Dilger and Strangeld 2006; Hart 2005; Hart and Ross 2005), but this has yet to filter directly through to any practical application (although Dilger and Strangeld (2006) make some suggestions).

^{1}Depending on the type of immune algorithm being studied, a set of immune-inspired dynamics and meta-dynamics are applied; at any subsequent moment in time, a set of cells exists as shown on the right (B). The existing cells are a subset of all possible cells in the network defined by A; some of the existing cells have affinity with other cells (shown by solid lines); others may have been just introduced by the meta-dynamics and as yet, have no affinity with other cells. Network B can be analysed using typical complex network theory tools to determine its topology and properties and then an attempt made to relate its structure to any observed functionality. This can be repeated with a number of shape-spaces and affinity measures, and finally an attempt made to generalise from the results to map observed topologies to function. Alternatively, a shape-space and affinity measure can be carefully selected which define an underlying network with topology \({\mathcal{T}}\) with the desired topological properties, and then some dynamics applied. This can be straightforward in some cases (some examples are discussed in later sections in this article), but more often is incredibly difficult—for example, what shape-space and affinity function lead to an underlying scale-free network?

In an attempt to address this difficulty, this paper introduces a novel methodology for systematically studying the relationship between topology and function, by directly studying the underlying network \({\mathcal{T}}\) as a proxy for a shape-space and affinity measure. A wealth of results from the complex network literature provides us with mechanisms for generating any desired topology of network. By studying the dynamics of evolution of a set of immune cells on a fixed underlying structure whose properties are known, we have a direct mechanism for tackling a question of interest to both immunologists and engineers: *what kind of topologies facilitate the emergence of specific functionalities?* We illustrate the effectiveness of the approach by presenting results obtained using a model of immune dynamics inspired by idiotypic network theory applied to various instances of regular and scale-free underlying networks.

This paper draws together previously published results regarding regular network topologies (Hart et al. 2007a, b), providing a more detailed interpretation of the relevance of previous results. Despite the use of an idiotypic model throughout, we stress however, that the approach presented is not restricted in any sense to idiotypic network models. The results are of interest to both AIS practitioners designing systems and to computational immunologists studying the functionality of *any* biological network formed by interactions between cells.

The paper is organised as follows. First, a brief review of one of the classical networks of immunology, the idiotypic network is presented to provide a context for the work. This concept is examined from both an AIS and an immunological perspective, leading to a conclusion that the field is still of surprising relevance in modern research, despite doubts over previous years as to its contribution to modern immunology. Two key concepts that underpin our research are next discussed. First, the role played by *shape-space* in computational immunology is discussed, again from a dual-perspective of immunology and engineering. A summary of the most relevant tools of complex network theory is then given, for a reader unfamiliar with the area, as these concepts are central to our argument. Bringing these ideas together, we discuss the topologies of some of the shape-spaces common in AIS, and provide empirical evidence to justify the discussion in Sect. 6. Several flaws in the approach are identified which lead to Sect. 7 which presents the main contribution of the paper; the concept of the potential network. The remainder of the paper presents results obtained in four potential networks, each with different topological properties using an idiotypic network model. Our results illustrate the utility of the potential network methodology and provide a key result relating structure to function: that the *topology* of a network can separate zones of tolerance from zones of immunisation.

## 2 Idiotypic networks

As stressed in the introduction, the goal of the paper is to present a methodology for studying the relationship between structure and function in a biological network which has implications for both immunologists and engineers. Our results are illustrated using an idiotypic network model, though they are not in any sense limited to explaining function in idiotypic networks only. Idiotypic networks have at times been treated with some scepticism by immunologists, with some knock-on effects in the engineering community. Therefore, we provide a brief summary and overview of recent trend in both AIS and immunology which has two functions in an attempt to highlight the relevance of the methodology proposed in this paper to both the AIS and immunological communities.

### 2.1 Idiotypic networks in immunology

Although idiotypic networks, based on seminal work by Jerne (1985) dominated the world of immunology for over a decade, they fell out of fashion in the nineties, partly due to a lack of experimental evidence which could justify the theories. However, signs of a possible renaissance are now visible (Behn 2007). Coutinho (2002) suggests that it is now pertinent to revisit Jerne’s early work and consider the role of cellular and molecular networks in the establishment and maintenance of natural tolerance. In a further publication (Coutinho 2003), Coutinho goes on to suggest that

… the time is now ripe for exploring the full potential of idiotypic interactions, both in basic aspects of immune operations and in the respective clinical operations.

Coutinho’s main point is that the failure of the network theories to make a real impact relates to the attempts to find experimental evidence for phenomena that were already adequately explained by ‘clonal’ immunology. A number of fundamental challenges such as pre-immune repertoire selection, global regulation and tolerance still remain; Coutinho advocates that a network based approach represents a way forward, and in Coutinho (2002) goes further, recommending large and coordinated efforts on the scale of the Human Genome project to address this.

His opinions are backed up by a renewed clinical interest in idiotypic interactions, particularly in the area of auto-immune diseases, e.g. (McGuire and Holmes 2005; Shoenfeld 2004). Increasing attention is now paid to the fact that T-Cells can also have idiotypic interactions (Tite 1986); recently experimental results even show that a dynamic idiotypic T-cell network may exist in the body (Lal et al. 2006). Other work exploits anti-idiotypic T-cell interactions in the control of auto-immune diseases (Cohen et al. 2004). The reader is referred to the recent review by Behn (2007) for more detailed insights.

### 2.2 Idiotypic networks in AIS

Regardless of the ongoing debate in immunological circles, Jerne’s idiotypic theory has inspired a number of models in the AIS community, typically applied to domains such as clustering (De Castro and Von Zuben 2000), classification (Timmis et al. 2001) and in robotic control (Ishiguro et al. 1997). A brief review is given here. Typically these are discrete models, in which entities in the model (equivalent to antibodies) interact with both each other and the environment (antigens). Recent theoretical work suggests that the type of models used in clustering may suffer problems, particularly when clustering data which contains some dense point regions (Stibor and Timmis 2007). Nonetheless, the models have been widely adopted by the AIS community—in fact aiNET, although originally developed for clustering, is perhaps best known as an optimisation algorithm. Perhaps more in the spirit of Jerne, a large body of ongoing work exists in the application of idiotypic networks to robotic applications, in which the network concept is used to induce flexible behavior mediation, exploiting its ability to provide a global perspective on an entire system (Whitbrook et al. 2007). A recent summary of AIS applications in this field is provided in Whitbrook et al. (2007).

Finally, lying somewhat in between the fields of immunology and applications of AIS, work begun by Bersini (summarised and reviewed in Bersini 2003 and continued for example by Hart et al. in (2005, 2006, 2007a, b) in an immunological context and by Dilger in (2006) and Dilger and Strangeld (2006) in a more engineering inspired context) has attempted to probe the underlying idiotypic models used in engineering applications, asking questions as to how the effects of the affinity function from which interactions are derived affects *function*. Further study into how idiotypic interactions can be better exploited in AIS seems relevant—in particular, moving towards some theoretical understanding of these interactions will place AIS on much firmer foundation than currently exists.

## 3 Shape-space

The computational study of immunology was greatly facilitated by the introduction of the concept of *shape-space* by Perelson in (1989) which offered the possibility of representing *cells* in some low-dimensional space in which properties such as mutual affinity and similarity of cells could be derived from the relative positions of points in the space. The main idea behind the concept is that a cell can be represented as a set of *N* parameters, describing physical properties such as molecular shape or molecular charge etc. These *N* parameters can be represented as an N-dimensional vector space known as shape-space *S*, and affinity between two cells can be measured by a suitable metric within *S*.

Although a seemingly elegant concept, there are a number of theoretical issues which arise when defining a shape-space to represent data, whether that data originates from an immunological or engineering source. In-depth discussion of these issues is beyond the scope of this paper, however, we note them very briefly below, in further support of our argument that the approach advocated in this paper avoids any explicit definition of a shape-space (and affinity function) altogether.

From an immunological perspective, a major drawback of defining shape-space is that in order for it to be realistic, the dimensionality of *N* (the vector representing a molecule) needs to be very high and it is likely that any function that specifies affinity between any two vectors is both irregular and discontinuous (Carneiro and Stewart 1994). Indeed Carneiro et al. (1996) even go as far as to suggest that *… the danger is that heuristically stimulating visions of the organisation of the immune system based on the shape-space concept may be illusions based on little else than wishful thinking*. The shape-space abstraction has been adopted wholesale by the AIS community as being isomorphic with the vectorial representation of a dataset: each data-point being an artificial *antigen*, perhaps falling under the recognition region of some artificial *lymphocytes*. Whilst pragmatic from a computational perspective, this has some often ignored undesirable effects. In brief, as the dimensionality of shape-space increases, its volume increases exponentially faster. Any metric defined across this volume becomes increasingly meaningless, as all points tend to become equidistant. An AIS perspective of the unintuitive effects of this “curse of dimensionality” is given by Stibor et al. (2006) in the context of negative/positive-selection algorithms; attempts have been made to address these concerns in a complete re-interpretation of shape-space outlined by McEwan and Hart in (2008).

## 4 Complex networks

In order to aid understanding of the ideas presented in the rest of this paper, we provide a brief summary of the key concepts from complex network theory that underpin our research. The summary is necessarily brief—for an excellent introduction to complex networks the reader is referred to the overview of Newman available in (2003). A number of definitions make the following discussion clearer. These definitions are adapted from Newman (2003).

### Degree distribution

We define *p*_{k} to be the fraction of vertices in a network with degree *k* or (equivalently) the probability that a randomly chosen vertex has degree *k*. A histogram plotted of this value *p*_{k} is referred to as the degree distribution.

### Cluster Coefficient

This measures the average probability in a network that two nodes which have a mutual parent are also connected and is calculated as: \(C =\frac{3 {\times} \text {triangles\ in\ the\ network}}{{\text{number\ of\ connected\ triples}}},\) where a connected triple is a single vertex connected to a pair of others (Newman 2003).

### 4.1 Regular graphs

*d*, i.e. is connected to

*d*other nodes. Figure 2a shows a regular network with degree = 4. In the mathematical field of graph theory, a further type of regular graph can be distinguished, the bipartite or biclique. In a complete bipartite graph, the nodes can be partitioned into two distinct sets

*V*

_{1},

*V*

_{2}such that every vertex of the first set

*V*

_{1}is connected to every vertex of the second set

*V*

_{2}, and no edges exist between nodes which are both members of

*V*

_{1}, and similarly for

*V*

_{2}. Figure 2b shows an example of a regular

*bi-partite*graph with degree 4. The cluster coefficient of this graph will be trivially equal to zero.

### 4.2 Scale free graphs

Random graphs have a binomial degree distribution (Newman 2003); on the other hand, most real-world networks are found to have a highly skewed distribution. Such distributions can be described as power law distributions, with *p*_{k} ∼ *k*^{−α}. These networks are referred to as scale-free networks^{2}, and are surprisingly common in the real-world. They were first observed by Price (1965) in a network of citations between scientific papers, but have since been observed in for example metabolic networks, protein networks, the Internet, and networks of human sexual contacts (Newman 2003).

It has been shown that most of the biochemical networks can be characterised with at least broad-scale degree distribution, if not by a scale-free degree distribution, and that these distributions can lead to remarkable robustness properties. Additionally, it has been shown that most of the biological networks have a degree distribution which is intrinsically heterogeneous, where some nodes, considering their intrinsic chemical properties, are able to stimulate a large number of cells, contrary to others that stimulates only a few number of cell types. The first ones are naturally *made* to connect to a large number of other cells—they are *natural hubs* (Bersini et al. 2006).

In contrast, the majority of published models of idiotypic models have made the assumption that the affinity function is defined in such a way that every cell type has intrinsically the same number of potential stimulation partners, i.e. the degree distribution is homogeneous (this is discussed in more depth later in Sect. 6). This disregards a plethora of recent results in the area of complex biochemical networks showing that the majority of the real-world network do not share this feature.

## 5 The topologies of some common shape-spaces

Having provided definitions of both shape-space in an immunological context and of the common ingredients of complex network theory, we attempt to draw the two concepts together by examining the topologies of some common shape-spaces used in both theoretical AIS research and in computational immunology from a complex networks perspective.^{3}

### 5.1 2D shape-space

We discuss a particular 2-dimensional shape-space which is characterised by points defined on a grid by integer values. Two categories of affinity measure can be defined in this space; the measures can be related to the immunological concepts of complementarity and similarity and are discussed in detail below.

#### 5.1.1 A similarity based affinity measure

A common application in the AIS community is that of classification or clustering of large datasets. In order to accomplish this, data is mapped to antibodies or antigens and represented as a vector. Two data-points which are within some defined distance of each other (where the distance is calculated according to some pre-determined measure, e.g. Euclidean distance or Manhattan distance) are said to have affinity with one another. An affinity measure *A* is defined such that two points are said to have affinity with each other if the distance between them according to the chosen measure, is less than some threshold. If two points have affinity, then in network terminology, they can be considered to be connected. Edges in the network thus exist between nodes that have *similar* vectors.

*X*,

*Y*as shown in Fig. 3a. The potential space consists of a maximum of

*X*×

*Y*data-points, of which any data-set will be a subset. A node (i.e. mapped from a data-point) is specified by a position (

*x*,

*y*) on the grid. Node

*A*stimulates

*B*if

*B*lies within a circular region of radius

*r*(defined using a Euclidean distance measure) centered on

*A*itself and therefore the nodes interact; in network terminology, an edge thus exists between the two nodes. In this 2D space, every node can potentially interact with exactly the same number of nodes—the actual number being proportional to

*r*

^{2}other nodes. The network is therefore regular, i.e. has a homogeneous degree distribution. (In practice, cells lying closer than

*r*to the edges of the 2D space have recognition regions which lie partially outside the 2D grid and therefore fewer potential partners, however this can be addressed by using a wrap around grid.) If node

*A*interacts with node

*B*, and node

*B*interacts with node

*C*, then node

*C*can also interact with node

*A*(Fig. 3b)—the potential clustering coefficient of the network formed in this case is high. We can therefore draw the following conclusions regarding the topology of a network formed in 2D shape-space by a similarity based affinity function:

The network is (essentially) regular

The cluster coefficient is high

The connectivity of the network is dependent on

*r*which can be correlated with the degree of the potential network.

#### 5.1.2 Complementary affinity

Although a similarity-based affinity measure is straightforward to interpret from a machine-learning or information-theoretic perspective, it is less intuitive from an immunological perspective—the ubiquitous metaphor used to describe interactions between cells in the immune system is that of the lock-and-key, where *complementary* shapes interact. To date, the use of complementary affinity measures in real-valued shape-spaces has not been investigated; intuitively, they do not perhaps have an obvious meaning.^{4} However, it is instructive to consider the topological implications of defining a complementary affinity measure in a 2D shape-space.

*A*stimulates another node

*B*if

*B*lies within a circular region of radius

*r*

*centered on the point*(

*X*−

*x*,

*Y*−

*y*). This is illustrated in Fig. 4. As in the case outlined in Sect. 2, the

*potential degree*of any cell is therefore the maximum number of other cells to which it can potentially connect, which is governed by

*r*. This quantity is equal for all cells and therefore the network is also regular.

^{5}In contrast to the similar affinity measure, however, the cluster-coefficient of this network is necessarily close to 0 as triangles cannot be formed—see Fig. 4b. The figure shows that if

*A*stimulates

*B*and

*B*stimulates

*C*, then

*A*cannot stimulate

*C*because

*A*is similar to

*C*. Therefore, there cannot be any triangles in the network and its clustering coefficient is expected to be zero. More precisely, the cluster coefficient is somewhere slightly above 0 as the stimulation zone of nodes in the centre of the space in fact is centred on the node itself.

^{6}Therefore, we can summarise the properties of the complementary affinity function used in a 2D shape space as follows:

The network is (essentially) regular

The cluster coefficient is close to zero

The connectivity of the network is dependent on

*r*which can be correlated with the degree of the potential network.

### 5.2 Bitstring shape-space

Although vector shape-spaces are more typical from a machine-learning perspective, many AIS algorithms and a number of computational immunological simulations adopt a bitstring shape-space. This has some advantages in that it offers a far richer matching space: a number of affinity measures can be defined which rely on (for example) contiguous matching regions, or Hamming Distance.

In Hart et al. (2006), the topological properties of a bit-string space are examined empirically. We formalise that discussion here.

*i*has an affinity with and therefore stimulates another cell

*j*if the Hamming distance between them is higher than a certain threshold

*T*. The parameter

*T*plays an equivalent role to the parameter

*r*in the 2D shape-space model in determining the number of possible interactions any cell may take part in. As in the previous case, any node can potentially interact with exactly the same number of other nodes, and hence the network is completely homogeneous. This property holds true for all values of T and is illustrated in Fig. 5 in a hypercube consisting of (for simplicity) bitstrings of length 3 and threshold

*T*= 1, i.e. nodes with a Hamming Distance ≥2 are connected.

*T*also plays another important role: it influences an important topological feature, the global cluster coefficient of the network. Given a suitable threshold value

*T*, it is possible for a proportion of the nodes to form triangles between triples of three bitstrings—an example is shown in Fig. 5b for 3 bitstrings of length 13 and a threshold

*T*= 7. It can further be proved mathematically that triangles can always form between three bitstrings for a threshold

*T*such that

*T*<

*floor*(2

*N*/3). The cluster coefficient will have maximum value at

*T*= 0 and a minimum value of 0 at

*T*= 2

*N*/3. We summarise the essential properties of bitstring spaces as:

The network is (completely) regular

The cluster coefficient is dependent on

*T*. It is non-zero if*T*<*floor*(2*N*/3) and zero otherwiseThe connectivity of the network is dependent on the threshold

*T*

## 6 An empirical perspective

*theoretical*perspective, and showed that the underlying networks formed can vary greatly. An alternative approach to understanding topology is to take an empirical route, and retrospectively analyse topologies that arise from applying an immune model in shape-spaces defined by the tuple \(({\mathcal{R,A}}).\) A detailed empirical investigation of this sort was undertaken by the authors and reported in Hart et al. (2006, 2007a). These studies analysed networks produced when running a simple immune model inspired by idiotypic network interactions, in a bitstring and a 2D shape-space, with both similarity based and complementary based affinity measures. The main findings of this work (Hart et al. 2006, 2007a) were that the presence of certain topological features in a network can help promote tolerance. In particular:

The

*cluster-coefficient*of the network plays a key role: low values of cluster-coefficient promote tolerance.The

*degree distribution*of the resulting network of cells also influences the ability of the network to tolerate antigen: extreme values of average degree (high or low) promote intolerance; in such cases, the effective network collapses through over or under stimulation.

The reader is referred to these publications for the exact details of the experiments and model used; the results are summarised here to support the theoretical analysis presented above. Although the studies highlight interesting relationships between topological features of a network and functionality (in this case, the ability of a network to tolerate antigen), we draw the readers attention to a number of difficulties associated with a purely empirical approach which lead to our proposal for a new methodology for studying the structure-function relationship in a more systematic manner.

### 6.1 Limitations of an empirical approach

*heterogeneity*of degree distribution are observed in the networks that result from running an immune algorithm in any given (\({\mathcal{R,A}}).\)

^{7}The degree distribution alone, however, masks a key topological observation: although the two distributions obtained at threshold 7 and 8 in the bitstring network appears very similar, the networks in fact differ completely in their internal structure; the network with

*T*= 8 has cluster coefficient of zero. Thus

*hubs*that appear temporarily in this network (highly connected nodes) must in fact be very unstable, and collapse immediately after appearing. This underlines a key in difficulty in utilising an empirical approach to deriving topology in a dynamical system: any network properties measured are merely a snapshot of the network at some moment in time.

A further difficulty arises when attempting to utilise an empirical approach. Such approaches attempt to infer global properties of an underlying network by examining a sample or subnet of that network. However, the sampled data only represents a small fraction of the complete network, for instance, in the experiments described in (Hart et al. 2007b) and those shown later in Fig. 9, the observed network is approximately 40% of the known global network.^{8}

Stumpf et al. (2005) question whether it is possible to extrapolate from subnet data to properties of the global network. In fact, they show mathematically that it is possible only if the degree distributions of the network and randomly sampled subnets belong to the same family of probability distributions. In particular, they conclude that observed scale-free topologies in partial samples cannot be confidently extrapolated to complete interactomes. This result is further verified by Han et al. (2005) through a theoretical study of protein–protein interaction networks. Although experimental results elaborated in Hart et al. (2007b) showed that for the 2D and bitstring shape-spaces evaluated, the subnets evolved do in fact exhibit many of the same topological properties as the known underlying networks (particularly with respect to cluster coefficient), it is therefore unclear that a subnet showing scale-free properties would necessarily reflect an underlying scale-free structure.

### 6.2 An alternative methodology

The previous sections have outlined two difficulties associated with empirical analysis of observed networks generating by running an immune algorithm on an implicit network generated via an explicit choice of representation and shape-space. The analysis suggests that it would be unwise to arbitrarily select representations and affinity measures and attempt to infer the structure of the implicit, underlying network from effective networks evolved using an immune algorithm. Clearly, it would be preferable to select a representation and affinity measure such that we can guarantee the implicit network formed as a result of this has the desired properties. How this can be achieved however is far from clear, particularly if we wish to study complex topologies, such as scale-free networks.

Therefore, in the remainder of the paper we outline a novel method which enables global topologies with known properties to be directly studied. As alluded to in the introduction, we exploit the fact that a representation and affinity measure *define an implicit network*, suggesting that the network defined by these quantities can be *directly* studied, as a proxy for some (\({\mathcal{R}}\) and \({\mathcal{A}})\) and without having to pre-define (\({\mathcal{R,A}}\)). This enables attention to focus directly on understanding the correlation between a topology \({\mathcal{T}}\) and an immune model with dynamics \({\mathcal{D}}. \) A systematic study can be performed investigating instantiations of \({\mathcal{T}}\) in conjunction with any instantiation of *D*, observing and analysing sub-graphs that emerge on a global topology that has definable characteristics. This directly facilitates a systematic study of the relationship between structure, dynamics, and function in a network. This is formalised in the next section. We then illustrate the utility of the approach using a model of dynamics \({\mathcal{D}}\) based on idiotypic interactions. The model chosen is merely an exemplar: *any* dynamical model of interactions between cells can be studied using this methodology, for example networks of regulatory T-cells, or B/T-cell interactions in regulating auto-immune diseases (Coutinho 2003).

## 7 Potential networks

We now generalise the intuitions described earlier in the paper, which were gleaned as a result of theoretical and experimental evaluation of the underlying networks formed using two types of representation and two different affinity measures.

*Remark 1*. A representation \({\mathcal{R}}\) in which cells interact via an affinity function \({\mathcal{A}}\) implicitly defines a network \({\mathcal{N}}\) with topology \({\mathcal{T}}\).

This leads to the following conjecture in terms of studying immune dynamics.

*Remark 2*. As any potential topology \({\mathcal{T}}\) is simply a result of a representation \({\mathcal{R}}\) and an affinity function \({\mathcal{A}},\) we can use any graph \({\mathcal{T}}\) as a proxy for a particular representation and affinity function and study a model of immune dynamics running directly on this topology.

*potential network*; running an immune model upon this network populates a subset of the nodes and results in an

*effective network*, as illustrated in Fig. 7. The model imposed determines how nodes in the potential network become populated and later die (

*meta-dynamics*) and how populated nodes interact and thus vary in concentration (

*dynamics*). The sub-graph produced is a sample of the potential network in the same way that experimental results obtained in protein-networks are a sample of the potential protein network.

The methodology enables a direct study of topology using models that are appropriate from an *immune* context: an effective network can be grown using immune-inspired dynamics in a manner constrained by the topology of the potential network. Although the complex network literature specifies many methods of *growing* networks with particular topologies (for example, scale-free, small world) the algorithms used are often often unrealistic from an immune perspective: for example, scale-free topologies can be grown via *preferential attachment* (Barabási and Albert 1999) in which new nodes preferentially attach to existing nodes. Whilst satisfying topological requirements, this method has no realistic interpretation in a biological context. We circumvent this problem in our approach by specifying a complete network, then populating nodes using models consistent with our knowledge from immunology, in contrast to attempting to grow the network using methods which result in topologically desired features but implausible from a biological perspective.

In the next section, we show how the potential network idea can be exploited, using a case-study which investigates the emergence of tolerance in an idiotypic network to illustrate the point. We define four potential topologies, each with different structure, a regular ring, a bipartite regular ring, a scale-free network and a bi-partite scale-free network. We then study the dynamics of the emergence of an idiotypic network on each of these topologies to directly determine whether or not the underlying network facilitates the emergence of tolerance regions in the network.

## 8 A simple model of an idiotypic network

*G*; we refer to nodes as cells and links between cells represent affinity. Cells can be designated as either antibodies or antigens; either class of cell can occupy any position on the potential network. We consider graphs in which a cell \(X\) stimulates another cell

*Y*if

*X*has concentration greater than 0 and if

*X*has a link with

*Y*in the underlying potential network ((

*X*,

*Y*) =

*true*). All connections are assumed to have the same weighting.

^{9}The stimulation

*S*

_{(X,Y)}received by a cell

*X*from a cell

*Y*is zero if

*X*does not have a link with

*Y*in the potential network, and is equal to α*

*C*

_{Y}otherwise, where α is simply a pre-defined global constant and

*C*

_{Y}is

*Y*’s concentration. We define two simple rules to govern interactions (following on from Bersini (2002, 2003), Calenbuhr et al. (1995), Hart et al. (2006), and Stewart 2004):

- (1)
Cells designated as

*antibodies*are stimulated by interactions with*antigens*and with other*antibodies* - (2)
Cells designated as

*antigen*are stimulated by interaction with*antibodies*.

*A*is therefore calculated according to Eqs. 1 to 3 where AB and AG refer to the set of all antibodies and antigens respectively.

## 9 Results

### 9.1 Experimental methodology

In Fig. 8, *P* = potential network, *N* = the number of nodes in the potential network, *i* = iterations, *I*_{max} = maximum iterations, *C*_{init} = initial concentration of added cells, *p* = periodicity of adding antigen, *n*_{Ag} = number of antigens added. The set of *n*_{Ag} antigens introduced can be generated in a number of ways; each new set can be generated at random, a set *AG* might be introduced repeatedly to study memory, or each set may be a slightly mutated version of the previously added one. Results are reported here use a new, random set of antigens each time. However, extensive experimentation undertaken (not reported here) showed that the results were independent of the method of generating the antigen set. The period *p* over which new antigens are introduced is selected such that antigens not tolerated by the network will have been completely cleared before a new set is introduced; given that they are added with initial concentration *C*_{init} = 100, a network should clear such antigen in 100 iterations. However, this period is deliberately set to 10 × *C*_{init} to ensure this is the case.

We report results using *N* = 10^{4}, *I*_{max} = 20,000, *p* = 1,000, and *n*_{Ag} = 50 for a variety of networks *P*. Twenty random sets of antigen were added in each experiment. The percentage of antigen remaining, *T*, is averaged over the 20 additions, and recorded for each network *P*. To ensure fair comparison, for each potential network *P* evaluated, 20 identical sets of random antigen are presented.

### 9.2 Regular potential networks

We first study, the topological effects on the emergence of tolerance, assuming a regular graph in which all nodes share the same potential degree.

Two types of regular graphs as shown in Fig. 2a, b are studied. The former contains *N* nodes whereas the latter is formed from two rings of size \(\frac{N} {2}.\) We consider potential networks of 10^{4} nodes in order to provide comparable results to those obtained empirically in Hart et al. (2006, 2007b). For both types of topologies, we generate graphs at random which have degree *k*. Graphs are generated for values of *k* varying between 0 and 800. The former case corresponds to an affinity function based on the similarity between cells and has non-zero cluster coefficient. The bi-partite graph has exactly the same spatial constraints and degree, but implicitly defines two groups of interacting cells in a bi-partite fashion. This bi-partite regular graph corresponds to the complementarity affinity function and has cluster coefficient = 0.

Figure 9 shows the results, using experimental parameters as described in the previous section. In the upper panel, we show the percentage of tolerated random antigens as a function of the average degree of the underlying regular network for two types of regular networks: a regular ring and bi-partite regular ring. The lower panel shows the dependence of the equilibrium size of the network in the potential average degree.

*k*>. This result corroborates with results obtained empirically with a 2D-shape space model (Hart et al. 2007b), validating the potential network abstraction introduced here.

*k*, the network degree) limited by abrupt transitions:

- (1)
With low degree, the potential network does not provide enough stimulation to maintain the majority of the antibodies inside the window range. Some antigen is tolerated; essentially the nodes on the potential network occupied by these antigens have a low probability of being connected to any nodes occupied by antibodies by virtue of their low degree.

- (2)
For a moderate level of potential links, the potential network offers the ideal conditions for the emergence of a self-sustained system of antibodies, avoiding both under and over stimulation. The network reaches a peak in size, which corresponds with a minimum in terms of the percentage of antigen tolerated. A fine balance exists: over a certain range of

*k*, the persistent network of antibodies is able to isolate regions of nodes (i.e. subgraphs) of the potential network, providing topological protection to those antigen that occupy nodes in these regions. As the network increases in size, the size of the regions able to offer topological protection decreases, resulting in a decrease in the number of antigen tolerated. Further increasing*k*reduces the size of the network once more (due to the stimulation levels exceeding the upper boundary), resulting in a corresponding increase in tolerance again. - (3)
At high degree, i.e. when the potential degree becomes too high, the majority of nodes become over-stimulated. Under these conditions, the network collapses. Tolerance is low; the few antibodies in the network are easily stimulated (by virtue of their high degree) and quickly eradicate antigen.

### 9.3 Scale-free potential networks

As noted in Sect. 1, networks occurring in natural system tend to be intrinsically heterogeneous with respect to degree distribution, in that they often contain hubs. Furthermore, they often exhibit scale-free properties.

In the context of studying the emergence of tolerant regions in a graph, it is plausible that heterogeneity in a topology may offer the potential for certain types of antigen (depending on their degree and position in the global interaction network) to become topologically protected against some antibodies: these antibodies would therefore be incapable of destroying the antigens, therefore increasing the global level of tolerance. Moreover, heterogeneity effects also play an important role in determining the equilibrium size of the in turn influences the capacity of an idiotypic immune network to tolerate antigens.

Therefore, we now consider two heterogeneous potential graphs; a scale-free potential network (similar to the WWW model proposed by Barabasi (Barabási and Albert 1999), and a bi-partite scale-free potential network. Networks are again generated containing 10^{4} nodes. For each topology, potential graphs of varying average degree (0–1,500) are generated at random, and then the effective network evolved on each graph as before. Experiments were repeated using exactly the same settings as in the previous case with regular graphs.

Both classes of scale-free network and the bi-partite regular network show a greater ability to tolerate certain classes of antigens than a regular network.

The results obtained with scale-free topologies concur with the previous conclusions regarding topologies with homogeneous degree distribution. Bi-partite topologies, either regular or scale-free, promote the emergence of high levels of tolerance for most values of <

*k*> .Bi-partite scale-free networks give rise to high levels of tolerance over a wide range of values of average degree. Moreover, contrary to bi-partite regular potential graphs, the scale-free network promotes a stable self-sustained effective network for a very broad range of parameters.

The ability of the regular bi-partite network to tolerate antigens drops by one order of magnitude when then size of the network increases.

Studying abstract potential topologies allows us to draw a further connection regarding the relationship between the structure and the function of a network. In addition to the role played by the *cluster coefficient* of the network already identified, it further seems clear that the *heterogeneity* of a network plays a key role.

## 10 Conclusion

This paper has highlighted the connection between the affinity function and representation chosen when studying an immune algorithm and the topology of the implicit network produced as a result of this. We have introduced the notion of the *potential network* as a replacement for these quantities, which then enables a direct study of the influence of different topologies on network behaviour. This mechanism allows topological characteristics to be isolated in order to study the relationship between the topology, the dynamics imposed on the topology through an immune model, and the resulting network functionality. It allows generic hypotheses to be proposed and tested, which are not restricted to a particular shape-space neither affinity function nor to any particular type of model dynamics. An AIS practitioner might utilise the method to gain insights into the topology required to produce a desired functionality (and thus by implication, design a suitable affinity measure). An immunologist might study interactions in networks of regulatory T-cells, or the role of B and T-cells interactions in regulating auto-immune diseases on biological plausible topologies. Thus, the method offers an additional tool to both computational immunologists and AIS practitioners in understanding their respective domains, and in particular, in attempting to understand the relationship between the structure and function of a network. In the spirit of Timmis et al. (2008) and Cohen (2007), we suggest *… that there is a great deal for both engineering and immunology to learn from each other through working in an interdisciplinary manner*.

The results obtained from studying both homogeneous and heterogeneous potential networks are a significant departure from many previous studies. The results obtained using these networks confirm suggestions from previous work that topology plays a key role in influencing the functionality of a network but crucially, give us new information regarding the properties that a realistic potential topology (and by implication, affinity function) must exhibit. Results in the simple regular scenario have shown that small differences in the underlying potential topology change completely the final outcome of the dynamical/topological system, but most importantly, we have shown that network heterogeneity can drive the topological evolution of a growing immune network, and provide protection to certain cells resulting in tolerance.

The way that nature has created the set of all possible cells also implicitly defines a potential network. This network provides the skeleton of all dynamical processes that can evolve in a immune system scenario. Studies in other domains such as the propagation of epidemic (May and Lloyd 2001; Pastor-Satorras and Vespignani 2001) and evolutionary systems (Santos et al. 2006) have already shown that it is essential to study the influence of topology in networks when trying to understand their functional properties; our results suggest that this point clearly holds true for studies of immune networks as well (and will likely extend to studies of other immune cells such as T-cells).

Moreover, as a result of their chemical and physical properties, some cells interact more than others because their physical features make such interactions extremely likely. Thus highly connected nodes arise because they have intrinsic features that allow them to connect to a higher number of partners. We can describe these nodes as *natural hubs*, i.e. nodes that were born to be hubs instead of being hubs just because of their presence in the network for a long period. These features are included in the definition of each cell and in its position in the global potential network. By disregarding the intrinsic heterogeneity of biological systems, one may lose one of the possible mechanisms that drives the topological evolution of a growing immune network. This observation has particular significance for the AIS community where heterogeneity in affinity functions is rarely, if ever, implemented.

Finally, we note that in both computational immunology and AIS, attention is often focussed on the *dynamics* of a model, in order to obtain desired properties. The dynamics of the concentration level of a node often produces the necessary mechanism for the tolerance and appearance of immune cells, but this in turn must be governed by the constraints imposed by the underlying potential topology.

## Footnotes

- 1.
Note that the structure of this network is rarely given any thought when choosing a combination of shape-space and affinity function in the AIS literature.

- 2.
Though strictly speaking it is only the distributions that are scale-free, scales can be present in other network properties.

- 3.
Theoretical studies in both disciplines tend to study low-dimensional models; although this is clearly in contrast to realistic engineering problems which are characterized by high-dimensionality, it still allows some insights to be gained.

- 4.
However, this has recently begun to be addressed by (McEwan and Hart 2008) using an approach which attempts to ground machine-learning more thoroughly in an immunological context with proposed benefits in terms of scaling and avoiding the ’curse of dimensionality’.

- 5.
Except for cells at the edges as described in 5.1.1.

- 6.
The same also holds at very large values of

*r*when the stimulation circles overlap. - 7.
Despite known regularity in the underlying networks.

- 8.
- 9.
This could be easily modified and does not weaken the results.

### References

- Albert R, Barabási AL (2002) Statistical mechanics of complex networks. Rev Mod Phys 74:47–97CrossRefGoogle Scholar
- Amaral LA, Scala A, Barthelemy M, Stanley HE (2000) Classes of small-world networks. Proc Natl Acad Sci USA 97(21):11149–11152CrossRefGoogle Scholar
- Barabási AL, Albert R (1999) Emergence of scaling in random networks. Science 286(5439):509–512CrossRefMathSciNetGoogle Scholar
- Barabási L, Oltvai Z (2004) Network biology: understanding the cell’s functional organization. Nat Rev Genet 5:101–113CrossRefGoogle Scholar
- Behn U (2007) Idiotypic networks: towards a renaissance? Immunological Reviews 216:142–152Google Scholar
- Bersini H (2002) Self-assertion vs self-recogntion: a tribute to francisco varela. In: Proceedings of ICARIS 2002, pp 107–112Google Scholar
- Bersini H (2003) Revisiting immune networks. In: Proceedings of ECAL 2003, pp 191–198Google Scholar
- Bersini H, Lenaerts T, Santos FC (2006) Growing biological networks: beyond the gene-duplication model. J Theor Biol 241(3):488–505CrossRefMathSciNetGoogle Scholar
- Calenbuhr V, Bersini H, Stewart J, Varela F (1995) Natural tolerance in a simple immune network. J Theor Biol 177:199–213CrossRefGoogle Scholar
- Carneiro J, Stewart J (1994) Rethinking “Shape Space”: evidence from simulated docking suggests that steric shape complementarity is not limiting for antibody–antigen recognition and idiotypic interactions. J Theor Biol 169:390–402CrossRefGoogle Scholar
- Carneiro J, Coutinho A, Stewart J (1996) A model of the immune network with B-T cell co-operation. II—the simulation of ontogenesis. J Theor Biol 182:531–554CrossRefGoogle Scholar
- Cohen IR (2007) Real and artificial immune systems: computing the state of the body. Immunol Rev 7:569–574Google Scholar
- Cohen IR, Quintana FJ, Mimran A (2004) T-regs in t-cell vaccination: exploring the regulation of regulation. J Clin Invest 114:1227–1232Google Scholar
- Coutinho A (2002) Immunology at the crossroads. EMBO Rep 3(11):1008–1011Google Scholar
- Coutinho A (2003) Will the idiotypic network help to solve natural tolerance? Trends Immunol 24(2):53–54Google Scholar
- De Castro L, Von Zuben F (2000) An evolutionary immune network for data clustering. Proceedings of the VI Brazilian symposium on neural networks (SBRN'00). IEEE Computer Society, Washington, DC, USA p 84Google Scholar
- Dorogotsev SN, Mendes JFF (2003) Evolution of networks: from biological nets to the internet and WWW. Oxford University Press, OxfordGoogle Scholar
- Dilger W (2006) Structural properties of shape-spaces. In: Proceedings of international conference on artificical immune systems (ICARIS), pp 178–192Google Scholar
- Dilger W, Strangeld S (2006) Properties of the bersini experiment on self-assertion. In: Proceedings of the genetic and evolutionary computation conference (GECCO), pp 95–102Google Scholar
- Frankenstein Z, Alon U, Cohen IR (2006) The immune-body cytokine network defines a social architecture of cell interactions. Biol Direct 1(32):1–15Google Scholar
- Guimerá R, Amaral LAN (2005) Functional cartography of complex metabolic networks. Nature 433:895–900Google Scholar
- Han J-DJ, Dupuy D, Bertin N, Cusick M, Vidal M (2005) Effect of sampling on topology predictions of protein–protein interaction networks. Nat Biotechnol 23(7):839–844Google Scholar
- Hart E (2005) Not all balls are round: an investigation of alternative recognition-region shapes. In: Jacob C, Pilat ML, Bentley PJ, Timmis JI (eds) Artificial immune systems, Proceedings of ICARIS 2005, vol 3627. Springer, Heidelberg, pp 9–42Google Scholar
- Hart E (2006) Analysis of a growth model for idiotypic networks. In: ICARIS 2006, Lecture notes in computer science, vol 4163. Springer, Berlin, pp 66–80Google Scholar
- Hart E, Ross P (2005) The impact of the shape of antibody recognition regions on the emergence of idiotypic networks. Int J Unconventional Comput 1(3):281–313Google Scholar
- Hart E, Bersini H, Santos FC (2006) Tolerance vs intolerance: how affinity defines topology in an idiotypic network. In: Jacob C, Pilat ML, Bentley PJ, Timmis JI (eds) ICARIS 2006, Lecture notes in computer science, vol 4163. Springer, Heidelberg, pp 109–121Google Scholar
- Hart E, Bersini H, Santos FC (2007a) Topological constraints in the evolution of idiotypic networks. In: ICARIS 2007, Lecture notes in computer science, vol 4628. Springer, Heidelberg, pp 252–263Google Scholar
- Hart E, Bersini H, Santos FC (2007b) How affinity influences tolerance in an idiotypic network. J Theor Biol 249(3):422–436CrossRefGoogle Scholar
- Ishiguro A, Kondo T, Watanabe Y, Uchikawa Y (1997) A reinforcement learning method for dynamic behavior arbitration of autonomous mobile robots based on the immunological information processing mechanisms. Trans IEE Japan, 117-C(1):42–49Google Scholar
- Jeong H, Mason SP, Barabsi A, Oltvai ZN (2001) Lethality and centrality in protein networks. Nature 411:41–42CrossRefGoogle Scholar
- Jerne N (1985) The generative grammar of the immune system. Science 4(4):439–451Google Scholar
- Lal G, Shaila MS, Nayak R (2006) Recombinant idiotypic tcrβ chain immunization in mice generates antigen specific T-cell response. Mol Immunol 43:1549–1556CrossRefGoogle Scholar
- May RM, Lloyd AL (2001) Infection dynamics on scale-free networks. Phys Rev E 64(6 Pt 2):66112Google Scholar
- McEwan C, Hart E (2008) The problem with shape-space: boosting the immune system. In: ICARIS ’08: Proceedings of the 7th international conference on artificial immune system, pp 316–327Google Scholar
- McGuire KL, Holmes DS (2005) Role of complementary proteins in autoimmunity: a old idea re-emerges with new twists. Trends Immunol 26:367–372CrossRefGoogle Scholar
- Newman MEJ (2003) Structure and function of complex networks. SIAM Rev 45:167–256MATHCrossRefMathSciNetGoogle Scholar
- Pastor-Satorras R, Vespignani A (2001) Epidemic spreading in scale-free networks. Phys Rev Lett 86:3200Google Scholar
- Perelson A (1989) Immune network theory. Immunol Rev 10:5–36CrossRefGoogle Scholar
- Price DJ (1965) Networks of scientific papers. Science 149:510–515CrossRefGoogle Scholar
- Santos FC, Pacheco JM, Lenaerts T (2006) Evolutionary dynamics of social dilemmas in structured heterogeneous populations. Proc Natl Acad Sci USA 103(9):3490–3494CrossRefGoogle Scholar
- Shoenfeld Y (2004) The idiotypic network in autoimmnunity. Nat Med 10:17–18CrossRefGoogle Scholar
- Stewart J (2004) The affirmation of self: a new perspective on the immune system. Artif Life 10(3):261–276CrossRefGoogle Scholar
- Stibor T, Timmis J (2007) An Investigation into the compression qualities of aiNET. In: Proceedings of Foundations of Computational Intelligence, FOCI 2007Google Scholar
- Stibor T, Timmis J, Eckert C (2006) On the use of hyperspheres in artificial immune systems as antibody recognition regions. In: Proceedings of ICARIS 2006, Lecture notes in computer science. Springer, Berlin, pp 215–228Google Scholar
- Stumpf MPH, Wiuf C, May RM (2005) Subnets of scale-free networks are not scale-free: sampling properties of networks. Proc Natl Acad Sci USA 102:4221–4224CrossRefGoogle Scholar
- Tanaka R, Yi TM, Doyle J (2005) Some protein interaction data do not exhibit power-law statistics. FEBS Lett 579:5140–5144CrossRefGoogle Scholar
- Timmis J, Neal M (2001) A resource limited artificial immune system for data analysis. Knowl Based Syst 14:121–130CrossRefGoogle Scholar
- Timmis J, Andrews P, Owen N (2008) An interdisciplinary perspective on artificial immune systems. Evol Intell 1:5–26Google Scholar
- Tieri P, Valensin V, Latora V, Castellani GC2, Marchiori M, Remondini D, Franceschi C (2004) Quantifying the relevance of different mediators in the human immune cell network. Bioinformatics 21(8):1639–1643CrossRefGoogle Scholar
- Tite JP (1986) Direct interactions between b and t lymphocytes bearing complementary Receptors. J Exp Med 163:189–202CrossRefGoogle Scholar
- Whitbrook A, Aickelin U, Garibaldi JM (2007) Idiotypic immune networks in mobile-robot control. IEEE Trans Syst Man Cybern B 37(6):1581–1598 Google Scholar