# Loop analysis and qualitative modeling: limitations and merits

## Authors

- First Online:

- Received:
- Accepted:

DOI: 10.1007/s10539-006-9050-x

- Cite this article as:
- Justus, J. Biol Philos (2006) 21: 647. doi:10.1007/s10539-006-9050-x

- 140 Views

## Abstract

Richard Levins has advocated the scientific merits of qualitative modeling throughout his career. He believed an excessive and uncritical focus on emulating the models used by physicists and maximizing quantitative precision was hindering biological theorizing in particular. Greater emphasis on qualitative properties of modeled systems would help counteract this tendency, and Levins subsequently developed one method of qualitative modeling, loop analysis, to study a wide variety of biological phenomena. Qualitative modeling has been criticized for being conceptually and methodologically problematic. As a clear example of a qualitative modeling method, loop analysis shows this criticism is indefensible. The method has, however, some serious limitations. This paper describes loop analysis, its limitations, and attempts to clarify the differences between quantitative and qualitative modeling, in content and objective. Loop analysis is but one of numerous types of qualitative analysis, so its limitations do not detract from the currently underappreciated and underdeveloped role qualitative modeling could have within science.

### Keywords

ComplexityIdealizationLoop analysisPerturbationQualitative analysisQualitative modelingPrecisionSign digraphStability## Introduction

Throughout his career, Richard Levins has championed the scientific merits of qualitative modeling against “a prejudice, perhaps derived, legitimately or illegitimately, from other disciplines, that in order to know something we must define it precisely and measure it precisely” (Levins 1970, 77). Armed with this prejudice and a view of physics as the paradigmatically precise science, Levins believed many biologists were uncritically imitating the highly idealized quantitative models used by physicists in an attempt to achieve similar precision (Levins 1968a). Unconvinced of this approach, other biologists were developing complex quantitative models intended to mimic the complex structure and dynamics of real biological systems. The problem with this strategy was that the large quantities of data needed to parameterize and properly test these models usually could not be collected feasibly (Levins 1966). This was one difficulty Levins saw with the “systems” approach to modeling ecosystems adopted by the International Biological Program in the 1960s (Levins 1968b). Another was that he believed these complicated models would not increase understanding of the complex systems they were intended to represent. A qualitative modeling strategy that stresses enhancing understanding, Levins (1966) emphasized, would help avoid these problems and counteract the excessive focus on emulating models used in physics, which he believed typified and hampered much of the biological theorizing at the time.

Levins developed one method of qualitative modeling, called “loop analysis,” as a complementary alternative to quantitative modeling (Levins 1974). Although aspects of this method were independently discovered several times (Wright 1921; Mason 1953; Harary et al. 1965; Maybee 1966; Roberts 1971), Levins developed and systematically applied it within a wide variety of different biological contexts (Levins 1975a, b, 1998; Lane and Levins 1977; Puccia and Levins 1985, 1991; Levins and Schultz 1996), and it has subsequently been used in several studies within and outside ecology (Roberts and Brown 1975; Levine 1976; Briand and McCauley 1978; Desharnais and Costantino 1980; Flake 1980; Vandermeer 1980; Boucher et al. 1982; Lane 1986; Dambacher et al. 2003a, b). Economists have extensively used the same qualitative method, although not under the label ‘loop analysis,’ to evaluate properties of economic systems (e.g. Lancaster 1962; Bassett et al. 1968; Hale et al. 1999).

Qualitative modeling has been criticized for being conceptually and methodologically problematic (Orzack and Sober 1993). As a typical example of a qualitative modeling method, loop analysis shows this criticism is indefensible (Justus 2005). The method has, however, some serious limitations. One is that it usually cannot determine whether a system exhibits a particular property, such as local stability. This is an unavoidable shortcoming of the method being qualitative and true to some degree of any modeling method that does not utilize quantitative information about the system being modeled. Loop analysis partially remedies this shortcoming by pinpointing the additional information needed to make the determination for some properties, but the amount of additional information typically required increases dramatically with model complexity. Another limitation specific to loop analysis is that the conditions under which it is applicable are severely restrictive. It is therefore not a method of qualitative analysis with wide scope or evaluative power, as Levins and others sometimes seem to suggest (e.g. Levins 1975b; Lane 1998).

Loop analysis is but one of numerous types of qualitative analysis, so its limitations do not detract from the (currently underappreciated and underdeveloped) role qualitative modeling could have within science. The goal of this paper is to describe loop analysis, its limitations, and to clarify the differences in content and objective between quantitative and qualitative modeling. To situate loop analysis with respect to the space of possible qualitative modeling methods, Section 2 outlines different types of qualitative properties of models that represent features of systems with different degrees of specificity. Section 3 describes loop analysis and two kinds of model properties Levins and others have used it to evaluate. Sections 4 and 5 discuss some scientific merits of qualitative modeling and specific limitations of loop analysis. Section 6 concludes by comparing the function of qualitative and quantitative modeling within science.

## Qualitative properties of scientific models

- (i)
parts of the modeled system are designated by variables;

- (ii)
factors that influence system dynamics but are (usually) uninfluenced by it are designated by parameters; and,

- (iii)
system dynamics—relationships among system parts and between these parts and extrasystematic factors—are described by model equations.

*x*

_{1},...,

*x*

_{n}and

*n*equations, one for each variable. Assume that equation

*i*expresses

*x*

_{i}as a function of

*x*

_{1},...,

*x*

_{n}(possibly including

*x*

_{i}) and a set of parameters

*c*

_{1},...,

*c*

_{m}:

*n*×

*n*matrix

**A**= [

*a*

_{ij}] can be constructed that represents interactions between system parts (designated by variables

*x*

_{1},...,

*x*

_{n}), i.e.,

*a*

_{ij}represents the influence of

*x*

_{j}on

*x*

_{i}.

^{1}If the influence of

*x*

_{j}on

*x*

_{i}does not change for different values of

*x*

_{j},

*a*

_{ij}is a constant real value. If it does,

*a*

_{ij}is a more complicated function of

*x*

_{j}. Different degrees of precision about the

*a*

_{ij}represent different degrees of specificity about the system’s dynamics.

^{2}

For most models, especially those representing complex systems, it is usually impossible or infeasible to determine the quantitative value or precise functional form of most, let alone all of the *a*_{ij}. It is often possible, however, to determine qualitative properties of their functional form. At the qualitative extreme, only that there is or is not some interaction between variables can be ascertained. For this reason, matrices representing this limited information are sometimes called *Boolean*. If **A** is Boolean, it is symmetric (*a*_{ij} = *a*_{ji} for all \({i\ne j}\)) and its entries take the values 0 or non-0 to represent whether there are or are not functional dependencies among the variables *x*_{1},...,*x*_{n} as determined by the equations *F*_{i}, *i* = 1,...,*n*. Specifically, \({a_{ij}\ne 0}\) represents that *x*_{j} is an argument of *F*_{i}, and \({a_{ij}\ne 0}\) represents that it is not. This is the kind of information portrayed in unweighted, undirected graphs. Note that *a*_{ij} = *a*_{ji} = 0 represents the absence of a qualitatively discernable interaction between *x*_{j} and *x*_{i} (i.e., a null effect), not that its precise quantitative value is 0.

More than Boolean information is usually required to derive significant insights about modeled systems. If the direction of the interactions—that changes in *x*_{j} change the value of *x*_{i} and/or vice versa—are determinable, **A** is called *directed*. Its entries take the same values as Boolean matrices, but **A** may not be symmetric. Asymmetric causal relations in a system, for instance, can be represented with directed matrices. If, becoming more precise, the signs as well as directions of interactions are determinable, **A** is called *sign-directed* and its entries take the values +1, −1, or 0 to represent enhancing, inhibiting, or null effects between variables. This is the kind of information portrayed in sign directed graphs (digraphs), which are the primary focus of loop analysis. Note that the fact that all positive (negative) entries of a sign digraph are +1 (−1) does not mean all the interactions between variables are of equal magnitude. This stipulation, made by Roberts and Brown (1975, 579), would severely restrict the range of systems representable by sign digraphs.

Further specification of *a*_{ij} is sometimes possible without full determination of its mathematical form. *a*_{ij} may be an increasing or decreasing *monotonic* function for instance.
^{3} If *a*_{ij} is monotonic, furthermore, the functional form of the increase or decrease may be *convex* or *concave* depending on whether the magnitude of the influence on *x*_{i} decreases or increases as *x*_{j} increases. Finally, the exact mathematical form of the relationship between *x*_{i} and *x*_{j} may be ascertainable. Notice that these different kinds of relations constitute a hierarchy of increasing specificity: directed *a*_{ij} are Boolean, sign-directed *a*_{ij} are obviously directed, monotonic *a*_{ij} are sign-directed, etc.
^{4}

This hierarchy concerns the form of the *interactions* between variables, and should be distinguished from relations that might hold between different *values* they take. If *a*_{ij} is positively monotonic, concave, and *a*_{ji} = 0 for instance, we know there is some interaction between *x*_{i} and *x*_{j} (because *a*_{ij} is Boolean non-zero), that *x*_{j} influences *x*_{i} but not *vice versa* (because \({a_{ij}\ne a_{ji}}\) =0), that increases in *x*_{j} induce increases in *x*_{i} (because *a*_{ij} is positively signed), that the larger the *x*_{j}-increase the larger the *x*_{i}-increase (because *a*_{ij} is positively monotonic), and that the magnitude of the effect on *x*_{i} decreases as *x*_{j} increases (because *a*_{ij} is concave). Yet none of this indicates the relative size of the values of *x*_{i} and *x*_{j}. Knowing *a*_{ij} is positively monotonic concave is consistent with *x*_{i} and *x*_{j} having approximately identical or radically different values.

It is sometimes possible to determine how values of different variables are qualitatively, but not quantitatively, related. The most imprecise of these relations are *categorical*, whereby values (which need not be numerical) are sorted into a finite number of mutually exclusive categories. If the categories exhaust the set of values being considered, they partition the values into equivalence classes, one of the simplest being the three-category partition of numerical values into positive, negative, and null (zero). Categorical classification is typically used to represent properties distinguished by qualitative features that cannot be put on a common quantitative or ordinal scale. Classifying individuals of a biological population into distinct phenotypes is an example of categorical classification.

Qualitative properties of different variables may support a ranking of their values. Whether one species consumes another species in a biological community, for example, is often ascertainable through observation or by study of their qualitative physiological and behavioral properties. Population sizes of consumed species are also usually greater than those of their consumers. Facts like this are designated by *ordinal* relations between variable values and can be incorporated into model equations to enhance representational precision. Ordering the values of the variables representing consumed and consumer species, for instance, could help model the consumer-resource dynamics of biological communities containing them more accurately.

Sometimes the degree one variable value is greater or less than another can be determined more precisely even when full quantification of the difference is not possible. While ordinal relations rank values, *interval* relations order the magnitudes of differences between values in terms of ratios of these differences. Ordinal relations specifying that *x*_{i} < *x*_{j} and *x*_{k} < *x*_{l} for instance, do not entail the difference between the former is greater or less than that of the latter. Interval relations provide that additional information by specifying that\({\frac{x_j -x_i}{x_l -x_k }}\) is greater or less than unity. These relations, moreover, are unique to increasing linear transformations of the values. Further specificity can be attained with other relations, for instance, ratios of variable values, or restrictions on the range of values a variable can take. Similar to the hierarchy of different types of qualitative interactions between variables mentioned above, relations between variable values also comprise a specificity hierarchy: ordinal relations are (even if trivially) categorical, and interval relations are ordinal.

*interactions*between variables (Fig. 1A). The other concerns relations between variable

*values*(Fig. 1B).

^{5}The full mathematical form of a model provides the highest degree of precision about the forms of interactions between its variables. For each interaction between two variables, the mathematical form shows what position, if any, it has in the first hierarchy (Fig. 1A). A completely parameterized mathematical model—that is, one with quantitative values assigned to all model parameters—also maximizes precision about the relations between variable values. It shows what place, if any, these relations have in the second hierarchy (Fig. 1B).

The focus thus far has been on interactions between and values of variables, but the same two hierarchies also characterize the *a*_{ij}. Consider interactions between them first. The magnitude and form of some *a*_{ij} may depend on others, and as the latter vary the former may change. If *a*_{ij} represents the intensity of interaction between two coevolving species, for instance, it may change over time and this may inhibit, enhance, or change the form of interactions between other species. Depending upon the specificity by which these interactions between *a*_{ij} can be ascertained, they can be represented by Boolean, directed, signed-directed, monotonic, or concave relations. Similar to relations between variable values, relations between values of different *a*_{ij} may also be determinable. An ordinal relation like *a*_{ij} > *a*_{kl} for constant *a*_{ij} and *a*_{kl}, for example, designates that the linear influence of *x*_{j} on *x*_{i} is stronger than that of *x*_{l} on *x*_{k}. If a specific *a*_{ij} represents a strong symbiosis between two mutualist species, for example, its magnitude will be greater than the *a*_{ij} representing weak interactions between species.

This discussion shows how systems can be modeled with varying degrees of precision by a wide variety of different types of qualitative relations. Qualitative modeling concerns what properties can be derived from these qualitative relations absent the precise mathematical form of the system’s model. The character and scope of different qualitative modeling methods depend upon the types of qualitative relations they analyze, and on any conditions their application requires. There are therefore many different possible qualitative methods, loop analysis being just one. Section 3 shows how loop analysis uses qualitative properties of models to evaluate the local stability of systems and how they respond to changes in parameter values.

## Loop analysis

*n*variables

*x*

_{1},...,

*x*

_{n}, its dynamics can be represented by

*n*differential equations:

*c*

_{j}are parameters, and 1≤

*i*≤

*n*. Loop analysis focuses on the relations among variables at equilibrium these equations specify. At a point equilibrium \({{\bf x}^\ast = < x_1^\ast ,\ldots,x_n^\ast > ,\left( {\forall i} \right)\left[ {\frac{\hbox{d}x_i (t)}{\hbox{d}t}=0} \right]}\) and relations among variables are given by the

*n*×

*n*Jacobian matrix evaluated at

**x**

^{*}, i.e., the matrix

**A**of constant interaction coefficients \({a_{ij} =\left. {\frac{\partial f_i }{\partial x_j }} \right|_{x_j =x_j^\ast }.}\) Thus, loop analysis evaluates system properties in the local neighborhood of

**x**

^{*}.

**A**and directed graphs (digraphs).

^{6}The variables

*x*

_{1},...,

*x*

_{n}correspond to vertices of a digraph. The coefficients

*a*

_{ij}correspond to digraph edges that represent the effects of

*x*

_{j}on

*x*

_{i}, specifically how increases in

*x*

_{j}affect

*x*

_{i}.

^{7}The matrix\({\left[ \begin{array}{lll} a_{11}&a_{12}&a_{13}\\ a_{21}&a_{22}&a_{23}\\ a_{31}&a_{32}&a_{33}\\ \end{array} \right],}\) for instance, corresponds to the digraph of Fig. 2.

A *path* is a series of directed edges from vertex *j* to vertex *i* that crosses no intermediate vertices more than once; a *loop* is a path from a vertex to itself. The number of edges in a path (loop) determines its length and *disjunct* paths (loops) share no vertices. *a*_{12}*a*_{21} and *a*_{33} in the digraph of Fig. 2, for instance, are disjunct loops of length 2 and 1.

*a*

_{11}and

*a*

_{22}) and one length-2 loop (

*a*

_{12}

*a*

_{21}). Levins (1975b, 20) generalized this relationship between determinants and loops to

*n*-order matrices:

*n*coefficients forming

*m*disjunct loops, and

**L**

_{m,n}is the set of all such products in the digraph of

**A**.

*L*(3,5), for instance, is the product of the five coefficients of three disjunct loops.

*k*” in

*n*-variable systems:

*k*≤

*n*. For instance,\({F_1 ({\bf A})=\sum\limits_{i=1}^n a_{ii} ,}\) which is the sum of the diagonal elements of

**A**, the length 1 loops. The underlying basis of this definition is the idea that feedback is a process by which changes in variables induce changes in other variables that then affect the variables originally changed (Puccia and Levins 1985). Positive feedback

*enhances*change: increase in variables induces further increase, and decrease induces further decrease. Negative feedback

*counteracts*change: increase induces decrease, and decrease induces increase. Notice that this definition is entirely consistent with a simple pendulum being a negative feedback system, contrary to Wimsatt’s (1970) more stringent adequacy conditions on a definition of feedback.

**x**

^{*}is locally asymptotic stable if systems beginning in a local neighborhood of

**x**

^{*}return to

**x**

^{*}after “small” perturbations. Two mathematical results form the theoretical basis of Levins’ use of loop analysis to evaluate local stability. The first is Lyapunov’s ([1892] 1992) proof that

**x**

^{*}is locally asymptotic stable iff:

_{i}(

**A**) designates the real part of λ

_{i}, the

*i*th eigenvalue of

**A**. The second is that (5) can be evaluated with the Routh-Hurwitz criterion (see Gantmacher 1960). With these results, Levins (1974) ingeniously used (3) and (4) to formulate the Routh-Hurwitz criterion in loop-theoretic terms. As reformulated, the criterion requires: (i) negative feedback at every level; and, (ii) stronger feedback at lower levels than higher ones.

^{8}

Local stability analysis is commonly taken to indicate how systems respond to small perturbations of finite duration called “pulse” perturbations.
^{9} The goal of press perturbation analysis, on the other hand, is to evaluate how systems respond to perturbations of indefinite duration, sometimes called “press” perturbations (Schmitz 1997).
^{10} Within the context of loop analysis, a press perturbation is represented by a change in one parameter (or variable) from one constant value to another, and system response is gauged by how the equilibrium values of variables subsequently change. Changes in equilibrium values of different fish population sizes that are caused by a change in the constant supply rate of some nutrient to a lake, for example, could be studied with this type of analysis.

*x*

_{h}(1≤

*h*≤

*n*) following a change in a parameter

*c*

_{k}, i.e., to determine\({\frac{\partial x_h }{\partial c_k}.}\) Assuming the system reaches a new equilibrium, application of the chain rule to (2) from above yields (see Puccia and Levins 1985, Appendix):

*i*,

*h*= 1,...,

*n*are the

*a*

_{ij}of the Jacobian matrix

**A**from above. Since\({\frac{\partial f_i }{\partial c_k }}\) represents the change in the parameter, which is given, and\({\frac{\partial x_h }{\partial c_k }}\) is unknown, the general form of the problem is to solve the matrix equation:

**x**where\({{\bf x}=\left[ {\frac{\partial x_h }{\partial c_k }} \right]}\) and\({{\bf b}=\left[ {-\frac{\partial f_i }{\partial c_k }} \right].}\) Solving for\({\frac{\partial x_h }{\partial c_k }}\) using Cramer’s rule shows that:

**A**with\({-\frac{\partial f_l }{\partial c_k }(l=1,\ldots,n)}\) replacing the

*i*th column.

*p*

_{ij}

^{k}be the product of the

*k*−1 directed edges that form a path of

*k*vertices from

*j*to

*i*, and let

*F*

_{n-k}

^{−p}be the feedback of the complementary subsystem of the remaining

*n*−

*k*vertices that are not part of this path. These definitions, (3), and (4) entail:

^{11}where

*c*

_{k}is the parameter that changes to a new constant level. (8) and (9) are equivalent, and therefore provide the same information about changes in variables following press perturbations. These equations can also both assess changes in variable values when there is an external input or output of a constant amount into or from one variable. This is done by simply interpreting

*c*

_{k}as the input or output from the relevant variable. (8) and (9) can be used to predict, therefore, the effects a constant migration of individuals of some species into or out of a biological community (designated by a change of a constant amount in one variable) may have on the equilibrium values of other species (designated by other variables).

*a*

_{ij}> 0, \({x_{j}\rightarrow x_{i}}\) designates a positive effect of

*x*

_{j}on

*x*

_{i}; if

*a*

_{ij}< 0,

*x*

_{j}

*x*

_{i}designates a negative effect of

*x*

_{j}on

*x*

_{i}; and if

*a*

_{ij}= 0, no edge exists, which designates a null effect between the variables. If \({\left[ \begin{array}{lll} a_{11}&a_{12}&a_{13}\\ a_{21}&a_{22}&a_{23}\\ a_{31}&a_{32}&a_{33}\\ \end{array} \right]=\left[ \begin{array}{lll} {-1}&{-1}&{-1}\\ {+1}&0&0\\ 0&{+1}&0\\ \end{array} \right],}\) for example, the corresponding sign digraph is Fig. 3.

This pattern could represent interactions between three species for which there is certainty about their sign and direction, but little is known about their functional form.

Generally, only the sign and direction of interactions between variables are determinable from a sign digraph. The locality restriction of loop analysis (see above), however, provides additional information about the interactions because it entails that the *a*_{ij} take constant real values. Loop analysis consequently only applies to sign digraphs with constant *a*_{ij}; sign digraphs with non-constant coefficients cannot be analyzed using loop analysis. Thus, the locality restriction significantly enhances the analytic power of loop analysis because it effectively specifies the precise functional form of interactions between variables. If *a*_{ij} is constant and positively (negatively) signed, the influence of *x*_{j} on *x*_{i} is positively (negatively) monotonic. In fact, the constancy and sign entail that the precise mathematical form of the interaction is positively or negatively linear. Thus, the locality condition required by loop analysis specifies the precise mathematical form of the *a*_{ij} of sign digraphs. With respect to the two hierarchies presented in Fig. 1, therefore, loop analysis is only qualitative in the weak sense that it presupposes the exact mathematical form of the interactions between variables (linear) but nothing about the numerical magnitude of their slope (the constant value of the *a*_{ij}).

With this information, it is sometimes possible to determine whether a sign digraph with constant coefficients is or cannot possibly be locally asymptotic stable, and how it will react to press perturbations. Local asymptotic stability can be assessed somewhat laboriously with Levins’ loop-theoretic formulation of the Routh-Hurwitz criterion in the same way as when quantitative values of the *a*_{ij} are available. Without quantitative values, however, the criterion usually provides ambiguous evaluations. The local stability of the sign digraph in Fig. 3, for instance, depends upon whether |*a*_{11}*a*_{12}| > |*a*_{32}*a*_{13}|, which usually cannot be determined from their signs.

*a*

_{ij}. Quirk and Ruppert (1965) were the first to find such patterns. They proved a sign digraph with constant coefficients such that (\({\forall i)(a_{ii} < 0}\)) is locally asymptotic stable if:

*sign-stable*and is locally asymptotic stable for any additional specification of the values of its edges, such as quantification, that preserve their sign pattern. Other criteria found thus far weaken the requirement that (\({\forall i)(a_{ii} < 0}\)),

^{12}but the intricacy of the additional conditions they require yield negligible insight into what properties of sign patterns guarantee local asymptotic stability.

Formulated in terms of (7), loop-theoretic press perturbation analysis assesses whether the signs of the entries of **x** can be determined when **A** and **b** are only specified as sign-directed. If they can, the system represented by (7) is called *sign-solvable*. The apparent generality of this property—systems remain sign-solvable for any specification of the entries of **A** and **b** preserving their sign pattern—prompted efforts to find qualitative criteria for it, similar to Quirk and Ruppert’s qualitative criterion for stability. Lancaster (1962) first found a sufficient condition for sign-solvability, and Bassett et al. (1968) first found necessary and sufficient conditions. Unfortunately, these conditions and others found thus far are rather complicated and consequently do not produce the same degree of insight into what sign pattern properties are required for sign-solvability as Quirk and Ruppert’s result did for sign-stability.
^{13}

How variables of sign digraphs respond to press perturbations can be somewhat laboriously assessed with (9) in the same way as when quantitative values of the *a*_{ij} are available. Similar to loop-theoretic local stability analysis, however, the results of such calculations are also usually ambiguous. Further information about the values of the *a*_{ij} is usually required to determine how variables will change. (9) also suffers from the same opacity as the loop-theoretic Routh-Hurwitz criterion: it does not clearly indicate what sign patterns, if any, entail how variables respond to press perturbations, regardless of any further specification of the *a*_{ij}.

## Merits of qualitative modeling

One frequently cited advantage of qualitative over quantitative analysis is that the data required to estimate parameter values and quantitatively test model predictions is often unavailable and cannot be acquired feasibly, especially within the budgetary constraints of most scientific projects (Levins 1966). Within ecology, this limitation is well-known to ecologists attempting to estimate typical population and community model parameters like intrinsic growth rate, carrying capacity, competition coefficients, etc. Qualitative properties of models, however, are more easily ascertained. Economists, for instance, regularly focus on sign-directed relations because they are confident of the sign and direction of interactions between most major parts of the economy but doubt their functional form can be determined more precisely (Hale et al. 1999). Natural history plays a similar role in ecology (Lane 1998; Dambacher et al. 2003b). Familiarity with the natural histories of species is often sufficient to determine the qualitative character of their interactions and qualitative relations between their population sizes.

For some subjects, moreover, quantitative modeling is inappropriate not because the requisite data cannot be collected feasibly, but because the data and phenomena being studied are essentially qualitative. This is often the case in the social and decision sciences. It is usually a mistake, for instance, to insist that a precise mathematical function accurately represents the imprecise belief states or preferences of typical real-world agents. This is not merely because psychology has yet to provide a complete account of these mental states, or because performing the numerous experiments and studies that would be required to determine the precise function of an agent (assuming it exists) would be practically infeasible. Principally, it is because there is little reason to expect human behavior is accurately represented by such a function.
^{14}

Incommensurability, to take one example, precludes comparisons of different evaluative criteria altogether. Decisions that fare differently on incommensurable criteria consequently cannot be ranked, ordered by interval relations, or otherwise compared with a precise utility function or by any other means. Different qualitative methods and a large literature have been developed to deal with this constraint systematically, and other qualitative methods have been developed to deal with other irreducibly qualitative features of agent behavior (see Figuera et al. 2005).
^{15} The expectation that the often unprincipled, opportunistic, and sometimes even inconsistent belief states and preferences of humans are best modeled by precise mathematical functions seems to reflect the quantitative prejudice that originally motivated Levins to develop loop analysis (see Section 1).

Accommodating these features of scientific research—(i) the inappropriateness of quantitative analysis for some phenomena, and (ii) the infeasibility of acquiring sufficient quantitative data—is one advantage of qualitative over quantitative modeling. Another advantage is the generality of any result it establishes. Precisely because qualitative modeling requires less commitment to details—e.g., the exact mathematical form of relationships between variables—its results generalize more broadly. The scope of generalization depends upon the qualitative properties and assumptions needed to establish the result. If only qualitative properties that provide low levels of specificity about the system’s structure are needed, such as Boolean or directed relations, knowing relatively little about the system is sufficient to derive the result. The result therefore holds for any system exhibiting this minimal structure, regardless of how the system’s structure is specified with more detail. Satisfaction of Quirk and Ruppert’s criterion, for instance, entails a system is sign-stable and further specification of the ordinal, interval, or quantitative values of the *a*_{ij} consistent with the criterion does not jeopardize this fact. This advantageous feature of qualitative modeling is a consequence of a tradeoff between precision and generality in scientific modeling (Weisberg 2004).

Another advantage of qualitative modeling is that the understanding of phenomena it provides is less susceptible to the drawbacks of common idealization techniques used in quantitative modeling. Qualitative models are idealized representations of phenomena *if* the phenomena being modeled are best described quantitatively. The dynamics of a mechanical system like a combustion engine, for instance, are most accurately modeled with quantitative variables, parameters, and parameterized mathematical equations.
^{16} Unlike idealizations often made in quantitative modeling, however, qualitative idealizations are not *mis*representations. Qualitative models avoid problem (ii) from above by veridically representing system properties at low degrees of specificity. Lack of specificity, however, does not jeopardize the verity of what a qualitative model says holds of a system. Call this type of idealization *specificity-idealization*.

The predominant idealization strategy of quantitative modeling, *veracity-idealization*, is different. To circumvent problem (ii), features of a quantitative model are often simplified to make it describable by precise and tractable mathematical equations with fewer or more easily estimable parameters. Rather than simplify by decreasing specificity, these simplifications usually involve making unrealistic assumptions about the modeled system, such as representing discrete system components or processes as continuous, treating some interactions and propagation of their effects as instantaneous, ignoring some system parts and interactions, etc.
^{17} The differential equations used to model biological communities, for example, often incorporate several of these simplifications. The hope is that these intentional misrepresentations will not distort the salient features of the system. If they do not, the system can be represented by simplified but tractable mathematical equations, thereby achieving representational precision.

The drawback of veracity-idealization, however, is that it may mischaracterize important features of the modeled system. For the same reason a model is idealized in the first place, moreover, the data collection needed to test model predictions and thereby ensure that important system features are not mischaracterized may be practically impossible. Consequently, the apparent understanding a veracity-idealized model provides about a system may be misleading and, ultimately, of negligible value.
^{18} Qualitative modeling avoids this problem, at the obvious expense of the ability to generate precise predictions, by focusing on qualitative properties of systems that are ascertainable with relative certainty. This ties the understanding qualitative modeling provides to more realistic, though imprecise representations of systems.

There are thus clear advantages of qualitative over quantitative modeling. In addition, it should be clear from Section 3 and Justus (2005) argues in detail that there is nothing conceptually or methodologically problematic, or mathematically unrigorous about qualitative analysis, loop analysis being the case study (*cf*. Orzack and Sober 1993). Some putatively qualitative methods of analysis are conceptually and methodologically problematic, such as visual assessment of randomness or the “fit” between a curve and plotted data.
^{19} The hope is that loop analysis, which is not flawed in this way, will be a powerful qualitative modeling technique, capable of establishing significant results about a wide variety of systems modeled in science. Some of Levins’ informal descriptions of loop analysis give this impression, for example:

This chapter will present a new method of analysis for the study of partially specified systems. The method, called “loop analysis,” proves particularly useful for examining the properties of biological communities in which the interactions between species can be specified in a qualitative but not a quantitative way. I will show that much can be deduced about the structure and behavior of such systems merely by using the sign of an interspecific interaction (Levins 1975b, 16);

and, “This procedure [loop analysis] may be the only one available in partially specified systems” (Levins 1974, 137). Subsequent expositions of loop analysis by others have sometimes reinforced this impression, for instance:

Central to understanding cumulative effects and the complex causality is our ability to diagram the important causal relationships and understand how they interrelate to cause system change. Qualitative network analyses, such as loop analysis, show the most promise in achieving this end...Loop analysis can also predict qualitative changes in all the network variables for a set of network stresses. (Lane 1998, 137–138)

Although Levins’ technical expositions of loop analysis are always explicit about its applicability conditions, the limitations these conditions impose on the method receive little attention. The next section makes clear, however, how serious these limitations are.

## Limitations of loop analysis

Loop analysis is the qualitative counterpart of Lyapunov’s (1992) indirect method.
^{20} As such, it only applies to a system in the local neighborhood of a point equilibrium. This restricted scope, Section 3 pointed out, allows the specific functional form of relationships between variables (positive or negative linear, or null) to be determined from the signs of coefficients in the Jacobian matrix. Without this information, local asymptotic stability and system response to press perturbation cannot be assessed loop-theoretically.

The restriction to local neighborhoods of point equilibria is therefore essential to loop analysis, but it also narrowly constrains the method’s scope. One limitation this imposes is that loop analysis only applies to models with an equilibrium. Equilibrium models, however, do not adequately represent several different kinds of systems. In ecology, for instance, the predominant focus on equilibrium models in the 1960s and 1970s has been supplanted with the recognition that non-equilibrium models best represent many types of ecosystems (Wiens 1984; Chesson and Case 1986). Although these and other non-equilibrium systems cannot be evaluated with loop analysis, their non-equilibrium dynamics clearly do not preclude qualitative modeling in general from providing insights into their dynamics.

A more serious limitation is that loop analysis only provides information about a system in the local neighborhood of a point equilibrium. Except in rare cases, this says nothing about how a system responds to real-world pulse or press perturbations. The problem is that local neighborhoods are infinitesimal neighborhoods; strictly speaking, they have no finite extension. Any real perturbation of a system describable as being at a point equilibrium will therefore displace the system out of the local neighborhood of its equilibrium, rendering loop analysis inapplicable. It is unreasonable to suggest, for example, that a system like a biological community initially at equilibrium would remain in an *infinitesimal* neighborhood of it following typical real-world perturbations, such as frosts, droughts, increases in certain compounds (as caused by fertilizer runoff or a chemical spill for instance), etc. Thus, even the common characterization of local stability analysis as assessing how systems respond to “small” perturbations is misleading (Hastings 1988).
^{21} If a system is linear, local stability entails global stability, but most systems studied in science are undeniably not linear. For these systems, loop analysis of local stability therefore provides little or no insight into how they respond to real-world perturbations.

These are drawbacks of any method of analysis focusing on local neighborhoods of equilibria, qualitative or quantitative. Some of the specific conditions application of loop analysis requires, however, also limit its scope, especially with respect to press perturbation analysis. Besides assuming a system is initially at a point equilibrium, loop-theoretic press perturbation analysis also requires the system evolve toward an asymptotically stable equilibrium after the press perturbation begins (Bassett et al. 1968). Without this asymptotic behavior, system variables may not tend towards the new equilibrium values consistently, which would prevent their direction of change from being reliably predicted (see Section 3). For a given system, therefore, only effects of those press perturbations, if any, that yield a new asymptotically stable equilibrium can be evaluated with loop analysis.

This places some substantial restrictions on the method. For one, a system can only establish a new asymptotically stable point equilibrium if the perturbation causes the parameter to change from one constant value to another. Loop analysis is limited, therefore, to evaluating the effects of a narrow range of perturbations. System responses to parameter changes that vary in magnitude or sign, i.e., the magnitude or sign of\({\frac{\partial f_i }{\partial c_k }}\) varies, usually cannot be evaluated loop-theoretically.
^{22} Many, perhaps most, press perturbations affecting real-world systems induce complex, non-constant changes in parameter values.

Besides this restriction, the asymptotic dynamics of the stable equilibrium must also be sufficiently strong for the results of loop theoretic press perturbation analysis to be informative. If they are not, the perturbed system may eventually establish a new equilibrium (assuming, often unrealistically, that subsequent perturbations do not preempt this), but on a time scale far exceeding those of scientific interest for the system being studied. Results derived from loop analysis about how variables will change would therefore be uninformative because the change would be too slow to be empirically detectable or considered different from zero for practical purposes in the time scales allocated for analysis. This obviously detracts from the import of the results of loop analysis for such systems. It is also problematic because loop analysis cannot assess the rate variables approach asymptotically stable equilibria, which is determined by the magnitude of the smallest real part of the system’s eigenvalues (see [4] above), called the dominant eigenvalue. By focusing strictly on sign-directed relations, loop analysis can only show that the real parts of eigenvalues are positive or negative; it cannot determine their magnitude. Thus, that system variables approach equilibrium sufficiently quickly, specifically that it is not so slow as to render the results of press perturbation analysis unimportant, cannot be established loop-theoretically.

The limitations considered thus far concern the scope of loop analysis. Within this scope, there are also significant limits to what loop analysis can show. One is that the sign pattern of a sign digraph alone is rarely sufficient to establish local asymptotic stability or how variables respond to press perturbations. Additional information—ordinal, interval, or quantitative specification of the *a*_{ij}—is usually required to derive these properties. Loop analysis partially ameliorates this difficulty by pinpointing the additional information required to determine whether a system exhibits these properties. This helps focus the limited resources available to scientists towards those measurements needed to make such a determination. The quantity of information required, however, amplifies with model complexity. Specifically, the frequency additional information is required and the number of *a*_{ij} for which it is increases dramatically with the number of model variables, parameters, and non-null interactions between them. For models with more than five variables or predominantly non-null interactions between them, Dambacher et al. (2003b, 81) suggest that, “signed digraph analysis grows not just exponentially, but factorially.”
^{23} Samuelson (1947, 26), an economist, was perhaps the first to recognize the difficulty this posed for qualitative analysis: “It can be seen then that purely qualitative considerations [in this case sign-directed relations] cannot take us very far as soon as the simple cases are left behind.”

Levins recognized this difficulty (Puccia and Levins 1985, 90 and 119–120). Since he intended loop analysis to be a method for modeling complex systems, Levins stressed the utility of lumping distinct variables into aggregate variables, and other model simplification strategies. Based partly on common sense and partly on his wealth of experience in scientific modeling, Levins offered several suggestions about when such simplifications are appropriate (Puccia and Levins 1985, 79–84). These suggestions were intended as heuristic guidelines for model simplification, and they eliminate *unnecessary* model complexity; they do not, nor should they, eliminate model complexity necessary to represent the complex system being modeled adequately. The guidelines cannot, therefore, circumvent the limitations of loop analysis when applied to complex systems.

The limitations of loop analysis considered above do not uniquely single loop analysis out among other qualitative or quantitative modeling methods as the one with serious limitations. Systems with large numbers of highly interconnected parts present formidable challenges for any modeling strategy, especially if they are governed by non-linear dynamics. Other methods of qualitative analysis developed by economists, for instance, also make restrictive assumptions, such as requiring the system be an optimizing process or that it exhibit purely competitive dynamics.
^{24} Like these methods, however, the restricted scope and weak evaluative power of loop analysis for complex systems limit the role it can play in scientific modeling.

## Conclusion

Scientific models must be assessed with respect to the purposes for which they were developed. These purposes are often specific: to find the factors responsible for some pattern, to identify the best way to manipulate a system to achieve some result or do so most efficiently, etc. This specificity is a characteristic feature of engineering research where the goal is usually to resolve a particular problem, such as to increase the tolerance of designed structures like buildings to environmental disturbances like earthquakes, or to minimize the heat generated by a microprocessor.

Specificity is also crucial when the modeling objective is to resolve a clearly defined scientific debate. Resolving such debates usually requires devising and testing quantitative models with definite mathematical structure to evaluate competing, specific hypotheses about some phenomenon. Doing so often involves large amounts of quantitative data, and quantitative methods of analysis for testing the models’ predictions. Qualitative analysis is consequently ill-suited to this task. In the context of the adaptationism debate, for instance, Orzack and Sober (1993) were correct to point out that only proper testing of quantitative models could resolve the debate given the nature of the issue.

The inability to *resolve* such debates does not, however, manifest a conceptual or methodological failing, nor does it entail qualitative analysis cannot *contribute* to their resolution. As Levins (1974, 1998; Puccia and Levins 1985) has repeatedly emphasized, qualitative analysis complements rather than supplants quantitative analysis. Orzack and Sober (1993, 542–543) observed, for example, that qualitative optimality models have enhanced biologists’ understanding of the process of adaptation, even though they cannot decide the adaptationism debate.

It might be thought that the limited scope of loop-theoretic evaluation of stability and the effects of press perturbation indicates an unavoidable lack of generality of qualitative modeling methods, and an accordingly narrow role for them within science in general. This conclusion should be resisted. As noted in Section 2, loop analysis is only one of a wide array of different possible qualitative methods. There are thus two ways of achieving greater generality with qualitative modeling. First, new qualitative methods may be developed with greater scope. With respect to qualitative stability analysis, for instance, what seems to be needed is a qualitative version of Lyapunov’s direct method. The basis of this method is Lyapunov’s (1992) proof that the existence of a special kind of function in a region of a model’s state space entails the stability of an equilibrium within that region.
^{25} Whether such a function, a Lyapunov function, exists for a given model depends upon the mathematical structure of the model; different Lyapunov functions are required for different types of models. Since there is no restriction on the size of the region of the model’s state space besides the existence of the Lyapunov function, the direct method provides an effective way of evaluating stability properties of systems outside the local neighborhood of an equilibrium. Its qualitative counterpart would involve proving the existence of a Lyapunov function using only qualitative properties of the form of interactions between variables (and parameters) and relations between their values. It would then sometimes be possible to demonstrate the stability of equilibria in *non*local domains based solely on qualitative model properties, which loop analysis cannot do.

Second, even if there are insurmountable obstacles to this general approach,
^{26} several different qualitative methods may be developed that, taken together, have a broad scope. To some degree, this seems to be the current state of affairs in mathematical economics (see Athey et al. 1998; Hale et al. 1999). There is little a priori or otherwise to suggest that both these strategies will inevitably fail. Despite the limitations of loop analysis, therefore, Levins’ emphasis on the merits of a qualitative approach to scientific modeling remains prescient and compelling.

More generally, a (*n* + *m*) × (*n* + *m*) matrix that also represents the effects of the *m* parameters on the variables could be constructed. Since variables do not affect parameters [(ii) above], the lower left quadrant of this matrix would be composed of zero entries. If the parameters do not influence one another, as is commonly assumed in scientific modeling, the lower right quadrant would also have zero entries. Only for simplicity of presentation, I have focused on relations between variables.

If the *F*_{i} are partially differentiable, **A** may be represented by the familiar Jacobian matrix (see Section 3).

A function *f* is monotonic increasing if\({\left( {\forall x_1,x_2 }\right)\left( {\left( {x_1 < x_2 } \right)\to \left( {f(x_1 ) < f(x_2 )} \right)} \right)}\) . The inequality of the consequent is reversed for a monotonic decreasing function.

The following counterexample shows that a positive sign-directed interaction from *x*_{k} to *x*_{j} is not equivalent to a monotonic increasing interaction. Consider two increases in *x*_{k}, from *x*_{k}^{0} to *x*_{k}^{1} and from *x*_{k}^{0} to *x*_{k}^{2} such that *x*_{k}^{1} < *x*_{k}^{2}. Since the interaction from *x*_{k} to *x*_{j} is positive sign directed, these increases in *x*_{k} will induce increases in *x*_{j}, to *x*_{j}^{1} and *x*_{j}^{2}, respectively. It may not be the case that *x*_{j}^{1} < *x*_{j}^{2}, however, as required if the interaction were increasing monotonic.

Wright (1921) probably first recognized this relationship between matrices and directed graphs, and consequently between determinants and loops (see below), in his development of path analysis: a method by which the effects of different factors inducing variation in a variable can sometimes be distinguished. Levins (1974) recognized this relationship independently of Mason (1953), who appreciated it in his analysis of the dynamics of electrical circuits, and Maybee’s (1966) similar recognition of the relationship in economics (see Bassett et al. 1968).

The definition of the Jacobian matrix in terms of partial derivatives requires *a*_{ij} be defined in terms of how *increases* in *x*_{j} affect *x*_{i}.

Press perturbation analysis is equivalent to a method of qualitative analysis developed in the field of ‘comparative statics’ in economics (see Athey et al. 1998).

Walley (1991, Ch. 5), for instance, argues that imprecise probabilities best represent uncertainty, lack of information, ambiguity, and other aspects of real-world agents’ belief states.

Another, more controversial, example is whether interpersonal utility comparisons are possible, and if so, with what degree of precision they can be done (see Weirich 1984; Harsanyi 1976).

This is not the case for the essentially qualitative phenomena discussed above, such as agent beliefs and preferences.

Typically, scientists in one field appropriate models developed in another to make these simplifications. This practice involves a judgment that, despite their apparent differences, the systems being analyzed share a similar structure. It is also frequently motivated by the desire to import scientific rigor from a science with a secure theoretical foundation to a science without one. Most of the highly idealized models developed by Lotka, Volterra, and their contemporaries in the early development of mathematical ecology, for instance, originated in physics (Scudo 1971; Gasca 1996).

*If* adequate data are available to test them, predictions of veracity-idealized models can serve as a means towards developing improved models with fewer unrealistic features (Wimsatt 1987).

There are some exceptions. In the special case that the perturbed parameter is strictly increasing or decreasing, subsequent changes in variables can be approximated by determining their change at successive values of the parameter (Levins 1974, 1975b), but only if the new system would establish an asymptotically stable point equilibrium at each new parameter value. The range of parameter values for which this holds depends upon how the dynamics of the system change as the parameter continues to increase or decrease. Flake (1980) has also shown that loop analysis can be extended using Laplace transforms to analyze the effects of some time-varying and periodic perturbations. This extension requires the precise mathematical form of these perturbations be specified, however.

It is unspecified whether this refers to the expected number of interaction coefficients appearing in the conditions required for local asymptotic stability, the number of loops in the model, or something else.

With respect to a qualitative version of Lyapunov’s direct method, this may be the case. The construction of a Lyapunov function for a given model usually depends on the precise details of the model’s mathematical form.

## Acknowledgements

A Josephine De Kármán fellowship supported this research. Thanks to Alexander Moffett, Michael Weisberg, and an anonymous reviewer for helpful comments.