Computer Modeling of Theory: Explanation for the Twenty-First Century

Abstract

Twenty-first century computing has given us new ways of doing science. Old notions of elegance and simplicity are not completely outmoded. But they need to be complemented by a greater awareness of the complexity of social, economic and demographic systems, and the realization that simple models, while tractable and intellectually satisfying, often are not adequate to the task at hand – explanation, prediction, or policy guidance. In the words of one biologist, ‘It is only now that we have the ability to do complex calculations and simulations that we are discovering that a great many systems seem to have an inherent complexity that cannot be simplified…’ (Rowe GW, Theoretical models in biology: the origin of life, the immune system, and the brain. Oxford University Press, Oxford, 1994).

Based on a presentation to the Centre Methodos Symposium on The Explanatory Power of Models in the Social Sciences, 14–17 November, 1998, Louvain-la-Neuve, Belgium; originally published in The Explanatory Power of Models, Robert Franck [ed.] Boston: Kluwer Academic Publishers, pp. 245–265.

1 Introduction

The words theory, model, and explanation are used in different ways by different writers. Complete agreement on their meanings among natural scientists, social scientists, philosophers of science, engineers and others seems unlikely, since meaning depends partly on context and on discipline-specific conventions. Accepted meanings often depend on subject matter, and on the purposes of research. In practice, a theory, model, or explanation – or a good theory, model, or explanation – for a physicist or chemist may differ in some respects from a theory, model, or explanation for a biologist, a meteorologist, or a demographer. These differences may appear all the greater if one looks at the use of models and theories in practical decision making, as in engineering or policy formation.

The question of which view of theory, models, and explanation is the ‘ correct’ view seems less relevant than the question of which view promises to be more fruitful for mainstream social science. In this chapter I argue for the fruitfulness of an approach to theory building, modeling, and explanation which (a) emphasizes the abstract character of all theories and models, indeed of all human knowledge, and (b) judges the value of a model or theory pragmatically, in terms of the purpose for which it is being used. All scientific knowledge involves the abstract representation of concrete, real-world phenomena, and as such involves simplified representation of indefinitely complex realities. All knowledge distorts, since all knowledge simplifies. The crucial question is whether a model or theory, be it simple or complex, is adequate to some well-defined scientific purpose – prediction, explanation, or intervention.

An approach to theory, models, and explanation developed from these assumptions will have the following features:

• 1. Explanation is seen first and foremost as a logical exercise, assuming, of course, some empirically described phenomenon to be explained. Something is explained provisionally when it follows logically from a theory or model.

• 2. The propositions in a theory or model used in explanation are to be judged initially on their clarity, logical coherence and explanatory power, less on their origins. Propositions need not be, or be derived from, primitive axioms (as in economics). Nor must they be, or be derived from, empirical generalizations or verified empirical laws (as in the covering-law view of explanation). Theoretical propositions can express hunches, prejudices, or guesses, so long as they are clearly formulated and coherent. Models can contain unobserved or even unobservable variables.

• 3. The empirical question is not whether a given theory or model is ‘true’ or ‘valid’. As suggested above, all theories or models simplify, that is distort and falsify the real world. The empirical issue is whether a logically sound model is close enough to some part of the real world to represent it with sufficient accuracy for some well-defined purpose. This is why the logical exercise mentioned just above (point 1) is only provisional. Given this relativity to purpose, no theory or model can be said to be absolutely better than another, even though scientific consensus at any point may favor one theory or model over others.

• 4. In social scientific explanation, theories and models may be more rather than less complex. There will be less emphasis on elegance and parsimony (theoretical aims inherited from medieval philosophy, early physics, and pure mathematics) in response to the complexity of social and historical phenomena. Simple models should not be abandoned, any more than classical mechanics has been abandoned in physics. They still are useful for many purposes, including teaching, and can provide well-developed building blocks for more complex models. But many social scientific and policy purposes will require more complex models.

• 5. Given (a) more complex theories and models and (b) the need for logical rigor in their manipulation (see point 1 above), natural language, logic and, analytic mathematics will be supplemented by computer languages in theoretical work. Apart from preliminary or simplified statements or sketches, theories and models will be stated in the form of computer code, or initially pseudo-code, using systems dynamics software, a programming language like R or C++, or some other suitable computer software.¹ Entailments, predictions, implications of the theories and models will be calculated by the computer, rather than inferred or eyeballed by the investigator.

In the further development of these ideas, the words theory and model are used more or less interchangeably. Both are abstract representations of some part of the real world.

For some authors, a theory is often general, in the dictionary sense of ‘involving only the main features rather than the precise details.’ A model moves toward specification of details. Market theory says that a rise in price tends to reduce demand and to increase supply. A model of the theory would define the relationships more precisely by characterizing the appropriate curves (e.g., convex upward or downward) or by giving specific mathematical functions. But the difference is one of degree. Model, theory, and theoretical model – all are abstract representations of reality. The phrase theoretical model is an apt one, in that it emphasizes this fact.

Explanation, as noted above, involves the logical deduction of some concrete phenomenon or class of phenomena from a theoretical model that has been further specified to apply to particular real-world situation. Relevant historical or cultural context must be added in the form of concrete facts. Parameters in mathematical functions or algorithms must be given specific values. Whether the theoretical model used is appropriate to the phenomenon to be explained is a matter of judgement and of purpose. But the fact that a model is inappropriate to explain one phenomenon does not mean it is inappropriate for all – such a mismatch does not necessarily invalidate or falsify a model.

In a social scientific context, it is important to distinguish theoretical models from empirical or statistical models. The latter focus on the mathematical representation of a set of empirical observations or data, for example, a multiple regression model. Theoretical models represent theoretical ideas, which in turn are meant to represent some real-world system. Many computer simulation models, often confused with statistical models because of their technical and quantitative character, are better seen as theoretical models.²

But whatever concepts and terminology one uses, theories, models and explanations crafted using the above approach remain imaginative structures – created by the human mind - and in empirical science they must be related to the real world in some way. The logical empiricist tradition, which has so dominated late twentieth century social science, including demography, would speak of testing the validity or truth of theories and models, seeking to base them on empirical generalizations, and to reject models that fail to agree with the data.

A different approach is found in the writings of a few social scientists, notably in the work of the political scientist Eugene Meehan (1968), and in what has come to be known as the semantic school of the philosophy of science (see, for example, Giere 1999 or Cartwright 1999). Giere questions whether so-called scientific laws are in fact true. Cartwright views them as the product of models – which she describes as ‘ nomological machines ’. Models and theories are not based on laws; they create them. Meehan thinks the search for laws in social science invites failure, since universal empirical generalizations are so rare. All argue, in different language, that the theoretical model (Meehan uses the word system) is the primary representational device in science. A good model is, in the words of Giere, ‘true in the sense that an explicit definition is true.’ Validity is a formal property of a model, relating to its logical clarity and internal consistency. The approach to empirical issues is more pragmatic, focusing on the fit between a theoretical model and some portion of the real world, with the adequacy of the fit judged in terms of the purpose of the analysis.

None of the key ideas in this chapter is novel, although in combination the approach they suggest is not common in day-to-day work in the social sciences. Typically, theoretical work is largely qualitative, discursive, and verbal, and empirical work consists largely of description and statistical modeling , often only loosely linked to theory. There remains a widespread conviction that a theory must rest on valid empirical generalizations, and that theories and models must be tested and verified or falsified.³

The equation of explanation with strict logical inference from explicans to explicandum is common enough in methodological and philosophical writings of those who adhere to an empirical science model of social science, as opposed to those who favor understanding or intuition. But it is encountered more in statements of methodological principle than in everyday work, where the logic often is informal, if not casual (see Platt 1964).

The notion that the origin of theoretical ideas or hypotheses is not so important so long as they are eventually compared to some part of the real world is classic Popper. A particularly forceful development of this view, as well as the equation of explanation with formal inference, is found in an early work of Meehan, Explanation in Social Science: A System Paradigm (1968) – a stimulating book that appears to have been neglected⁴ – arguing against the covering law model of explanation . I refer to Meehan often because his ideas seem to me to clarify so many of the issues posed in the present volume, from the standpoint of a working social scientist. As noted above, more developed statements of these ideas are found in recent works by philosophers of science such as Giere (1999) and Cartwright (1999).

The move towards complexity in models and theories is well under way in many scientific fields. With the ‘discovery’ of chaos, it has become fashionable, but that does not mean it is mistaken. The quote that opens this chapter is representative of a changing, somewhat qualified outlook on simplicity as a scientific virtue (see also Waldrop 1992).

The idea of computer simulation or modeling of theoretical ideas is now commonplace in the physical and biological sciences, and would need no special attention were it not for the fact that it is still looked on with some suspicion in many social science circles.

The systems dynamics approach to modeling has a special history in social science, and has been roundly criticized, especially in connection with The Limits to Growth study ( Meadows et al. 1972). My suggestion that we make more use of the software emerging from this tradition does not imply that we must accept all the ideas of ‘general systems theory,’ or that we should imitate some of the early misuse of the software. I view the software simply as a practical tool for the more rigorous statement and manipulation of theoretical ideas, a tool that goes beyond logic or analytic mathematics in power and flexibility. In any case, it is a tool that is accessible to the average demographer, who does not now have, and is unlikely to have in the future, the mathematical abilities of the average physical scientist or engineer, especially the ability to work with complex systems of non-linear equations.⁵

2 Explanation as Logical Inference

The influence of multivariate statistical methods has been so powerful in empirical social science that for many the word explanation tends to be equated with the phrase accounting for variance. X helps explain Y if it has a large and significant regression coefficient; a good explanation has a large R². This is a valid use of the word, but in many respects an impoverished one (see Abbott 1988; Lieberson 1985). Such an approach limits explanation to variables that are not only measurable but have actually been measured, if only indirectly (e.g., latent variables models). It tends to discourage, or at least often does not require, deep thought about process or mechanisms. It easily leads to atheoretical analysis; or at best theory is pared down to fit statistical models, in what might be called Procrustean empirical modeling.

The idea of explanation as inference of the explicandum from a set of premises is common enough among some social scientists of a theoretical bent, such as mathematical economists, exchange theorists (see Homans 1967), and those who subscribe to logical empiricism and the ‘ covering law model of explanation.’

A particularly striking development of the idea is to be found in the work of Meehan noted above (Meehan 1968; see also Meehan 1981). Meehan dismisses the covering law approach to explanation (he cites Braithwaite, Hempel, and Nagel) and offers as an alternative what he terms the ‘system paradigm of explanation’. Proposed at a time when computers and computer simulation were still in their infancy, his approach provides a convincing methodological foundation for computer modeling as a powerful tool for the statement and manipulation of behavioral theory.

Meehan characterizes the covering law or deductive paradigm of explanation as follows:

An event is explained when it can be related to an established ‘empirical generalization ’ or ‘general law’ according to the canons of formal logic; generalizations in turn are explained when they can be deduced from sets of ‘higher’ generalizations or theories. The structure is held together by the rules of formal logical inference. The elements of the structure, the empirical generalizations or laws, must be available before explanation is possible. If the relation is to be deductive, generalizations must take the form ‘all A is B’, or in some few cases ‘n percent of A is B’. Other forms of generalization are not amenable to deductive inference. The generalizations, in other words, are established independently of the explanation; they are subject to ‘empirical verification’ or test (1968, p. 9). Meehan’s characterization of the covering-law approach to explanation agrees with that of Miller (1987) who comments: ‘ Covering laws in the deductive nomological pattern must be, not just general, but empirical, subject to disconfirmation by observational data’ (p. 19).

Meehan’s criticism of the deductive approach is twofold: (a) ‘… the paradigm collapses or merges the logical and the empirical aspects of explanation …’ (the classic problem of induction); and (b) ‘…the definition attaches no weight to the purposes for which explanations are sought or to the manner in which they are used’ (1968, p. 10).

In practice, Meehan finds adherence to the deductive paradigm of explanation severely restricting for social science, since there are so few empirical laws or ‘nomic empirical generalizations’ of the sort the paradigm requires, which leads to a pessimistic view of the explanatory capacities of the social sciences. Meehan sees the situation not so much as a reflection of ‘the weakness of social science’ as of ‘the limited usefulness of the deductive paradigm’ (1968, p. 3).⁶

Simply stated, Meehan’s notion of explanation of an observation or event involves: (a) creation of a logical structure of variables and their relationships, a structure which logically implies or entails the event; (b) demonstration that there is correspondence or ‘isomorphism’ between the logical structure and the real-world context in which the event is embedded.

In its emphasis on a ‘formal logical structure’, Meehan’s approach bears some resemblance to traditional mathematical modeling, the axiomatic structure of modern economics, and the covering law model of explanation.⁷ The difference lies in the origin and character of the propositions in the formal structure, to be discussed below.

The following summary statement by Meehan captures the spirit and essence of his approach:

The instrument that makes explanation possible is here called a system. It is defined as a formal logical structure, an abstract calculus that is totally unrelated to anything in the empirical world. The system, as a system, says nothing whatever about empirical events; it generates expectations within its own boundaries (p. 48).

Expectations are generated through strict logical inference:

Since the instrument used for explanation of empirical events must contain timeless or general propositions, and since it must generate expectations that can be warranted or justified, there is really no choice of instruments involved. Of all the structures that [one] can create, only a formal calculus can create warranted expectations. Given the axioms of a formal logical structure, certain conclusions are inescapable; if the axioms are accepted, the conclusions can be denied only by self-contradiction […]. Barring errors in calculation, the entailments of a logical system are necessarily and indefeasibly true (p. 48).

Explanation is a form of applied mathematics or calculation, using a closed formal system (1968, pp. 62, 125). Meehan’s system’ is the equivalent of a theoretical model, as defined above.⁸ In either case, it remains abstract. The system must be further specified (Meehan speaks of the formal calculus as being ‘loaded’) in order to apply to and explain a concrete event or class of events.

The notion of a computer template provides a contemporary analogy. A spreadsheet program for making a cohort-component population projection is an abstract algorithm. It must be ‘loaded’ with data for a particular country before it can be applied to predict or to explain past demographic dynamics. But first and foremost, it must be a logically and mathematically correct template. A similar idea is found in Miller’s (1987) distinction between theories and ‘event explanations’: ‘… a theory is a description of a repertoire of causal mechanisms; a theoretical explanation, an explanation appealing to instances of such a repertoire’ (p. 139).

A theoretical explanation that does not logically entail its explicandum, or a theoretical prediction that is not logically implied by its theoretical model, are non-starters. If an explanation is not logical, it is not an explanation. Many, perhaps most, social scientists would agree with this view in principle. But as Platt pointed out many years ago (1964), in his classic chapter on ‘strong inference’, it often is neglected in everyday scientific work as researchers ‘feel their way to conclusions’ or investigate hypotheses ‘loosely based on’ or ‘suggested by’ theory. And, as noted above, explanation often is equated with explaining variance.

The origins of Meehan’s ideas are somewhat obscure. His 1968 work, quoted above, makes no reference to philosophers of science with which he agrees – only those with which he disagrees. Later work (1981) contain passing reference to early proponents of the ‘ semantic’ school of philosophy of science.

Recent works by philosophers of science, notably Ronald Giere’s Science Without Laws (1999), share Meehan’s abandonment of the logical positivist’s search for scientific laws, which is seen as futile, even in the physical sciences. Nancy Cartwright (1983, 1999) writes of ‘how the laws of physics lie’, and argues that so-called scientific laws are derived from models – what she terms ‘ nomological machines ’ – more than from nature. Giere comments that most scientific laws are not universal, and that they are not even true: ‘… understood as general claims about the world, most purported laws of nature are in fact false’ (1999, p. 24).

For Giere, the primary representational device in science is not the law but the model , of which there are three types: physical models ; visual models; and theoretical models . Models are inherently abstract constructions that attempt to represent certain features of the real world. They are true only in the sense that definitions are true. The question of whether they are empirically true is irrelevant, since they cannot be. The world is too complex. The relevant question is whether they correspond to some part of the real world a] in some respects, b] to a sufficient degree, and c] for certain well-defined purposes.

Giere gives the example of the standard model for the earth-moon system. This model is adequate to describe and account for the moon’s orbit around the earth and for putting a rocket on the moon, but is inadequate to describe the Venus-earth system. The prototype of scientific knowledge is not the empirical law , but a model plus a list of real-world systems to which it applies.

A model explains some real-world phenomenon if: (a) the model is appropriate to the real-world system in the three respects noted above; and (b) if the model logically implies the phenomenon, in other words, in the phenomenon follows logically from the model as specified to fit part of the real world. It would never occur to most physical scientists to add the second condition. But in social science, including demography, we are so accustomed to loose inference that its explicit statement is necessary.

With respect to evaluating a model’s fit to a real-world system, Giere dismisses the notion of strict logical inference from data, as in Popperian falsification: ‘… the relative evaluation of rival paradigms is not something that can be reduced to any sort of logic. It is fundamentally a matter of choice by scientists acting as individuals within a scientific community’ (1999, p. 119). And, ‘Coming to hold that one model fits better than others is not a matter of pure reasoning or logical inference. Rather it is a matter of making a decision’ (p. 7, emphasis in original).

But note that the prior evaluation – whether there is a logical connection between theory or model and an explicandum – isa matter of logic and inference. Logic cannot evaluate the empirical adequacy of several logically cogent theories or models. But it can dismiss a model that is not logically cogent, that is, that does not imply or entail the explicandum, what might be called ‘logical falsification’.

3 The Origins of Theoretical Ideas Are Irrelevant

One of the best definitions of theory that I have encountered was in a small English dictionary in the library of The Netherlands Institute for Advanced Study in Wassenaar:

Conceptions, propositions or formula [as relating to the nature, action, cause origin of a phenomenon or group of phenomena] formed by speculation or deduction or by abstraction and generalization from facts [exact reference unknown].

The definition properly does not limit theoretical ideas to one source or origin. Most important, it does not limit them to valid empirical generalizations, as in the covering law approach to explanation. Theoretical propositions arrived at ‘by abstraction and generalization from facts’ are included, but others sources of theoretical propositions are not excluded. In fact, it doesn’t matter where one’s ideas come from in science, so long as they are reasonably clear and coherent, relevant to the matter at hand, have explanatory power, and are subject to empirical evaluation.

This is a central theme in the work of Popper , who emphasizes the imaginative and creative character of theorizing and hypothesis formation, counterbalanced by a strict program of attempts at falsification. ‘Bold ideas, unjustified anticipations, and speculative thought, are our only means for interpreting nature…. And we must hazard them to win our prize’ (1959, p. 280).

This is also a central theme in the work of Meehan, as described earlier. If explanation in social science must rely on empirically valid generalizations (‘ covering laws’), its scope will be severely restricted at the outset and the explanatory enterprise will barely leave the ground. In his system paradigm of explanation, ‘timeless or general propositions are assumed to belong to the logical rather than the empirical world’ (1968, p. 32).

If an explanation does not require valid empirical generalizations as premises (along with relevant factual premises), it of course cannot contain statements which are clearly false. But one should be careful not to throw out the baby with the bath water. In demography, for example, the documentation of several instances (both nations and provinces) in which secular decline in aggregate fertility was not preceded by substantial mortality decline (Coale 1973) is commonly taken to invalidate classic ‘ transition theory ’ or any other explanatory theory to the extent that they assume such a time sequence is universal, or that mortality decline is a necessary condition for fertility decline. But the generalization applies to most historical cases in Europe and to virtually all recent non-European cases. And it should find a place in theories or explanatory models pertaining to the cases to which it applies (Giere’s model plus the list of real-world systems to which it applies). It is too powerful a finding to discard because of exceptions.

This probably is the motivation behind past efforts to introduce probabilistic generalizations into covering law explanations. This is a difficult technical issue. But suffice it to say that the kind of strict logical inference that Meehan favors becomes more difficult with probabilistic statements. If A and B are positively but less than perfectly correlated and B and C are positively but less than perfectly correlated, it does not necessarily follow that A and C are positively correlated. Or if A causes B with moderately high probability, and B causes C with moderately high probability, the occurrence of A will lead to the occurrence of C with much lower probability – that is, there is a good chance that C won’t happen at all.⁹

It is not clear how to reconcile the notion of explanation as strict logical inference with the introduction of non-universal, probabilistic premises, since explaining that something had a high probability of happening is not quite the same as explaining that it happened. One approach might be to keep theoretical models deterministic and therefore subject to strict logical inference, but to introduce stochastic elements into specific explanations. Meehan finesses the problem by urging resort to ceteris paribusassumptions, as does the more recent work by Cartwright (1999). Perhaps something deeper is at work, namely a backing off from the explanatory and predictive standards of ‘celestial mechanics’, which, when all is said and done, may not be quite suited to biological and human systems (Ekeland 1988).

4 Towards More Complexity

The influence of a traditional view of classical physics on our notions of good science is nowhere more evident than in the commonplace that theory should strive for elegance and simplicity. The physicist Steven Weinberg has written (1980): ‘Our job in physics is to see things simply, to understand a great many complicated phenomena in a unified way, in terms of a few simple principles’. A hundred years earlier, J. Willard Gibbs had written: ‘One of the principal objects of theoretical research in any department of knowledge is to find the point of view from which the subject appears in its greatest simplicity’ (quoted in Tanford 1978). The idea has philosophical and theological origins with William of Ockham – after all, God is the one explanation for everything. It pervades physics from Newton right up to the present.

In social science, the self-conscious quest for elegant models is most pronounced in mainstream economics, based as it is on three basic axioms. A classic methodological reference is to Milton Friedman (1953), who not only favors explanation with as few axioms as possible, but with human behavioral axioms that are counter-intuitive. That the quest for parsimonious explanation continues to influence thought is to be seen in a paper on fertility theory (Friedman et al. 1994). The authors argue that their theory of fertility is better than the standard microeconomic theory because it is based on only two axioms rather than three.

In sociology, Jasso (1988) holds to the reasonable view that other things equal, a theory that can explain many things with relatively few assumptions is better than one that requires more assumptions to explain the same things. There is a certain common sense to this way of thinking – why use a shotgun to kill a fly? But a reasonable notion of efficiency in explanation may become an obsession, with as much emphasis on simplicity as on explanation. Moreover, what will work in one field of study may not work in another. Only time will tell, but it may well be that biological and human systems are indeed more complicated than those studied by physicists and chemists. It already is clear that many natural systems are more complicated than those designed by engineers. Even within a discipline, demography, for example, some systems and models will be more complex than others. The exponential function applied to population growth is a relatively simple model. A model of the social, cultural, and economic changes involved in demographic transition (in turn involving mortality, fertility, marriage, and migration) will be appreciably more complex, except at the most general, almost tautological level, as will an individual-level model of demographic decision making.¹⁰

So, if the reality is more complex , then our theories and explanations also must be more complex, at least for some purposes This is the force of the quote from Rowe at the beginning of this paper. It is the theme of Wunsch’s paper (1995) ‘God gave the easy problems to the physicists’.

There seems little doubt that the sciences of the future will work with theories and models of greater complexity, and indeed this already is happening. The ultimate ideal of theoretical elegance no doubt will remain, based as it is on human aesthetics and common sense notions of efficiency. Simple models will be studied as sources of insight, will serve some explanatory purposes, and will remain the starting point for students. But for many purposes – for explanation, prediction, and practical application – useful theoretical models will be more complicated than we are accustomed to. The greater complexity will arise on several fronts. There will be more variables in models, including theoretically important variables for which empirical data are sparse or non-existent. The functional relationships will often be non-linear. The models will be inherently dynamic, with feedback processes. And there will be stochastic elements.

5 Manipulating Complex Systems

This greater complexity will strain the analyst’s ability to derive logical implications of model assumptions using ordinary language and logic. Similarly, model manipulation will often exceed the capacity of analytic mathematics. The obvious tool for the manipulation of such complex models is numerical simulation by computer. In an introduction to an issue of Science on computers and fluid dynamics, the authors comment:

Efforts to understand the formation of stars, the motions of ocean and atmosphere that control our weather, and other fundamental processes on Earth and in space face a double challenge. Data on these phenomena are often sparse, and they are governed by complex fluid motions. Thus they are tailor-made for study with a computer. (Hanson and Appenzeller 1995, p. 1353)

Gould and Tobochnik (1996), writing of computer simulation in physics, comment: ‘Asking the question “How can I formulate the problem on a computer?” has led to new formulations of physical laws and to the realization that it is both practical and natural to express scientific laws as rules for a computer rather than in terms of differential equations’ (p. 4).¹¹

I have argued elsewhere (Burch 1997a) that demography – and by implication other fields of social science – faces a similar challenge, and need to pay far more attention to computer modeling, in both research and training, than has been the case up to now. Computer modeling is the only way to work with complex models while preserving logical rigor. The alternatives are rigorous simple reasoning or less than rigorous complex reasoning.

Meehan is cautious about how complex effective explanatory systems can be. His acquaintance with early examples of computer simulation apparently only underlined this caution:

…logical limits preclude the development of large and cumbersome theories that contain a great many variables. Most of the complex structures that have been produced in the computer era are actually very weak and unreliable, and their internal operations simplistic. Good theories are likely to contain only two or three variables, with an elaborate set of limiting conditions. If the system is characterized by interactions among variables [feedback], those limits must be maintained. Calculi containing four variables with feedback are for all practical purposes impossible to calculate. If that number is increased to five, calculation becomes impossible in principle. This argues strongly against the development of theories containing dozens of interacting variables. (1981, p. 123)¹²

Such a view of early attempts at computer simulation is understandable given the ambitious character of some early models , notably, macroeconomic models of whole economies and ‘world models’, as in The Limits to Growth. With greater experience, we now understand that there are limits to the amount of substantive complexity that can be effectively dealt with even with the most modern computer hardware and software. There are limits to the intelligibility of overly complex models, limits to control over internal processes (we don’t really know if the model is behaving the way we intended it too), and even limits to error-free programming.¹³

But computer modeling does allow for the rigorous manipulation of systems considerably more complex than those that can manipulated by means of traditional logic and analytic mathematics. These more complex systems can be dynamic (rather than static or based on equilibrium assumptions); they can contain non-linear relationships; and they can contain feedbacks and stochastic elements. Practical limits on the number variables no doubt exist, but they are much higher than the two or three variables that Meehan speaks of. It seems that modern computer modeling greatly extends the potential power of Meehan’s ‘system paradigm of explanation’ – by expanding our ability to deduce, in strict logic, our explicanda from formal explanatory systems.

6 Relating Theoretical Models to the Real World

Meehan’s explanatory system is a formal system in the same sense as plane geometry. It consists of well-defined variables and well-defined relationships between or among them (propositions), such that the system can be rigorously manipulated to infer implications or entailments. The variables do not need to relate to observables. The propositions do not need to have been empirically verified. They can themselves be formal; they can result from hunch or intuition; they can, but need not be, inspired by propositions widely accepted as empirical generalizations. The first requirement for an explanation is that the thing to be explained follows logically from the formal system. The second requirement is that the formal system, when ‘loaded’ or further specified to relate to a given empirical situation, is adequately ‘isomorphic’ with respect to that situation. It is not enough that the model’s predicted outcomes match the explicandum; in some sense the whole model must match the whole empirical situation:

When a system is applied to an empirical situation it is not enough to show that one particular entailment of the system can be found in the empirical situation…. The aim is to match the total explanatory system with an empirical situation so that all of the entailments of the system have empirical counterparts in observation. The goal in explanation is a perfect match or fit between a complete system and a description rather than a logical fit between a single event and a general proposition, as in the deductive paradigm. (1968, pp. 50–51)

Meehan is firm in his rejection of ‘black box’ explanations . An explanation must contain an account of the causal mechanism producing the outcome. It is not enough to show than an explanatory system can predict outcomes; Meehan makes a sharp distinction between explanations and forecasts. An explanation must relate to causes, and it must fit empirical reality in a broad sense:

The nature of the isomorphism required is immutable and unambiguous. The whole structure must fit the observation. It does not suffice to show that some of the implications of the calculus appear in the observation. That result can be produced using false assumptions. But assumptions that are known to be false and to be imperfectly isomorphic to observation cannot be incorporated into theories. They are only useful for producing forecasts or predictions. The point is vital, particularly for criticism of much of the work in economics and econometrics. (1981, pp. 89–90, emphasis in original)

Unlike a prediction, which enables us to anticipate an outcome and adjust to it, explanation, given its clear causal structure, also provides a basis for intervention in a system and control of the outcome, at least in principle.

The meaning of Meehan’s concept of isomorphism is not always clear.¹⁴ Certainly no conceptual system or model can completely match a real-world system, with its infinite complexity. Speaking of the laws of nature, Giere notes that laws contain ‘… only a few physical quantities, whereas nature contains many quantities which often interact one with another, and there are few if any isolated systems’ (1999, p. 24).¹⁵ At times Meehan seems to be speaking of such a complete match. At other times, the emphasis seems to be on a correspondence between all the logical implications of the system, and specific empirical findings. He is concerned with accepting too readily a model that predicts a few empirical observations correctly, but not all or most.

Giere would view the requirement of ‘perfect isomorphism’ as too strong, and in any case unrealistic. The degree of fit need only be sufficient to one’s purposes, in line with the three requirements noted earlier.

But there is no easy way to demonstrate isomorphism. This problem of how to assess the relationship between complex simulation models and empirical data has plagued the practice of computer modeling from the beginning, and has yet to be adequately resolved. It is one of the chief reasons why mainstream empirical social science has tended to hold simulation at arm’s length, as being ‘made up’ rather than based directly on data, as are statistical models.

Insofar as they are supposed to refer to some part of the real world , computer models or simulations need to be empirically assessed in some way. This seems an obvious point, but it often has been overlooked by proponents of simulation, especially enthusiasts. Just because a computer model is complex and convincing, and produces plausible results, it is not therefore a realistic representation of some part of the world. That it produces results using a computer or produces results that are numerically precise (in table or graph) – even less does this guarantee the suitability of a model, despite a widespread mystique with respect to numbers and computers.

A strong tradition of computer modeling that can claim special relevance to social science problems is the ‘systems dynamics ’ school, originating at MIT in the late 1960s and early 1970s, and associated especially with the names of Jay W. Forrester (author of such works as Urban Dynamics, 1969), and of Dennis L. and Donella H. Meadows – famous or infamous, depending on one’s view – for The Limits to Growth. The systems dynamics school has generated a large literature, both general works and simulations of specific systems, and has helped foster the development of software specifically designed for the modeling of dynamic systems with feedback.¹⁶

It is characteristic of much of the literature of the system dynamics school that more attention is paid to the building of models than to their relationship to the real world. A basic hardback text from the MIT group (Roberts et al. 1983), for example, a work of over 500 pages, contains no chapter on testing, validation, parameter estimation, goodness of fit; indeed, these words don’t even appear in the index. This exclusion apparently is deliberate. The authors include ‘model evaluation’ as one of the phases in the model-building process, and comment:

… numerous tests must be performed on the model to evaluate its quality and validity. These tests range from checking for logical consistency, to matching model output against observed data collected over time, to more formal statistical tests of parameters used within the simulation. Although a complete discussion of model evaluation is beyond the scope of the book, some of the important issues involved are presented in the case examples… (p. 9).

The main technique of model evaluation is the demonstration that the model fits one or more empirical time series of outputs. If the model can generate the output reasonably closely then it is considered a good model. Whatever the intent, it is hard for the reader to avoid the impression that testing a model against real world data is less interesting and less important than model building.

An earlier work from the same group (Richardson and Pugh 1981) makes clear that the emphasis on model building rather than model estimation or testing goodness of fit reflects a deep-seated attitude towards scientific and policy analysis, one somewhat at odds with traditional statistical methodology: ‘The systems dynamics approach to complex problems…takes the philosophical position that feedback structures are responsible for the changes we experience over time. The premise is that dynamic behavior is the consequence of system structure’ (p. 15, emphasis in original). That is, if one has the structure right, the details (e.g., specific parameter values) don’t matter so much. And later:

…experience with feedback models will convince the reader that model behavior really is more a consequence of structure than parameter values. One should therefore be more concerned with developing the arts of conceptualization and formulation than finding ultimate parameter selection methods. Our advice for beginners would be to estimate parameters with good statistics [data] but not Statistics [mathematical methods]. In the systems dynamics context the latter are a collection of power tools that just might cut off your intuition [p. 240].

In general, they are skeptical about the value of ‘correlational approaches’ and standard regression techniques (ordinary and generalized least-squares) when dealing with dynamic models with feedback (pp. 238–39).¹⁷

Validating a model in this tradition , as noted above, is achieved primarily by comparison of model output of key variables with ‘reference behavior modes’, essentially actually observed time-series measures of the phenomena of interest. But still the greater emphasis is placed on ‘causal understanding’ – how does the process really work? Regression equations, with coefficients attached to a set of distinct factors to reflect their relative importance, are viewed as uninformative, at least as a representation of process in an underlying system. In Abbott’s (1988) words, they reject a ‘representational’ approach to linear regression models in favor of an approach that they feel accords better with our intuitions of how a system actually works.¹⁸

A later example in this tradition (High Performance Systems 1996) criticizes an econometric analysis of milk production, expressed as a function of GNP, interest rates, etc., because the model nowhere mentions cows, and a model of human births (as a function of birth control, education, income, health, religion, etc.) because the model nowhere mentions mothers (pp. 25–28).

Robert Hanneman pioneered in the application of systems dynamics to social and demographic modeling in Computer-Assisted Theory Building: Modeling Dynamics Social Systems (1988). But sociologists and demographers have been slow to follow his lead, to adopt systems dynamics as a standard analytic tool.

More recent work suggests that the systems dynamics approach itself has matured. Sterman’s 900-page text, Business Dynamics: Systems Thinking and Modeling for a Complex World (2000) is a prime example. He comments on the relevance of econometrics for systems modeling: ‘…[the] wise use of numerical data and statistical estimation is central to good systems dynamics practice, and more effort should be devoted to the use of these tools in simulation model development and testing. A whole chapter is devoted to this subject – ‘Truth and Beauty: Validation and Model Testing’ (pp. 845ff.).

Support for the value of the systems dynamics approach is also found in a careful re-examination of The Limits to Growth studies, The Limits to Growth Revisited, by Ugo Bardi , an Italian chemist (2011). Bardi argues convincingly that much of the criticism was based on misunderstanding of their purpose, to provide ‘what-if’ scenarios, not literal predictions. Some of the criticism was based on incorrect reading of tables. Seen from a modeling perspective, The Limits to Growth and subsequent updates were perfectly sound first attempt to model a very complex, but very important system of interrelationships among population, the economy, technology, and the environment. Greater understanding of these interrelationships and potential problems they may create may be said, without exaggeration, to be crucial to the survival of human society as we know it.

The intellectual history of the systems dynamics tradition remains to be written.¹⁹ Based on an incomplete review of the literature, I would hazard the guess that most proponents are theorists at heart, more interested in ideas about how and why things work the way they do, and less interested in the technical details of measurement, statistical modeling, parameter estimation – the lifeblood of contemporary empirical research.

A central part of the problem is that there is no clearly defined or ‘ neat’ processes for testing the fit of simulation models, processes analogous to goodness of fit measures for statistical models, of tests of significance and magnitude for coefficients attached to particular variables. Part of the difference arises from the fact that computer models may often contain variables for which there are no empirical measures. Another difference is that the computer model often assumes complex, non-linear functional relations between variables, and must postulate some value for key parameters, values which may not be firmly established through empirical research. A consequence is that it is often possible – or even easy – to modify a given model until it agrees with some empirical result, after the fact. That the model can yield predictions in close accord with empirical data is an important fact. But it does not prove that it is an appropriate model (much less the best model) for the data at hand. ‘Correct’ predictions can result from a model with incorrect assumptions and inputs. In any case, there may well be other models which predict the same empirical results as well or better.

My approach to this problem is to view the validation of a complex computer model of the sort I have been discussing as being closer to theory confirmation than to the estimation of statistical models, or the testing of one or a small set of specific hypotheses, as in many contemporary multivariate analyses. The process will be more like that described by Miller in his account of confirmation:

Confirmation, I will argue, is the fair causal comparison of a hypothesis with its current rivals. A hypothesis is confirmed just in case its approximate truth, and the basic falsehood of its rivals, is entailed in the best causal account of the history of data-gathering and theorizing out of which the data arose…. In arguments for hypotheses, as against textbook expositions of findings, the best scientists sound like honest, intelligent lawyers and like principled, mutually respectful people in engaged in political controversy (p. 155).

The overall process is thus a far cry from a chi-square or t test of a statistical model. The fact that a theoretical model agrees with some empirical data is important. But there also is the question whether the data adequately represent the part of the real world under investigation. In assessing theoretical models, unlike some specific empirical statistical analyses, definitive conclusions and closure are hard to come by. In any case, a theoretical model will always remain provisional, the best available until something better comes along.

But assessing a simulation model and estimating a statistical model are not mutually exclusive endeavors, even if they are different. The tension between the two approaches described above can be and needs to be reconciled, so that computer modeling leads to better empirical research and so that standard statistical methods may be of service in the evaluation of complex computer models. Several approaches can be mentioned.

A complex computer model or simulation can be tested in the classic way using the ‘hypothetico-deductive’ method. If the model is applied to a particular empirical situation, it predicts that certain empirical outcomes should be observable, perhaps a time series or a linear structure among variables. The failure to observe these outcomes leads to weaker acceptance if not outright rejection of the model as formulated and as applied to a particular case. Here the multivariate model is being used in the entailment sense (Abbott 1988). There is no thought that the statistical model represents the system or its mechanisms. Indeed, this use of statistics to test complex computer models may help avoid the very reification that Abbot is concerned with in the representational use of the general linear model.

An extension of this approach , used by Jacobsen and Bronsen (1995) to test a systems model of deviant behavior, might be called the ‘ multiple entailment’ approach. They compare model predictions of deviant behavior in Israel not just with one or two empirical examples (time series of deviant behavior) but with as many as 15. That model predictions are in close accord with empirical time series in 12 of the cases studied is taken as strong evidence for the plausibility of model of deviant behavior and the more general theory underlying it, at least as applied to Israel. Jacobsen and Bronson do not claim proof, however, expressing a view that would be consistent with those of Giere or of Miller sketched just above.

Note that both the above approaches focus on comparing predicted output with empirical output. More comprehensive approaches to model validation are emerging, approaches that echo Meehan’s use of the word ‘isomorphism.’ They are as concerned with the relation to empirical reality of the inputs and internal workings of the model as they are with the relation to empirical reality of outputs.²⁰

Each element of the model is examined separately for its empirical or logical validity. For instance, in a simple model of fertility (see Burch 1997b, combining ideas from Easterlin’s ‘socioeconomic theory’, and from Rosero-Bixby and Casterline’s diffusion model), many of the inputs are of empirical data from surveys, and deal with well-defined concepts.²¹ Others (e.g., natural fertility) can be estimated based on well-developed procedures in demography. Relationships among variables are often logically true (surviving children equals total children times survival probability), or seem to represent common-sense behavioral assumptions (e.g., that behavior is more apt to result from strong rather than weak motivation, or that behavioral responses to perceived external conditions are often delayed). At the end, only a few of the elements in the model are questionable or arbitrary, notably, the conceptualization and measurement of costs of fertility control. But overall, the model is close to reality at most points, as well as predicting empirical time series closely. Again, it is not proven, but its credibility and its applicability to one or more concrete cases is strengthened by a process of triangulation, or what Miller might refer to as a process of causal, comparative, and historical confirmation.

Hammel and Wachter have validated their SOCSIM model (a microsimulation model of household, family, and kinship) by showing that it can produce current population figures when run from 1900 to the present with the best available data for demographic input (see, for example, Wachter 1997; Wachter et al. 1997). This is similar to the approach taken by climate modelers, who try to predict current climate from long-term historical observations the presumed determinants, a task which has recently proven successful (R.A.K. 1997).

Recent versions of systems modeling computer software incorporate elements of this broader approach, in important steps towards reconciliation of the simulation and statistical approaches. ModelMaker and Vensim , for example, provide procedures for estimating a limited number of parameters in a model, given empirically observed output. Not all parameters in a complex model can be meaningfully estimated in this manner, and the procedure is not as cut and dried as least-squares. But, as in the case of my fertility model described above, when only a few parameters are unknown or in doubt, the procedure can be of great help.

Vensim also has a program feature which it calls ‘reality check’. If we know certain things that must be true in the modeled system, we can specify them as conditions and have the computer check whether they hold in model runs. As a simple example, a production model should produce zero widgets if the number of employees falls to zero. Some quantities in a model cannot increase without limit; others cannot drop below zero. If they do so, because of some complex set of interrelations within the model, then something is wrong with the model, and it must be re-worked. The model remains subject to common sense and sound scientific intuition.

The spirit of this approach to the evaluation of theories and models seems to me to accord with Giere’s model-based view of science, which he terms perspectival realism (1999, pp. 212–15, 240–41).²² He rejects reliance on the notion of isomorphism,interpreted strictly as a complete correspondence between scientific models and real-world systems.²³ He comments:

… models need only be similar to particular real-world systems in specified respects and to limited degrees of accuracy. The question for a model is how well it ‘fits’ various real-world systems one is trying to represent…. ‘fit’ is not simply a relationship between a model and the world. It requires a specification of which aspects of the world are important to represent and, for those aspects, how close a fit is desirable (p. 93).

His last remark comes close to Meehan’s emphasis on pragmatism in judging models and explanations – that is, judging them in terms of the purposes for which they will be used.

7 Concluding Comment

Computer modeling has been developed largely by mathematicians, statisticians, computer scientists, and engineers. It requires numerical inputs and specification of functional relations, and produces seemingly precise numbers and graphs. Not surprisingly, many social scientists associate computer modeling or simulation with quantitative, empirical social science, with ‘number crunching.’

Of the many types of models that have been constructed, many justify this association. But, I would argue, the association is not a necessary one. For twenty-first century social science, one of the most fruitful application of computer modeling technologies will be to the statement, manipulation, and evaluation of our more promising complex theoretical models. This application does not represent as sharp a departure from past practice as may appear at first. Computer models of theories can be used to generate empirical predictions (implications, entailments) to assess their applicability to real-world systems. Computer models can be used to explain, even if one takes the narrow covering law view of explanation, but even more powerfully if one takes a broader view of explanation, similar to those of Meehan, Giere or Miller, as sketched earlier.

Computer models can be used to generate precise numerical outputs. But working with them also tends to heighten one’s awareness that the precise numbers are largely the result of one’s assumptions. This can lead to a greater emphasis on broad qualitative results, an idea that has always characterized that quintessential tool of the hard sciences, differential equations.

The key point is that the computer and associated software has extended much more than our ability to do numerical computations. It has extended our powers of logical inference and reasoning. We can deduce the strict logical consequences or entailments of systems of propositions much more complicated than can be dealt with using logic or analytic mathematics.²⁴

These systems will be richer and more realistic than those of modern economics, for example, based on mathematical deduction from a limited number of axioms, some of them behaviorally questionable. They will be more flexible and intuitive than those permitted by the covering law approach to explanation, requiring verified empirical generalizations before one can even begin. Such theoretical systems have always existed in social science, but in the past their statement has often been less than clear, their logical manipulation somewhat less than rigorous, and their empirical assessment problematic.

The newer approach will lay to rest the notion that one must ‘let the facts speak for themselves’ – an empiricist bias that can be traced to no less a figure than Newton – Hypotheses non fingo. It also will break free from an excessive concern with simplicity – a reductionist bias that can be traced to William of Ockham, and has been perpetuated by the dominance of a physics model of science. There will be less concern with where the assumptions in a theoretical system come from – empirical evidence, intuition, even fantasies – so long as they can be and eventually are subjected to rigorous empirical evaluation. If the theoretical systems become complex rather than simple and elegant, so be it, if that is the character of the reality being studied.

In short, the computer will enable twenty-first century social scientists to match the breadth and depths of their insights with expanded powers of logical inference, leading to a true marriage of theory and empirical research.