Aligning simulation models: A case study and results
This paper develops the concepts and methods of a process we will call “alignment of computational models” or “docking” for short. Alignment is needed to determine whether two models can produce the same results, which in turn is the basis for critical experiments and for tests of whether one model can subsume another. We illustrate our concepts and methods using as a target a model of cultural transmission built by Axelrod. For comparison we use the Sugarscape model developed by Epstein and Axtell.
The two models differ in many ways and, to date, have been employed with quite different aims. The Axelrod model has been used principally for intensive experimentation with parameter variation, and includes only one mechanism. In contrast, the Sugarscape model has been used primarily to generate rich “artificial histories”, scenarios that display stylized facts of interest, such as cultural differentiation driven by many different mechansims including resource availability, migration, trade, and combat.
The Sugarscape model was modified so as to reproduce the results of the Axelrod cultural model. Among the questions we address are: what does it mean for two models to be equivalent, how can different standards of equivalence be statistically evaluated, and how do subtle differences in model design affect the results? After attaining a “docking” of the two models, the richer set of mechanisms of the Sugarscape model is used to provide two experiments in sensitivity analysis for the cultural rule of Axelrod's model.
Our generally positive experience in this enterprise has suggested that it could be beneficial if alignment and equivalence testing were more widely practiced among computational modelers.
KeywordsSimulation replication computational models alignment culture
Unable to display preview. Download preview PDF.
- Anderson, Paul A. and Gregory W. Fischer (1985), “A Monte Carlo Model of a Garbage Can Decision Process” in James March and Roger Weissinger-Baylon (eds.),Ambiguity and Command: Organizational Perspective on Military Decision Making. Pitman Publishing, Inc.: Marshfield, MA.Google Scholar
- Axelrod, R. (1995), “The Convergence and Stability of Cultures: Local Convergence and Global Polarization,” Santa Fe Institute working paper 95-03-028.Google Scholar
- Blake W. (1804/1974),Jerusalem, the Emanation of the Giant Albion. B. Quarich: London.Google Scholar
- Burton, R.M. and B. Obel (1995), “The Validity of Computational Models in Organization Science: From Model Realism to Purpose of the Model,”Computational and Mathematical Organization Theory, 1, 57–71.Google Scholar
- Carley, K. (1991), “A Theory of Group Stability,”American Sociological Review, 56, 331–354.Google Scholar
- Cavalli-Sforza, L.L. and M.W. Feldman (1981)Cultural Transmission and Evolution: A Quantitative Approach. Princeton, NJ: Princeton University Press.Google Scholar
- Cohen, M.D., J.G. March and J.P. Olsen (1972), “A Garbage Can Model of Organizational Choice,”Administrative Science Quarterly, 17, 1–25.Google Scholar
- Cyert, R.M., and J.G. March (1963),A Behavioral Theory of the Firm Prentice-Hall: Englewood Cliffs, New Jersey.Google Scholar
- Dutton, J.M. and W.H. Starbuck (1971),Computer Simulation of Human Behavior. Wiley: New York.Google Scholar
- Efron, B. and R.J. Tibshirani (1993),An Introduction to the Bootstrap Chapman and Hall: New York.Google Scholar
- Epstein, J.M., and R. Axtell (forthcoming 1955),Growing Artificial Societies: Social Science From the Bottom Up. The Brookings Institution: Washington, D.C.Google Scholar
- Friedkin, N.E. and E.C. Johnson (1990), “Social Influence and Opinions,”Journal of Mathematical Sociology, 15, 193–205.Google Scholar
- Harrison, J.R. and G.R. Carroll (1991), “Keeping the Faith: A Model of Cultural Transmission in Formal Organizations,”Administrative Science Quarterly, 36, 552–582.Google Scholar
- Hoel, P.G. (1962),Introduction to Mathematical Statistics. Third ed. Wiley: New York.Google Scholar
- Knepell, P.L. (1993),Simulation Validation, A Confidence Assessment Methodology. IEEE Computer Society Press: Los Alamitos, CA.Google Scholar
- Kramer, E.E. (1970),The Nature and Growth of Modern Mathematics, 2 Fawcett, Greenwich, Connecticut.Google Scholar
- Lane, D. (1993), “Artificial Worlds and Economics, Parts 1 and 2,”Journal of Evolutionary Economics, 3, 89–107 and 177–197.Google Scholar
- Latour, B. and S. Woolgar (1979),Laboratory Life: the Social Construction of Scientific Facts. Sage Publications: Beverly Hills.Google Scholar
- March, J.G. (1991), “Exploration and Exploitation in Organizational Learning,”Organization Science, 2, 71–87.Google Scholar
- Nowak, A., J. Szamrej and B. Latane (1990), “From Private Attitude to Public Opinion: A Dynamic Theory of Social Impact,”Psychological Review, 97, 362–376.Google Scholar
- Renfrew, C. (ed.). (1973),The Explanation of Social Influence: Models in Prehistory. Pittsburgh: University of Pittsburgh Press.Google Scholar
- Sabloff, J.A. (ed.). (1981),Simulations in Archeology. Albuquerque: University of New Mexico Press.Google Scholar
- Siegel, S. (1956),Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill: New York.Google Scholar
- Wuensche, A. and M.J. Lesser (1992),The Global Dynamics of Cellular Automata: An Atlas of Basin of Attraction Fields of One-Dimensional Cellular Automata. Addison-Wesley: Reading, MA.Google Scholar