Skip to main content
Log in

Simulation-Based Research in Information Systems

Epistemic Implications and a Review of the Status Quo

  • State of the Art
  • Published:
Business & Information Systems Engineering Aims and scope Submit manuscript

Abstract

Simulations provide a useful methodological approach for studying the behavior of complex socio-technical information systems (IS), in which humans and IT artifacts interact to process information. However, the use of simulations is relatively new in IS research and the current presence and impact of simulation-based studies is still limited. Furthermore, simulation-based research is quite different from other approaches, making it difficult to position and evaluate it adequately. Therefore, this paper first analyses the epistemic particularities of simulation-based IS research. Based on this analysis, a structured literature review of the status quo of simulation-based IS research was conducted, to understand how IS scholars currently employ simulation. A comparison of the epistemic particularities of simulation-based research with its status quo in IS literature allows to critically examine epistemic inferences in the respective research process. The results provide guidance for prospective simulation-based IS research through discussing the theory-based derivation of simulation models, as well as different simulation techniques, validation techniques, and simulation uses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. An earlier version of this literature review was previously presented at the 36th International Conference on Information Systems (Beese et al. 2015). In the paper at hand we not only significantly extend the literature review and use a completely different coding approach, but also focus on the additional insights that follow from the discussion of the epistemic particularities of simulation-based IS research.

References

  • Adner R, Pólos L, Ryall M, Sorenson O (2009) The case for formal theory. Acad Manag Rev 34:201–208. https://doi.org/10.5465/AMR.2009.36982613

    Article  Google Scholar 

  • Aminof B, Murano A, Rubin S, Zuleger F (2016) Automatic verification of multi-agent systems in parameterised grid-environments. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems. Richland, pp 1190–1199

  • Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79. https://doi.org/10.1214/09-SS054

    Article  Google Scholar 

  • Ashish AK, Aghav J (2013) Automated techniques and tools for program analysis. In: Proceedings of the 4th International Conference on Computing, Communications and Networking Technologies. pp 1–7

  • Axelrod R (1997) Advancing the art of simulation in the social sciences. In: Conte R, Hegselmann R, Terna P (eds) Simulating social phenomena. Springer, Heidelberg, pp 21–40

    Chapter  Google Scholar 

  • Axtell R, Axelrod R, Epstein JM, Cohen MD (1996) Aligning simulation models: a case study and results. Comput Math Organ Theory 1:123–141. https://doi.org/10.1007/BF01299065

    Article  Google Scholar 

  • Becker J, Niehaves B, Klose K (2005) A framework for epistemological perspectives on simulation. JASSS 8(4):1–16

    Google Scholar 

  • Becker J, Brocke J, vom Heddier M, Seidel S (2015) In search of information systems (grand) challenges. Bus Inf Syst Eng 57(6):377–390. https://doi.org/10.1007/s12599-015-0394-0

    Article  Google Scholar 

  • Bednar P, Sadok M (2015) A socio-technical toolbox for business systems analysis and design. In: 1st International Workshop on Socio-Technical Perspective in IS Development, pp 29–31

  • Beese J, Haki K, Aier S (2015) On the conceptualization of information systems as socio-technical phenomena in simulation-based research. In: Proceedings of the 36th International Conference on Information Systems. Fort Worth

  • Bélanger F, Cefaratti M, Carte T, Markham S (2014) Multilevel research in information systems: concepts, strategies, problems, and pitfalls. J Assoc Inf Syst 15(9):614–650

    Google Scholar 

  • Bichler M, Frank U, Avison D, Malaurent J, Fettke P, Hovorka D, Krämer J, Schnurr D, Mueller B, Suhl L, Thalheim B (2016) Theories in business and information systems engineering. Bus Inf Syst Eng 58(4):291–319

    Article  Google Scholar 

  • Birdsey L, Szabo C (2014) An architecture for identifying emergent behavior in multi-agent systems. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. Richland, pp 1455–1456

  • Boero R, Squazzoni F (2005) Does empirical embeddedness matter? Methodological issues on agent-based models for analytical social science. JASSS 8(4)

  • Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci 99:7280–7287. https://doi.org/10.1073/pnas.082080899

    Article  Google Scholar 

  • Bostrom RP, Heinen JS (1977) MIS problems and failures: a socio-technical perspective. Part I: the causes. MIS Q 1:17–32. https://doi.org/10.2307/248710

    Article  Google Scholar 

  • Bratley P, Fox BL, Schrage LE (2011) A guide to simulation. Springer, New York

    Google Scholar 

  • Burton RM, Obel B (2011) Computational modeling for what-is, what-might-be, and what-should-be studies—and triangulation. Organ Sci 22:1195–1202. https://doi.org/10.1287/orsc.1100.0635

    Article  Google Scholar 

  • Carley KM (2001) Computational approaches to sociological theorizing. In: Turner JH (ed) Handbook of sociological theory. Springer, US, pp 69–83

    Google Scholar 

  • Chang P-C, Lai C-Y, Lai KR (2006) A hybrid system by evolving case-based reasoning with genetic algorithm in wholesaler’s returning book forecasting. Decis Support Syst 42:1715–1729. https://doi.org/10.1016/j.dss.2006.02.014

    Article  Google Scholar 

  • Coleman JS (1994) Foundations of social theory. Harvard University Press, Cambridge

    Google Scholar 

  • Curşeu PL (2006) Emergent states in virtual teams: a complex adaptive systems perspective. J Inf Technol 21:249–261. https://doi.org/10.1057/palgrave.jit.2000077

    Article  Google Scholar 

  • D’Silva V, Kroening D, Weissenbacher G (2008) A survey of automated techniques for formal software verification. IEEE Trans Comput Aided Des Integr Circuits Syst 27:1165–1178. https://doi.org/10.1109/TCAD.2008.923410

    Article  Google Scholar 

  • Davis JP, Eisenhardt KM, Bingham CB (2007) Developing theory through simulation methods. Acad Manag Rev 32:480–499. https://doi.org/10.5465/AMR.2007.24351453

    Article  Google Scholar 

  • Doan TT, Yao Y, Alechina N, Logan B (2014) Verifying heterogeneous multi-agent programs. In: Proceedings of the 2014 International Conference on autonomous agents and multi-agent systems. Richland, pp 149–156

  • Dooley K (2002) Simulation research methods. In: Baum J (ed) Companion to organizations. Blackwell, London, pp 829–848

    Google Scholar 

  • Dowling D (1999) Experimenting on theories. Sci Context 12:261–273. https://doi.org/10.1017/S0269889700003410

    Article  Google Scholar 

  • Edmonds B, Hales D (2003) Replication, replication and replication: some hard lessons from model alignment. JASSS 6(4):1–4

    Google Scholar 

  • Epstein LG (1999) A definition of uncertainty aversion. Rev Econ Stud 66(3):579–608

    Article  Google Scholar 

  • Epstein JM (2008) Why model? JASSS 11(4):1–12

    Google Scholar 

  • Frank U (2011) Multi-perspective enterprise modelling: background and terminological foundation. ICB-Research Report, No. 46. University Duisburg-Essen, Essen

  • Frank U (2014) Multilevel modeling. Bus Inf Syst Eng 6(6):319–337. https://doi.org/10.1007/s12599-014-0350-4

    Article  Google Scholar 

  • Frank U, Troitzsch KG (2005) Epistemological perspectives on simulation. JASSS 8(4):1–5

    Google Scholar 

  • Frank U, Strecker S, Fettke P, Brocke J, vom Becker J, Sinz E (2014) The research field “modeling business information systems”. Bus Inf Syst Eng 6(1):39–43. https://doi.org/10.1007/s12599-013-0301-5

    Article  Google Scholar 

  • Frigg R, Hartmann S (2017) Models in science. In: Zalta EN (ed) The Stanford encyclopedia of philosophy, Spring, 2017th edn. Metaphysics Research Lab, Stanford University, Stanford

    Google Scholar 

  • Frigg R, Reiss J (2008) The philosophy of simulation: hot new issues or same old stew? Synthese 169:593–613. https://doi.org/10.1007/s11229-008-9438-z

    Article  Google Scholar 

  • Graupe D (2013) Principles of artificial neural networks, 3rd edn. World Scientific, Singapore

    Book  Google Scholar 

  • Gregor S, Hovorka D (2011) Causality: the elephant in the room in information systems epistemology. In: Proceedings of the 19th European Conference on Information Systems. Helsinki

  • Gregoriades A, Sutcliffe A (2008) A socio-technical approach to business process simulation. Decis Support Syst 45:1017–1030. https://doi.org/10.1016/j.dss.2008.04.003

    Article  Google Scholar 

  • Grüne-Yanoff T, Weirich P (2010) The philosophy and epistemology of simulation: a review. Simul Gaming 41:20–50. https://doi.org/10.1177/1046878109353470

    Article  Google Scholar 

  • Harrison JR, Lin Zhiang, Carroll GR, Carley KM (2007) Simulation modeling in organizational and management research. Acad Manag Rev 32:1229–1245. https://doi.org/10.5465/AMR.2007.26586485

    Article  Google Scholar 

  • Hedström P, Ylikoski P (2010) Causal mechanisms in the social sciences. Annu Rev Social 36:49–67. https://doi.org/10.1146/annurev.soc.012809.102632

    Article  Google Scholar 

  • Houy C, Fettke P, Loos P (2012) Understanding understandability of conceptual models—what are we actually talking about? Conceptual modeling. Springer, Heidelberg, pp 64–77

    Chapter  Google Scholar 

  • Houy C, Fettke P, Loos P (2015) Stylized Facts as an instrument for literature review and cumulative information systems research. Commun AIS 37:1

    Google Scholar 

  • Hua Z, Sun Y, Xu X (2011) Operational causes of bankruptcy propagation in supply chain. Decis Support Syst 51:671–681. https://doi.org/10.1016/j.dss.2011.03.007

    Article  Google Scholar 

  • Humphreys P (1990) Computer simulations. In: Proceedings of the Biennial Meeting of the Philosophy of Science Association, pp 497–506

  • Humphreys P (2008) The philosophical novelty of computer simulation methods. Synthese 169:615–626. https://doi.org/10.1007/s11229-008-9435-2

    Article  Google Scholar 

  • Jamroga W, Konikowska B, Penczek W (2016) Multi-valued verification of strategic ability. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems. Richland, pp 1180–1189

  • Johnson CF (1996) Deductive versus inductive reasoning: a closer look at economics. Soc Sci J 33:287–299. https://doi.org/10.1016/S0362-3319(96)90024-5

    Article  Google Scholar 

  • Johnson SLF, Samer F, Kudaravalli S (2014) Emergence of power laws in online communities: the role of social mechanisms and preferential attachment. MIS Q 38:795–808 (A13)

    Article  Google Scholar 

  • Kim Y, Street WN (2004) An intelligent system for customer targeting: a data mining approach. Decis Support Syst 37:215–228. https://doi.org/10.1016/S0167-9236(03)00008-3

    Article  Google Scholar 

  • Kochanowicz J, Ah-Hwee T, Thalmann D (2013) Enabling generative, emergent artificial culture. In: Proceedings of the 2013 International conference on autonomous agents and multi-agent systems. Richland, pp 789–792

  • Kulik BW, Baker T (2008) Putting the organization back into computational organization theory: a complex Perrowian model of organizational action. Comput Math Organ Theory 14:84–119. https://doi.org/10.1007/s10588-008-9022-6

    Article  Google Scholar 

  • Kuo RJ, Liao JL, Tu C (2005) Integration of ART2 neural network and genetic K-means algorithm for analyzing web browsing paths in electronic commerce. Decis Support Syst 40:355–374. https://doi.org/10.1016/j.dss.2004.04.010

    Article  Google Scholar 

  • Küppers G, Lenhard J (2005) Validation of simulation: patterns in the social and natural sciences. JASSS 8(4):1–3

    Google Scholar 

  • Lacy S, Riffe D (1996) Sampling error and selecting intercoder reliability samples for nominal content categories. J Mass Commun Q 73:963–973. https://doi.org/10.1177/107769909607300414

    Google Scholar 

  • Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174. https://doi.org/10.2307/2529310

    Article  Google Scholar 

  • Law AM (2008) How to build valid and credible simulation models. In: Proceedings of the 40th conference on winter simulation. Winter Simulation Conference, Miami, pp 39–47

  • Lee AS (2010) Retrospect and prospect: information systems research in the last and next 25 years. J Inf Technol 25:336–348. https://doi.org/10.1057/jit.2010.24

    Article  Google Scholar 

  • Lee J-S, Filatova T, Ligmann-Zielinska A, Hassani-Mahmooei B, Stonedahl F, Lorscheid I, Voinov A, Polhill JG, Sun Z, Parker DC (2015) The complexities of agent-based modeling output analysis. JASSS 18(4):1–4

    Article  Google Scholar 

  • Lim S, Saldanha T, Malladi S, Melville N (2013) Theories used in information systems research: insights from complex network analysis. JITTA 14(2):5–46

    Google Scholar 

  • Lombard M, Snyder-Duch J, Bracken CC (2002) Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum Commun Res 28:587–604. https://doi.org/10.1111/j.1468-2958.2002.tb00826.x

    Article  Google Scholar 

  • Loos P, Mettler T, Winter R, Goeken M, Frank U, Winter A (2013) Methodological pluralism in business and information systems engineering? Bus Inf Syst Eng 5(6):453–460. https://doi.org/10.1007/s12599-013-0299-8

    Article  Google Scholar 

  • Lowry PB, Moody GD, Gaskin J, Galletta DF, Humphreys S, Barlow JB, Wilson D (2013) Evaluating journal quality and the association for information systems (AIS) senior scholars’ journal basket via bibliometric measures: do expert journal assessments add value?. Social Science Research Network, Rochester

    Google Scholar 

  • Luna-Reyes LF, Zhang J, Ramón Gil-García J, Cresswell AM (2005) Information systems development as emergent socio-technical change: a practice approach. Eur J Inf Syst 14:93–105. https://doi.org/10.1057/palgrave.ejis.3000524

    Article  Google Scholar 

  • Lyytinen K, Newman M (2008) Explaining information systems change: a punctuated socio-technical change model. Eur J Inf Syst 17:589–613. https://doi.org/10.1057/ejis.2008.50

    Article  Google Scholar 

  • Macal CM, North MJ (2009) Agent-based modeling and simulation. In: Winter Simulation Conference. Winter Simulation Conference, Austin, pp 86–98

  • Manzo G (2007) Variables, mechanisms, and simulations: can the three methods be synthesized? Revue française de sociologie 48:35–71

    Article  Google Scholar 

  • McKelvey B (2002) Model-centered organization science epistemology. Companion to organizations. Oxford University Press, New York, pp 752–780

    Google Scholar 

  • McLeod L, Doolin B (2012) Information systems development as situated socio-technical change: a process approach. Eur J Inf Syst 21:176–191. https://doi.org/10.1057/ejis.2011.43

    Article  Google Scholar 

  • Montali M, Calvanese D, De Giacomo G (2014) Verification of data-aware commitment-based multiagent system. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems. Richland, pp 157–164

  • Morgan MS, Morrison M (1999) Models as mediators: perspectives on natural and social science. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Nan N (2011) Capturing bottom-up information technology use processes: a complex adaptive systems model. MIS Q 35:505–532 (A7)

    Article  Google Scholar 

  • Neuendorf KA (2016) The content analysis guidebook. SAGE, London

    Google Scholar 

  • Olson DL, Delen D, Meng Y (2012) Comparative analysis of data mining methods for bankruptcy prediction. Decis Support Syst 52:464–473. https://doi.org/10.1016/j.dss.2011.10.007

    Article  Google Scholar 

  • Orlikowski WJ (1992) The duality of technology: rethinking the concept of technology in organizations. Organ Sci 3:398–427. https://doi.org/10.1287/orsc.3.3.398

    Article  Google Scholar 

  • Orlikowski WJ, Iacono CS (2001) Research commentary: desperately seeking the “IT” in IT research—a call to theorizing the IT artifact. Inf Syst Res 12:121–134. https://doi.org/10.1287/isre.12.2.121.9700

    Article  Google Scholar 

  • Parunak HVD, Savit R, Riolo RL (1998) Agent-based modeling vs. equation-based modeling: a case study and users’ guide. In: Sichman JS, Conte R, Gilbert N (eds) Multi-agent systems and agent-based simulation. Springer, Heidelberg, pp 10–25

    Chapter  Google Scholar 

  • Pias C (2011) On the epistemology of computer simulation. Zeitschrift für Medien- und Kulturforschung 2011(1):29–54

    Article  Google Scholar 

  • Poile C, Safayeni F (2016) Using Computational modeling for building theory: a double edged sword. JASSS 19(3):1–20. https://doi.org/10.18564/jasss.3137

    Article  Google Scholar 

  • Prat N, Comyn-Wattiau I, Akoka J (2015) A taxonomy of evaluation methods for information systems artifacts. J Manag Inf Syst 32:229–267. https://doi.org/10.1080/07421222.2015.1099390

    Article  Google Scholar 

  • Ripley BD (2009) Stochastic simulation. Hoboken, NJ, USA

    Google Scholar 

  • Rivkin JW (2001) Reproducing knowledge: replication without imitation at moderate complexity. Organ Sci 12:274–293. https://doi.org/10.1287/orsc.12.3.274.10106

    Article  Google Scholar 

  • Roberts N, Deal R, Shaffer W, Garet M, Anderson D (1994) Introduction to computer simulation; a system dynamics approach. Productivity Press Inc. CRC press

  • Rohrlich F (1990) Computer simulation in the physical sciences. In: Proceedings of the Biennial Meeting of the Philosophy of Science Association, pp 507–518

  • Rowe F (2014) What literature review is not: diversity, boundaries and recommendations. Eur J Inf Syst 23:241–255. https://doi.org/10.1057/ejis.2014.7

    Article  Google Scholar 

  • Saldaña J (2013) The coding manual for qualitative researchers. SAGE, London

    Google Scholar 

  • Sargent RG (2005) Verification and validation of simulation models. In: Proceedings of the 37th conference on winter simulation. Orlando, pp 130–143

  • Sidorova A, Evangelopoulos N, Valacich J, Ramakrishnan T (2008) Uncovering the intellectual core of the information systems discipline. MIS Q 32:467–482

    Article  Google Scholar 

  • Spagnoletti P, Za S, Winter R (2013) Exploring foundations for using simulations in IS research. In: Proceedings of the 34th international conference on information systems. Milan

  • Sterman JD (2002) Systems dynamics modeling: tools for learning in a complex world. IEEE Eng Manag Rev 30:42. https://doi.org/10.1109/EMR.2002.1022404

    Article  Google Scholar 

  • Straub D (2012) Editor’s comments: does MIS have native theories? MIS Q 36:iii–xii

    Google Scholar 

  • Sun Z-L, Choi T-M, Au K-F, Yu Y (2008) Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 46:411–419. https://doi.org/10.1016/j.dss.2008.07.009

    Article  Google Scholar 

  • Tiwana A, Bush AA (2007) A comparison of transaction cost, agency, and knowledge-based predictors of IT outsourcing decisions: a US-Japan cross-cultural field study. J Manag Inf Syst 24:259–300. https://doi.org/10.2753/MIS0742-1222240108

    Article  Google Scholar 

  • Trier M (2008) Research note—towards dynamic visualization for understanding evolution of digital communication networks. Inf Syst Res 19:335–350. https://doi.org/10.1287/isre.1080.0191

    Article  Google Scholar 

  • vom Brocke J, Simons A, Riemer K, Niehaves B, Plattfaut R, Cleven A (2015) Standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. Commun AIS 37(1):205–223

    Google Scholar 

  • Wang C-H, Chuang J-J (2016) Integrating decision tree with back propagation network to conduct business diagnosis and performance simulation for solar companies. Decis Support Syst 81:12–19. https://doi.org/10.1016/j.dss.2015.10.004

    Article  Google Scholar 

  • Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26:xiii–xxiii

    Google Scholar 

  • Weinhardt JM, Vancouver JB (2012) Computational models and organizational psychology: opportunities abound. Organ Psychol Rev 2:267–292. https://doi.org/10.1177/2041386612450455

    Google Scholar 

  • Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4:65–85. https://doi.org/10.1007/BF00175354

    Article  Google Scholar 

  • Winsberg E (1999) Sanctioning models: the epistemology of simulation. Sci Context 12:275–292. https://doi.org/10.1017/S0269889700003422

    Article  Google Scholar 

  • Winsberg E (2001) Simulations, models, and theories: complex physical systems and their representations. Philos Sci 68:S442–S454

    Article  Google Scholar 

  • Winsberg E (2003) Simulated experiments: methodology for a virtual world. Philos Sci 70:105–125. https://doi.org/10.1086/367872

    Article  Google Scholar 

  • Winsberg E (2009) Computer simulation and the philosophy of science. Philos Compass 4:835–845. https://doi.org/10.1111/j.1747-9991.2009.00236.x

    Article  Google Scholar 

  • Wöhner T, Köhler S, Peters R (2015) Managed wikis—a new approach for web 2.0. Bus Inf. Syst Eng 57(3):155–166

    Google Scholar 

  • Woodard CJ, Clemons E (2014) Modeling the evolution of generativity and the emergence of digital ecosystems. In: Proceedings of the 35th international conference on information systems. Auckland

  • Wu PP-Y, Fookes C, Pitchforth J, Mengersen K (2015) A framework for model integration and holistic modelling of socio-technical systems. Decis Support Syst 71:14–27. https://doi.org/10.1016/j.dss.2015.01.006

    Article  Google Scholar 

  • Xiao S, Dong M (2015) Hidden semi-Markov model-based reputation management system for online to offline (O2O) e-commerce markets. Decis Support Syst 77:87–99. https://doi.org/10.1016/j.dss.2015.05.013

    Article  Google Scholar 

  • Zhang M, Gable G (2014) Rethinking the value of simulation methods in the Inf Syst Res field: a call for reconstructing contribution for a broader audience. In: Proceedings of the 35th international conference on information systems. Auckland

  • Zhuge H (2006) Knowledge flow network planning and simulation. Decis Support Syst 42:571–592. https://doi.org/10.1016/j.dss.2005.03.007

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jannis Beese.

Additional information

Accepted after two revisions by Prof. Dr. Frank.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 152 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beese, J., Haki, M.K., Aier, S. et al. Simulation-Based Research in Information Systems. Bus Inf Syst Eng 61, 503–521 (2019). https://doi.org/10.1007/s12599-018-0529-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12599-018-0529-1

Keywords

Navigation