Skip to main content
SpringerLink
Log in

Hierarchy, Process, and Cessation: Contributions to When and How to Validate

  • Chapter
  • First Online:
Co-Evolution of Standards in Innovation Systems

Part of the book series: Contributions to Management Science ((MANAGEMENT SC.))

  • 672 Accesses

Abstract

In the domain of dynamic modeling and simulation, the assurance of model validity is a prominent challenge. An extensive number of contributions concerning model tests, terminology, and the epistemological foundations of validation have been elaborated. These contributions, however, do not fully answer the questions for novice modelers, which validation tests to choose, when and how to apply them, and at what point to cease their validation efforts. The intention here is to help close this gap by introducing a complexity hierarchy of validation tests, an integrative validation process, and a decision heuristic about when to stop validation efforts. The chapter concludes by providing directions for future research.

You can understand and enjoy the adventure of science, because the thinking used in science is the same thinking you use in daily life. You use reality checks to decide whether the way you think the world is matches the way the world is. Craig Rusbult (2011)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We intentionally excluded commonly known procedures, e.g., evaluation of the model’s face validity, walkthroughs, or group model building, from Table 7.1. These are procedures which provide the environment in which validation tests are executed. For instance, a structured walkthrough meeting assembles a group of experts who together inspect the structure and behavior of a simulation model and discuss its validity, and detect and document faults (Balci, 1994). To achieve this purpose, the tests as outlined in Table 7.1 are applied.

References

  • Anastasakis, L., Olphert, C. W., & Wilson, J. M. (2008). Experiences in using a contingency factor-based validation methodology for spreadsheet DSS. Journal of the Operational Research Society, 59(6), 756–761.

    Article  Google Scholar 

  • Back, G., Love, G., & Falk, J. (2000). The doing of model verification and validation: Balancing cost and theory. Paper presented at the 18th International Conference of the System Dynamics Society, Bergen, Norway.

    Google Scholar 

  • Balci, O. (1994). Validation, verification and testing techniques throughout the life cycle of a simulation study. Annals of Operations Research, 53(1), 121–173.

    Article  Google Scholar 

  • Balci, O., & Sargent, R. G. (1981). A methodology for cost-risk analysis in the statistical validation of simulation-models. Communications of the ACM, 24(4), 190–197.

    Article  Google Scholar 

  • Barlas, Y. (1989). Multiple tests for validation of system dynamics type of simulation models. European Journal of Operational Research, 42(1), 59–87.

    Article  Google Scholar 

  • Barlas, Y. (1990). An autocorrelation function test for output validation. Simulation, 55(1), 7–16.

    Article  Google Scholar 

  • Barlas, Y. (1996). Formal aspects of model validity and validation in system dynamics. System Dynamics Review, 12(3), 183–210.

    Article  Google Scholar 

  • Barlas, Y., & Carpenter, S. (1990). Philosophical roots of model validation: Two paradigms. System Dynamics Review, 6(2), 148–166.

    Article  Google Scholar 

  • Bossel, H. (2004). Systeme, dynamik, simulation: Modellbildung, analyse und simulation komplexer systeme. Norderstedt, Germany: Books on Demand.

    Google Scholar 

  • Bratley, P., Fox, B., & Schrage, L. (1987). A guide to simulation. New York: Springer.

    Book  Google Scholar 

  • Coyle, R. G., & Exelby, D. R. (2000). The validation of commercial system dynamics models. System Dynamics Review, 16(1), 27–41.

    Article  Google Scholar 

  • Defense Modeling and Simulation Office. (1996). Verification, validation and accreditation: Recommended practices guide. US Department of Defense, Washington.

    Google Scholar 

  • Déry, R., Landry, M., & Banville, C. (1993). Revisiting the issue of model validation in OR: An epistemological view. European Journal of Operational Research, 66(2), 168–183.

    Article  Google Scholar 

  • Eberlein, R. L., & Wang, Q. (1983). Validation of oscillatory behavior modes using spectral analysis. Paper presented at the International Conference of the System Dynamics Society, Chestnut Hill, PA.

    Google Scholar 

  • Finlay, P. N., Forsey, G. J., & Wilson, J. M. (1988). The validation of expert systems—Contrasts with traditional methods. Journal of the Operational Research Society, 39(10), 933–938.

    Google Scholar 

  • Finlay, P. N., & Wilson, J. M. (1997). Validity of decision support systems: Towards a validation methodology. Systems Research and Behavioral Science, 14(3), 169–182.

    Article  Google Scholar 

  • Finlay, P. N., & Wilson, J. M. (2000). A survey of contingency factors affecting the validation of end-user spreadsheet-based decision support systems. Journal of the Operational Research Society, 51(8), 949–958.

    Google Scholar 

  • Ford, D. N. (1999). A behavioral approach to feedback loop dominance analysis. System Dynamics Review, 15(1), 3–36.

    Article  Google Scholar 

  • Forrester, J. W. (1961). Industrial dynamics. Cambridge, MA: Productivity Press.

    Google Scholar 

  • Forrester, J. W. (1994). Policies, decisions, and information sources for modeling. In J. D. W. Morecroft & J. D. Sterman (Eds.), Modeling for learning organizations (pp. 51–84). Portland, OR: Productivity Press.

    Google Scholar 

  • Forrester, J. W. (2007). System dynamics: The next fifty years. System Dynamics Review, 23(2–3), 359–370.

    Article  Google Scholar 

  • Forrester, J. W., & Senge, P. M. (1980). Tests for building confidence in system dynamics models. In A. A. Legasto, J. W. Forrester, & J. M. Lyneis (Eds.), System dynamics: TIMS studies in the management sciences (Vol. 14). Amsterdam: North-Holland.

    Google Scholar 

  • Forrester, J. W. (1985). The ‘model’ versus a modeling ‘process’. System Dynamics Review, 1(1), 133–143.

    Article  Google Scholar 

  • Gass, S. I. (1983). Decision-aiding models—validation, assessment, and related issues for policy analysis. Operations Research, 31(4), 603–631.

    Article  Google Scholar 

  • Glaser, B. G. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Mill Valley, CA: Sociology Press.

    Google Scholar 

  • Glaser, B. G. (1992). Basics of grounded theory analysis. Mill Valley, CA: Sociology Press.

    Google Scholar 

  • Glaser, B. G., & Strauss, A. L. (Eds.). (1967). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine De Gruyter.

    Google Scholar 

  • Graham, A. K. (1980). Parameter estimation in system dynamics modeling. In J. Randers (Ed.), Elements of the system dynamics method (pp. 143–161). Cambridge, MA: Productivity Press.

    Google Scholar 

  • Groesser, S. N., & Bruppacher, S. (2007). Decisions in the planning process of a building: development of a dynamic model about individual’s energy efficiency intention over time. Paper presented at the 25th International Conference of the System Dynamics Society, Boston.

    Google Scholar 

  • Groesser, S. N., & Schaffernicht, M. (2012). Mental models of dynamic systems: Taking stock and looking ahead. System Dynamics Review.

    Google Scholar 

  • Groesser, S. N., & Ulli-Beer, S. (2006). Diffusion dynamics of energy-efficient innovations in the residential building environment: A simulation approach. Paper presented at the 24th International Conference of the System Dynamics Society, Nijmegen, The Netherlands.

    Google Scholar 

  • Groesser, S. N., & Ulli-Beer, S. (2008). Long-Term innovation diffusion in the building construction industry: Empirically-based theory building. Paper presented at the 26th International Conference of the System Dynamics Society, Athens, Greece.

    Google Scholar 

  • Güneralp, B. (2006). Towards coherent loop dominance analysis: Progress in Eigenvalue elasticity analysis. System Dynamics Review, 22(3), 263–289.

    Article  Google Scholar 

  • Hall, R. I. (1976). A system pathology of an organization: The rise and fall of the old Saturday evening post. Administrative Science Quarterly, 21(2), 185–211.

    Article  Google Scholar 

  • Homer, J. B. (1983). Partial-model testing as a validation tool for system dynamics. Paper presented at the International Conference of the System Dynamics Society, Chestnut Hill, PA.

    Google Scholar 

  • Homer, J. B. (1996). Why we iterate: Scientific modeling in theory and practice. System Dynamics Review, 12(1), 1–19.

    Article  Google Scholar 

  • Kahneman, D., & Tversky, A. (1979). Prospect theory – analysis of decision under risk. Econometrica, 47(2), 263–291.

    Article  Google Scholar 

  • Kampmann, C. E., & Oliva, R. (2009). System dynamics: Analytical methods for structural dominance analysis. In Encyclopedia of complexity and systems science. New York/London/Berlin: Springer.

    Google Scholar 

  • Kennedy, P. (2003). A guide to econometrics (5th ed.). Oxford: Basil Blackwell.

    Google Scholar 

  • Kleijnen, J. P. C. (1995). Statistical validation of simulation-models. European Journal of Operational Research, 87(1), 21–34.

    Article  Google Scholar 

  • Kleindorfer, G. B., & Geneshan, R. (1993). The philosophy of science and validation in simulation, Proceedings of the 25th conference on winter simulation. Los Angeles: Winter simulation conference.

    Google Scholar 

  • Kirchner, J. W. (1984). A methodological framework for system dynamics model evaluation. Dynamica, 10(1), 9–15.

    Google Scholar 

  • Lane, D. C. (1995). The folding star: A comparative reframing and extension of validity concepts in system dynamics. Paper presented at the International Conference of the System Dynamics Society, Tokyo.

    Google Scholar 

  • Law, A. M., & Kelton, D. W. (1982). Simulation modeling and analysis. New York: McGraw-Hill.

    Google Scholar 

  • Oliva, R. (2003). Model calibration as a testing strategy for system dynamics models. European Journal of Operational Research, 151(3), 552–568.

    Article  Google Scholar 

  • Olphert, C. W., & Wilson, J. M. (2004). Validation of decision-aiding spreadsheets: The influence of contingency factors. Journal of the Operational Research Society, 55(1), 12–22.

    Article  Google Scholar 

  • Oral, M., & Kettani, O. (1993). The facets of the modeling and validation process in operations research. European Journal of Operational Research, 66(2), 216–234.

    Article  Google Scholar 

  • Peterson, D. W., & Eberlein, R. L. (1994). Reality checks: A bridge between systems thinking and system dynamics. System Dynamics Review, 10(2/3), 159–174.

    Article  Google Scholar 

  • Richardson, G. P., & Pugh, A. L., III. (1981). Introduction to system dynamics modeling with DYNAMO. Cambridge, MA: Productivity Press.

    Google Scholar 

  • Rusbult, C. (2011). Introduction to scientific method: Using logical reality checks in science and other areas of life. Madison, WI. Accessed May 2, 2011, from http://www.asa3.org/ASA/education/think/scientific-method.htm.

  • Sargent, R. G. (1992). Validation and verification of simulation models, Proceedings of the 24th conference on winter simulation. Arlington, VA: Winter simulation conference.

    Google Scholar 

  • Sargent, R. G. (2008). Verification and validation of simulation models, Proceedings of the 40th conference on winter simulation. Miami, FL: Winter simulation conference.

    Google Scholar 

  • Saysel, A. K., & Barlas, Y. (2006). Model simplification and validation with indirect structure validity tests. System Dynamics Review, 22(3), 241–262.

    Article  Google Scholar 

  • Schaffernicht, M., & Groesser, S. N. (2011). Comparison of mental models: Development of a comprehensive method. European Journal of Operational Research, 210(2), 57–67.

    Article  Google Scholar 

  • Schwaninger, M. (2009). Intelligent organizations: Powerful models for systemic management (2nd ed.). Berlin/Heidelberg: Springer.

    Google Scholar 

  • Schwaninger, M., & Groesser, S. N. (2008). Model-based theory-building with system dynamics. Systems Research and Behavioral Science, 25(4), 447–465.

    Article  Google Scholar 

  • Schwaninger, M., & Groesser, S. N. (2009). System dynamics modeling: Validation for quality assurance. In Encyclopedia of complexity and system science. Berlin/London/Paris: Springer.

    Google Scholar 

  • Simon, H. A. (1964). Models of man: Social and rational mathematical essays on rational human behavior in a social setting. New York: Wiley.

    Google Scholar 

  • Sterman, J. D. (1984). Appropriate summary statistics for evaluating the historical fit of system dynamics models. Dynamica, 10(2), 51–66.

    Google Scholar 

  • Sterman, J. D. (2000). Business dynamics: Systems thinking and modeling for a complex world. Boston: McGraw-Hill.

    Google Scholar 

  • Sterman, J. D. (2002). All models are wrong: Reflections on becoming a systems scientist. System Dynamics Review, 18(4), 501–531.

    Article  Google Scholar 

  • Strauss, A. L., & Corbin, J. M. (Eds.). (1997). Grounded theory in practice. Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Strauss, A. L., & Corbin, J. M. (1998). Basics of qualitative research techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Taylor, A. J. (1980). Loop analysis and validation. Dynamica, 6(1), 2–8.

    Google Scholar 

  • Weick, K. E. (1989). Theory construction as disciplined imagination. Academy of Management Review, 14(4), 516–531.

    Google Scholar 

  • Weil, H. B. (1983). What is an adequate model? Paper presented at the International Conference of the System Dynamics Society, Chestnut Hill, PA.

    Google Scholar 

  • Wolstenholme, E. F. (1999). Qualitative vs quantitative modelling: The evolving balance. Journal of the Operational Research Society, 50(4), 422–428.

    Google Scholar 

  • Zeigler, B. P., Praehofer, H., & Kim, T. G. (2000). Theory of modeling and simulation (2nd ed.). San Diego, CA: Academic.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Management (IfB-HSG) HSG System Dynamic Group, University of St. Gallen, St.Gallen, Switzerland

    Stefan N. Grösser

Authors
  1. Stefan N. Grösser
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Grösser, S.N. (2013). Hierarchy, Process, and Cessation: Contributions to When and How to Validate. In: Co-Evolution of Standards in Innovation Systems. Contributions to Management Science. Physica, Heidelberg. https://doi.org/10.1007/978-3-7908-2858-0_7

Download citation

Publish with us

Policies and ethics

Search

Navigation