Advertisement

Verification and Validation Principles from a Systems Perspective

  • David J. Murray-SmithEmail author
Chapter
Part of the Simulation Foundations, Methods and Applications book series (SFMA)

Abstract

This chapter introduces concepts and principles associated with the verification and validation of simulation models, mainly in the context of models of complete systems. The word ‘verification’ is used here to describe testing processes to establish whether a computer-based representation correctly describes the underlying mathematical, logical and theoretical structure of the model. The word ‘validation’ is used to describe procedures for establishing whether the model fidelity is adequate for the purposes of the given application. Verification is internal to the model and the computer-based representation while validation processes involve information external to the model, normally using data or observations from the corresponding real system. The goal of the testing process for a simulation model must always be to establish the extent to which a model has the quality and credibility required for the intended application. These model testing processes, involving both verification and validation, are inherently iterative.

Keywords

Continuous system simulation models Testing Sensitivity Identifiability Documentation Model acceptance Model upgrading 

References

  1. Anderson, J., & Papachristodoulou, A. (2009). On validation and invalidation of biological models. BMC Bioinfomatics, 10(1), 132.CrossRefGoogle Scholar
  2. Beck, J. V., & Arnold, K. J. (1977). Parameter estimation in science and engineering. New York: Wiley.zbMATHGoogle Scholar
  3. Bellman, R., & Åstrӧm, K. J. (1970). On structural identifiability. Mathematical Biosciences, 7, 329–339.CrossRefGoogle Scholar
  4. Bradley, R., Padfield, G. D., Murray-Smith, D. J., et al. (1990). Validation of helicopter mathematical models. Transactions of the Institute of Measurement and Control, 12, 186–196.CrossRefGoogle Scholar
  5. Butterfield, M. H., & Thomas, P. J. (1986). Methods of quantitative validation for dynamic system models—part 1: Theory. Transactions of the Institute of Measurement & Control, 8, 182–200.CrossRefGoogle Scholar
  6. Caro, J. J., Briggs, A. H., Siebert, U., et al. (2012). Modeling good research practices—overview: A report of the ISPOR-SMDM modeling good research practices task force-1. Medical Decision Making, 32, 667–677.CrossRefGoogle Scholar
  7. Chatfield, C. (2003). The analysis of time series: An introduction (6th ed.). Boca Raton: Chapman and Hall/CRC.zbMATHGoogle Scholar
  8. Cloud, D. J., & Rainey, L. B. (Eds.). (1998). Applied modelling and simulation: An integrated approach to development and operation. New York: McGraw-Hill.Google Scholar
  9. Eddy, D. M., Hollingsworth, W., Caro, J. J., et al. (2012). Model transparency and validation: A report of the ISPOR-SMDM modeling good research practices task force-7. Medical Decision Making, 32, 733–743.CrossRefGoogle Scholar
  10. Frank, P. M. (1978). An introduction to system sensitivity theory. London: Academic.zbMATHGoogle Scholar
  11. Gábor, A., Villaverde, A. F., & Banga, J. R. (2017). Parameter identifiability analysis and visualization in large-scale kinetic models of biosystems. BMC Systems Biology, 11, 54.  https://doi.org/10.1186/s12918-017-0428-y.CrossRefGoogle Scholar
  12. Goodwin, G. C., & Payne, R. L. (1977). Dynamic system identification: Experiment design and data analysis. New York: Academic.zbMATHGoogle Scholar
  13. Gore, R., & Diallo, S. (2013). The need for usable formal methods in verification and validation. In R. Pasupthy, S.-H. Kim & A. Tolk et al. (Eds.) Proceedings of the 2013 Winter Simulation Conference (pp 1257–1268). Washington DC: IEEE.  https://doi.org/10.1109/wsc.2013.6721513.
  14. Grewal, M. S., & Glover, K. (1976). Identifiability of linear and nonlinear dynamical systems. IEEE Transactions on Automatic Control, 21, 833–837.MathSciNetCrossRefGoogle Scholar
  15. Hamby, D. M. (1994). A review of techniques for parameter sensitivity analysis of environmental models. Environmental Monitoring and Assessment, 32, 135–154.CrossRefGoogle Scholar
  16. Heitmeyer, C. L. (2007). Formal methods for specifying, validating and verifying requirements. Journal of Universal Computer Science, 13(5), 607–618.Google Scholar
  17. Hemez, F. M. (2004). The myth of science-based predictive modelling. In Proceedings foundtions’04 workshop for verification, validation and accreditation (VV&A) in the 21st century. Arizona State University, Tempe, Arizona, 13–15 October 2004. Report LA-UR-04-6829, Los Alamos National Laboratory, USA.Google Scholar
  18. Heyhurst, K. L., Veerhusen, D. S., Chilenski, J. L., et al. (2001). A practical tutorial on modified condition/decision coverage, NASA/TM-2001-210876. Hampton, VA, USA: National Aeronautics and Space Administration, Langley Research Center.Google Scholar
  19. Kaner, C., Falk, J., & Nguyen, H. Q. (1999). Testing computer software (2nd ed.). New York: Wiley.zbMATHGoogle Scholar
  20. Kit, E. (1995). Software testing in the real world. Harlow: Addison Wesley.Google Scholar
  21. Kuhn, D. R., Chandramouli, R., & Butler, R. W. (2002). Cost effective use of formal methods in verification and validation. Invited paper, Presented at Foundations’02 Workshop, US Department of Defense, Laurel, Maryland, October 22–23, 2002. Retrieved from http://csrc.nist.gov/staff/Kuhn/kuhn-chandramouli-butler-02.pdf.
  22. Murray-Smith, D. J. (2015). Testing and validation of computer simulation models: Principles, methods and applications. Cham: Springer.CrossRefGoogle Scholar
  23. Oberkampf, W. L., & Roy, C. J. (2010). Verification and validation in scientific computing. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  24. Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation and confirmation of numerical models in the earth sciences. Science, 263(5147), 641–646.CrossRefGoogle Scholar
  25. Pace, D. K. (2004). Modeling and simulation verification and validation challenges. Johns Hopkins APL Technical Digest, 25, 163–172.Google Scholar
  26. Pranja, S. (2006). Barrier certificates for nonlinear model validation. Automatica, 42, 117–126.MathSciNetCrossRefGoogle Scholar
  27. Raue, A., Kreutz, C., Maiwald, T., et al. (2009). Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics, 25, 1923–1929.CrossRefGoogle Scholar
  28. Schlacher, K., & Schöberl, M. (Eds.) (2011). Special issue; Modelling, analysis and control of distributed parameter systems. Mathematical and Computer Simulation of Dynamical Systems, 17, 1–121.Google Scholar
  29. Smith, M. I., Murray-Smith, D. J., & Hickman, D. (2007). Verification and validation issues in a generic model of an electro-optic sensor system. Journal of Defense Modeling & Simulation, 4, 17–27.CrossRefGoogle Scholar
  30. Steinberg, S., & Roache, P. J. (1985). Symbolic manipulation and computational fluid dynamics. Journal of Computational Physics, 57, 251–284.MathSciNetCrossRefGoogle Scholar
  31. Tomović, R. (1963). Sensitivity analysis of dynamic systems. New York: McGraw-Hill.Google Scholar
  32. The Mitre Corporation. (2014). Verification and validation of simulation models. In Mitre systems engineering guide (pp 461–469). Bedford: The Mitre Corporation. www.mitre.org/publications/technical-papers/the-mitre-systems-engineering-guide.
  33. Winsberg, E. (2018). Computer simulations in science. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy (Summer 2018 Edition), forthcoming. https://plato.stanford.edu/archives/sum2018/entries/simulations-science/.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of EngineeringUniversity of GlasgowRankine Building, Glasgow G12 8QQUK

Personalised recommendations