Advertisement

The Management of Simulation Validation

  • Fei LiuEmail author
  • Ming Yang
Chapter
Part of the Simulation Foundations, Methods and Applications book series (SFMA)

Abstract

In this chapter, we discuss the management of simulation validation for complex simulation systems. We first present nine principles for simulation validation, which are important to achieve good management and determine the success of simulation validation. By considering these principles, we present a management framework of simulation verification and validation (V&V), which includes four components: V&V process, V&V scheme, V&V metrics, and V&V tools. That is, we adopt a process-oriented, optimized, quantitative, and automatic management manner for simulation V&V of complex simulation systems. We then describe each component of the framework in detail and discuss the involved management issues. We hope this chapter could help the user to understand the management of simulation validation for complex simulation systems, and guide the user to manage the validation of practical simulation systems.

Keywords

Simulation validation Management Verification and validation 

Notes

Acknowledgements

This work has been supported by National Key R&D Program of China (2018YFC0830900), National Natural Science Foundation of China (61873094), Science and Technology Program of Guangzhou, China (201804010246), and Natural Science Foundation of Guangdong Province of China (2018A030313338).

References

  1. (2012). IEEE Standard for System and Software Verification and Validation. IEEE Std 1012-2012 (Revision of IEEE Std 1012-2004), pp. 1–223.Google Scholar
  2. Balci, O. (1995). Principles and techniques of simulation validation, verification, and testing. Winter Simulation Conference Proceedings, 1995, 147–154.Google Scholar
  3. Balci, O. (2003). Verification, validation, and certification of modeling and simulation applications. In Proceedings of the 2003 Winter Simulation Conference, 2003 (Vol. 1, pp. 150–158).Google Scholar
  4. Balci, O., Arthur, J. D., & Nance, R. E. (2008). Accomplishing reuse with a simulation conceptual model. In 2008 Winter Simulation Conference (pp. 959–965).Google Scholar
  5. Balci, O., Ormby, W. F., Carr, J. T., & Saadi, S. D. (2000). Planning for verification, validation, and accreditation of modeling and simulation applications. In 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165) (Vol. 1, pp. 829–839).Google Scholar
  6. Chew, J. & Sullivan, C. (2000). Verification, validation, and accreditation in the life cycle of models and simulations. In 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165) (Vol. 1, pp. 813–818).Google Scholar
  7. DMSO (1996). Verification, validation and accreditation recommended practice guide.Google Scholar
  8. Fang, K., Yang, M., & Wang, Z. (2005). The Hitvice VV&A environment. Proceedings of the Winter Simulation Conference, 2005, 1220–1227.Google Scholar
  9. Fayol, H. (1917). Administration industrielle et generale; prevoyance, organisation, commandement, coordination, controle. Pinat, Paris: H. Dunod et E.Google Scholar
  10. Fujimoto, R. M. (2003). Distributed simulation systems. In Proceedings of the 2003 Winter Simulation Conference, 2003 (Vol. 1, pp. 124–134).Google Scholar
  11. Hill, R. R., Miller, J. O., & McIntyre, G. A. (2001). Applications of discrete event simulation modeling to military problems. In Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304) (Vol. 1, pp. 780–788).Google Scholar
  12. Hollmann, D. A., Cristi, M., & Frydman, C. (2015). CML-DEVS: A specification language for devs conceptual models. Simulation Modelling Practice and Theory, 57, 100–117.CrossRefGoogle Scholar
  13. Jennifer, C. & Cindy, S. (2000). Verification, validation, and accreditation in the life cycle of models and simulations. In Proceedings of the Winter Simulation Conference (pp. 813–818). Orlando, FL, USA: IEEE.Google Scholar
  14. Klock, S. K. & Kemper, P. (2010a). An automated technique to support the verification and validation of simulation models. In 2010 IEEE/IFIP International Conference on Dependable Systems Networks (DSN) (pp. 595–604).Google Scholar
  15. Klock, S. K. & Kemper, P. (2010b). An automated technique to support the verification and validation of simulation models. In 2010 IEEE/IFIP International Conference on Dependable Systems Networks (DSN) (pp. 595–604).Google Scholar
  16. Li, Y., Liu, F., & Yang, M. (2009). Research of knowledge-based method to simulation model validation. Journal of System Simulation, 21(8).Google Scholar
  17. Liu, F., Blätke, M.-A., Heiner, M., & Yang, M. (2014). Modelling and simulating reactiondiffusion systems using coloured petri nets. Computers in Biology and Medicine, 53, 297–308.Google Scholar
  18. Liu, F., Ma, P., Yang, M., Sun, G., & Wang, Z. (2006a). Study on the credibility quantification of large complex smimulation systems. Journal of Sichuan University (Engineering Science Edition), 38(5), 169–174.Google Scholar
  19. Liu, F., Ma, P., Yang, M., & Wang, Z. (2009). Key problems in validation of intelligent models. Journal of Harbin Institute of Technology, 16(3), 371–375.Google Scholar
  20. Liu, F., & Yang, M. (2005a). Validation of system models. In IEEE International Conference Mechatronics and Automation, 2005 (Vol. 4, pp. 1721–1725).Google Scholar
  21. Liu, F., & Yang, M. (2005b). Verification and validation of artificial neural network models (pp. 1041–1046). Heidelberg: Springer.Google Scholar
  22. Liu, F., & Yang, M. (2009). An optimal design method for simulation verification, validation and accreditation schemes. Simulation: Transactions of The Society for Modeling and Simulation International, 85(6), 375–386.Google Scholar
  23. Liu, F., Yang, M., & Wang, Z. (2005). Study on simulation credibility metrics. In Proceedings of the Winter Simulation Conference, 2005 (p. 7).Google Scholar
  24. Liu, F., Yang, M., & Wang, Z. (2006b). Design and development of an expert system-like validation tool for distributed simulation systems. Journal of Jiangsu University (Natural Science Edition), 27(3).Google Scholar
  25. Liu, F., Yang, M., & Wang, Z. (2006c). Formal verification method of simulation scenario based on high-level petri nets. Control and Decision, 21(11).Google Scholar
  26. Liu, F., Yang, M., & Wang, Z. (2008). VV&A solution for complex simulation systems. International Journal of Simulation: Systems, Science and Technology, 9(1), 10–18.Google Scholar
  27. Mostafa, H., Liu, F., & Heiner, M. (2018). Efficient modelling of yeast cell cycles based on multisite phosphorylation using coloured hybrid petri nets with marking-dependent arc weights. Nonlinear Analysis: Hybrid Systems, 27, 191–212.MathSciNetzbMATHGoogle Scholar
  28. Muessing, P., & Laack, D. (1997). Optimizing the selection of VV&A activities: A risk/benefit approach. In Proceedings of the 1997 Summer Computer Simulation Conference (pp. 60–66).Google Scholar
  29. Naylor, T. H., & Finger, J. M. (1967). Verification of computer simulation models. Management Science, 2, 92–101.CrossRefGoogle Scholar
  30. Qin, L., Fang, K., & Yang, M. (2010). Research on the simulation credibility evaluation assistant tool based on hierarchical evaluation. Computer Simulation, 27(6).Google Scholar
  31. Robinson, S. (1997). Simulation maidel verification and validation: Increasing the users’ confidence. Winter Simulation Conference Proceedings, 53–59.Google Scholar
  32. Robinson, S. (2001). Modes of simulation practice in business and the military. In Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304), (Vol. 1, pp. 805–811).Google Scholar
  33. Robinson, S. (2006). Conceptual modeling for simulation: Issues and research requirements. In Proceedings of the 2006 Winter Simulation Conference (pp. 792–800).Google Scholar
  34. Robinson, S. (2012). Tutorial: Choosing what to model; conceptual modeling for simulation. In Proceedings of the 2012 Winter Simulation Conference (WSC) (pp. 1–12).Google Scholar
  35. Robinson, S. (2013). Conceptual modeling for simulation. In 2013 Winter Simulations Conference (WSC) (pp. 377–388).Google Scholar
  36. Robinson, S. (2017). A tutorial on simulation conceptual modeling. In 2017 Winter Simulation Conference (WSC) (pp. 565–579).Google Scholar
  37. Sargent, R. G. (1991). Simulation model verification and validation. In 1991 Winter Simulation Conference Proceedings (pp. 37–47).Google Scholar
  38. Sargent, R. G. (2010). Verification and validation of simulation models. In Proceedings of the 2010 Winter Simulation Conference (pp. 166–183).Google Scholar
  39. Sargent, R. G. (2011). Verification and validation of simulation models. In Proceedings of the 2011 Winter Simulation Conference (WSC) (pp. 183–198).Google Scholar
  40. Sargent, R. G. (2013). An introduction to verification and validation of simulation models. In 2013 Winter Simulations Conference (WSC) (pp. 321–327).Google Scholar
  41. Sargent, R. G. (2015). An introductory tutorial on verification and validation of simulation models. In 2015 Winter Simulation Conference (WSC) (pp. 1729–1740).Google Scholar
  42. Sargent, R. G., & Balci, O. (2017). History of verification and validation of simulation models. In 2017 Winter Simulation Conference (WSC) (pp. 292–307).Google Scholar
  43. Shannon, R. E. (1975). Systems simulation: the art and science. Englewood Cliffs, New Jersey: Prentice-Hall.Google Scholar
  44. Shi, P., Liu, F., & Yang, M. (2008). Research on validation method for complex simulation systems. In 2008 Asia Simulation Conference7th International Conference on System Simulation and Scientific Computing (pp. 888–892).Google Scholar
  45. Shi, P., Liu, F., & Yang, M. (2009a). Quantify simulation verification and validation. In 2009 11th International Conference on Computer Modelling and Simulation (pp. 123–128).Google Scholar
  46. Shi, P., Liu, F., Yang, M., & Wang, Z. (2009b). A fuzzy rules-based approach to analyzing human behavior models. In 2009 11th International Conference on Computer Modelling and Simulation (pp. 346–351).Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Software EngineeringSouth China University of TechnologyGuangzhouChina
  2. 2.Control and Simulation CenterHarbin Institute of TechnologyHarbinChina

Personalised recommendations