Abstract
In this chapter, we discuss the management of simulation validation for complex simulation systems. We first present nine principles for simulation validation, which are important to achieve good management and determine the success of simulation validation. By considering these principles, we present a management framework of simulation verification and validation (V&V), which includes four components: V&V process, V&V scheme, V&V metrics, and V&V tools. That is, we adopt a process-oriented, optimized, quantitative, and automatic management manner for simulation V&V of complex simulation systems. We then describe each component of the framework in detail and discuss the involved management issues. We hope this chapter could help the user to understand the management of simulation validation for complex simulation systems, and guide the user to manage the validation of practical simulation systems.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
(2012). IEEE Standard for System and Software Verification and Validation. IEEE Std 1012-2012 (Revision of IEEE Std 1012-2004), pp. 1–223.
Balci, O. (1995). Principles and techniques of simulation validation, verification, and testing. Winter Simulation Conference Proceedings, 1995, 147–154.
Balci, O. (2003). Verification, validation, and certification of modeling and simulation applications. In Proceedings of the 2003 Winter Simulation Conference, 2003 (Vol. 1, pp. 150–158).
Balci, O., Arthur, J. D., & Nance, R. E. (2008). Accomplishing reuse with a simulation conceptual model. In 2008 Winter Simulation Conference (pp. 959–965).
Balci, O., Ormby, W. F., Carr, J. T., & Saadi, S. D. (2000). Planning for verification, validation, and accreditation of modeling and simulation applications. In 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165) (Vol. 1, pp. 829–839).
Chew, J. & Sullivan, C. (2000). Verification, validation, and accreditation in the life cycle of models and simulations. In 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165) (Vol. 1, pp. 813–818).
DMSO (1996). Verification, validation and accreditation recommended practice guide.
Fang, K., Yang, M., & Wang, Z. (2005). The Hitvice VV&A environment. Proceedings of the Winter Simulation Conference, 2005, 1220–1227.
Fayol, H. (1917). Administration industrielle et generale; prevoyance, organisation, commandement, coordination, controle. Pinat, Paris: H. Dunod et E.
Fujimoto, R. M. (2003). Distributed simulation systems. In Proceedings of the 2003 Winter Simulation Conference, 2003 (Vol. 1, pp. 124–134).
Hill, R. R., Miller, J. O., & McIntyre, G. A. (2001). Applications of discrete event simulation modeling to military problems. In Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304) (Vol. 1, pp. 780–788).
Hollmann, D. A., Cristi, M., & Frydman, C. (2015). CML-DEVS: A specification language for devs conceptual models. Simulation Modelling Practice and Theory, 57, 100–117.
Jennifer, C. & Cindy, S. (2000). Verification, validation, and accreditation in the life cycle of models and simulations. In Proceedings of the Winter Simulation Conference (pp. 813–818). Orlando, FL, USA: IEEE.
Klock, S. K. & Kemper, P. (2010a). An automated technique to support the verification and validation of simulation models. In 2010 IEEE/IFIP International Conference on Dependable Systems Networks (DSN) (pp. 595–604).
Klock, S. K. & Kemper, P. (2010b). An automated technique to support the verification and validation of simulation models. In 2010 IEEE/IFIP International Conference on Dependable Systems Networks (DSN) (pp. 595–604).
Li, Y., Liu, F., & Yang, M. (2009). Research of knowledge-based method to simulation model validation. Journal of System Simulation, 21(8).
Liu, F., Blätke, M.-A., Heiner, M., & Yang, M. (2014). Modelling and simulating reactiondiffusion systems using coloured petri nets. Computers in Biology and Medicine, 53, 297–308.
Liu, F., Ma, P., Yang, M., Sun, G., & Wang, Z. (2006a). Study on the credibility quantification of large complex smimulation systems. Journal of Sichuan University (Engineering Science Edition), 38(5), 169–174.
Liu, F., Ma, P., Yang, M., & Wang, Z. (2009). Key problems in validation of intelligent models. Journal of Harbin Institute of Technology, 16(3), 371–375.
Liu, F., & Yang, M. (2005a). Validation of system models. In IEEE International Conference Mechatronics and Automation, 2005 (Vol. 4, pp. 1721–1725).
Liu, F., & Yang, M. (2005b). Verification and validation of artificial neural network models (pp. 1041–1046). Heidelberg: Springer.
Liu, F., & Yang, M. (2009). An optimal design method for simulation verification, validation and accreditation schemes. Simulation: Transactions of The Society for Modeling and Simulation International, 85(6), 375–386.
Liu, F., Yang, M., & Wang, Z. (2005). Study on simulation credibility metrics. In Proceedings of the Winter Simulation Conference, 2005 (p. 7).
Liu, F., Yang, M., & Wang, Z. (2006b). Design and development of an expert system-like validation tool for distributed simulation systems. Journal of Jiangsu University (Natural Science Edition), 27(3).
Liu, F., Yang, M., & Wang, Z. (2006c). Formal verification method of simulation scenario based on high-level petri nets. Control and Decision, 21(11).
Liu, F., Yang, M., & Wang, Z. (2008). VV&A solution for complex simulation systems. International Journal of Simulation: Systems, Science and Technology, 9(1), 10–18.
Mostafa, H., Liu, F., & Heiner, M. (2018). Efficient modelling of yeast cell cycles based on multisite phosphorylation using coloured hybrid petri nets with marking-dependent arc weights. Nonlinear Analysis: Hybrid Systems, 27, 191–212.
Muessing, P., & Laack, D. (1997). Optimizing the selection of VV&A activities: A risk/benefit approach. In Proceedings of the 1997 Summer Computer Simulation Conference (pp. 60–66).
Naylor, T. H., & Finger, J. M. (1967). Verification of computer simulation models. Management Science, 2, 92–101.
Qin, L., Fang, K., & Yang, M. (2010). Research on the simulation credibility evaluation assistant tool based on hierarchical evaluation. Computer Simulation, 27(6).
Robinson, S. (1997). Simulation maidel verification and validation: Increasing the users’ confidence. Winter Simulation Conference Proceedings, 53–59.
Robinson, S. (2001). Modes of simulation practice in business and the military. In Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304), (Vol. 1, pp. 805–811).
Robinson, S. (2006). Conceptual modeling for simulation: Issues and research requirements. In Proceedings of the 2006 Winter Simulation Conference (pp. 792–800).
Robinson, S. (2012). Tutorial: Choosing what to model; conceptual modeling for simulation. In Proceedings of the 2012 Winter Simulation Conference (WSC) (pp. 1–12).
Robinson, S. (2013). Conceptual modeling for simulation. In 2013 Winter Simulations Conference (WSC) (pp. 377–388).
Robinson, S. (2017). A tutorial on simulation conceptual modeling. In 2017 Winter Simulation Conference (WSC) (pp. 565–579).
Sargent, R. G. (1991). Simulation model verification and validation. In 1991 Winter Simulation Conference Proceedings (pp. 37–47).
Sargent, R. G. (2010). Verification and validation of simulation models. In Proceedings of the 2010 Winter Simulation Conference (pp. 166–183).
Sargent, R. G. (2011). Verification and validation of simulation models. In Proceedings of the 2011 Winter Simulation Conference (WSC) (pp. 183–198).
Sargent, R. G. (2013). An introduction to verification and validation of simulation models. In 2013 Winter Simulations Conference (WSC) (pp. 321–327).
Sargent, R. G. (2015). An introductory tutorial on verification and validation of simulation models. In 2015 Winter Simulation Conference (WSC) (pp. 1729–1740).
Sargent, R. G., & Balci, O. (2017). History of verification and validation of simulation models. In 2017 Winter Simulation Conference (WSC) (pp. 292–307).
Shannon, R. E. (1975). Systems simulation: the art and science. Englewood Cliffs, New Jersey: Prentice-Hall.
Shi, P., Liu, F., & Yang, M. (2008). Research on validation method for complex simulation systems. In 2008 Asia Simulation Conference—7th International Conference on System Simulation and Scientific Computing (pp. 888–892).
Shi, P., Liu, F., & Yang, M. (2009a). Quantify simulation verification and validation. In 2009 11th International Conference on Computer Modelling and Simulation (pp. 123–128).
Shi, P., Liu, F., Yang, M., & Wang, Z. (2009b). A fuzzy rules-based approach to analyzing human behavior models. In 2009 11th International Conference on Computer Modelling and Simulation (pp. 346–351).
Acknowledgements
This work has been supported by National Key R&D Program of China (2018YFC0830900), National Natural Science Foundation of China (61873094), Science and Technology Program of Guangzhou, China (201804010246), and Natural Science Foundation of Guangdong Province of China (2018A030313338).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Liu, F., Yang, M. (2019). The Management of Simulation Validation. In: Beisbart, C., Saam, N. (eds) Computer Simulation Validation. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-70766-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-70766-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70765-5
Online ISBN: 978-3-319-70766-2
eBook Packages: Computer ScienceComputer Science (R0)