The Foundations of Verification in Modeling and Simulation

  • William J. RiderEmail author
Part of the Simulation Foundations, Methods and Applications book series (SFMA)


The practice of verification is grounded in mathematics highlighting the fundamental nature of its practice. Models of reality are fundamentally mathematical and verification assures the connection between the modeling intended and achieved in code. Code verification is a process where the correctness of a computer code for simulation and modeling is proven. This “proof” is defined by the collection of evidence that the numerical approximations are congruent with the model for the physical phenomena. The key metric in code verification is the order of accuracy of the approximation that should match theoretical expectations. In contrast, solution verification is an aspect of uncertainty estimation associated with numerical error in simulations. Solution verification uses many of the same approaches as code verification, but its principal outcome is an estimate of the numerical error. The order of convergence is a secondary outcome. Together these two practices form an important part of the foundation of quality and credibility in modeling and simulation.


Verification Error estimate Convergence Order-of-accuracy Solution verification Robust statistics 



The author would like to thank Tim Trucano, Vince Mousseau, Patrick Knupp, Bill Oberkampf, Chris Roy, Greg Weirs, and Patrick Roache for numerous discussions and informative suggestions. The DOE ASC V&V program has provided support for the development of the verification methodology. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, National Technology and Engineering Solutions of Sandia, LLC. Operated for the United States Department of Energy by National Technology and Engineering Solutions of Sandia, LLCDE-AC04-94AL85000.


  1. Ascher, U. M., & Petzold, L. R. (1998). Computer methods for ordinary differential equations and differential-algebraic equations, vol. 61. Siam.Google Scholar
  2. Banks, J. W., Aslam, T., & Rider, W. (2008). On sub-linear convergence for linearly degenerate waves in capturing schemes. Journal of Computational Physics, 227(14), 6985–7002.MathSciNetCrossRefGoogle Scholar
  3. Bjork, A. (1996). Numerical Methods for Least Squares Problems. SIAM.Google Scholar
  4. Brown, L. M., & Feynman, R. P. (2000). Selected papers of Richard Feynman: With commentary (Vol. 27), World scientific series in 20th century physics Singapore: World Scientific.Google Scholar
  5. Drikakis, D., & Rider, W. (2006). High-resolution methods for incompressible and low-speed flows. Springer Science & Business Media.Google Scholar
  6. Eça, L., Hoekstra, M., Hay, A., & Pelletier, D. (2007). On the construction of manufactured solutions for one and two-equation eddy-viscosity models. International Journal for Numerical Methods in Fluids, 54(2), 119–154.CrossRefGoogle Scholar
  7. Galli, G., & Pasquarello, A. (1993). First-principles molecular dynamics. In Computer simulation in chemical physics (Springer, pp. 261–313).Google Scholar
  8. Gartling, D. K. (1990). A test problem for outflow boundary conditions-flow over a backward-facing step. International Journal for Numerical Methods in Fluids, 11(7), 953–967.CrossRefGoogle Scholar
  9. Gottlieb, J., & Groth, C. P. (1988). Assessment of Riemann solvers for unsteady one-dimensional inviscid flows of perfect gases. Journal of Computational Physics, 78(2), 437–458.CrossRefGoogle Scholar
  10. Hairer, E., Nørsett, S. P., & Wanner, G. (1993). Solving ordinary differential equations. I, volume 8 of Springer Series in Computational Mathematics. Springer-Verlag, Berlin.Google Scholar
  11. Huber, P. (1996). Robust Statistical Procedures. SIAM.Google Scholar
  12. Kamm, J. R., Brock, J. S., Brandon, S. T., Cotrell, D. L., Johnson, B., Knupp, P., et al. (2008). Enhanced verification test suite for physics simulation codes. Los Alamos National Laboratory (LANL), Los Alamos, NM: Tech. rep.Google Scholar
  13. Kim, J., & Moin, P. (1985). Application of a fractional-step method to incompressible navier-stokes equations. Journal of Computational Physics, 59(2), 308–323.MathSciNetCrossRefGoogle Scholar
  14. Kotschenreuther, M., Dorland, W., Beer, M., & Hammett, G. (1995). Quantitative predictions of tokamak energy confinement from first-principles simulations with kinetic effects. Physics of Plasmas, 2(6), 2381–2389.CrossRefGoogle Scholar
  15. Lamb, H. (1932). Hydrodynamics. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  16. Lavery, D., & Burkhead, C. (Eds.). (2011). Joss Whedon: Conversations. Jackson: University Press of Mississippi.Google Scholar
  17. Lax, P. D., & Richtmyer, R. D. (1956). Survey of the stability of linear finite difference equations. Communications on Pure and Applied Mathematics, 9(2), 267–293.MathSciNetCrossRefGoogle Scholar
  18. Le, H., Moin, P., & Kim, J. (1997). Direct numerical simulation of turbulent flow over a backward-facing step. Journal of Fluid Mechanics, 330, 349–374.CrossRefGoogle Scholar
  19. Lin, P. T., Sala, M., Shadid, J. N., & Tuminaro, R. S. (2006). Performance of fully-coupled algebraic multilevel domain decomposition preconditioners for incompressible flow and transport. International Journal for Numerical Methods in Engineering, 67, 208–225.CrossRefGoogle Scholar
  20. Moin, P., & Mahesh, K. (1998). Direct numerical simulation: a tool in turbulence research. Annual Review of Fluid Mechanics, 30(1), 539–578.MathSciNetCrossRefGoogle Scholar
  21. Moser, R. D., Kim, J., & Mansour, N. N. (1999). Direct numerical simulation of turbulent channel flow up to re \(\tau \)= 590. Physics of Fluids, 11(4), 943–945.CrossRefGoogle Scholar
  22. Oberkampf, W. L., & Roy, C. J. (2010). Verification and validation in scientific computing. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  23. Pawlowski, R. P., Shadid, J. N., Simonis, J. P., & Walker, H. F. (2006). Globalization techniques for Newton-Krylov methods and applications to the fully coupled solution of the Navier-Stokes equations. SIAM Review, 48(4), 700–721.MathSciNetCrossRefGoogle Scholar
  24. Pippart, K. (2015). Operation Mexico!: Carl Kiekhaefer vs the 1951–1953 Pan American Road Race. Minneapolis: Mill City Press.Google Scholar
  25. Prabhakar, V., & Reddy, J. (2006). Spectral/hp penalty least-squares finite element formulation for the steady incompressible Navier-Stokes equations. Journal of Computational Physics, 215(1), 274–297.MathSciNetCrossRefGoogle Scholar
  26. Rider, W., Witkowski, W., Kamm, J. R., & Wildey, T. (2016). Robust verification analysis. Journal of Computational Physics, 307, 146–163.MathSciNetCrossRefGoogle Scholar
  27. Rider, W. J., Kamm, J. R., & Weirs, V. G. (2016). Procedures for calculation verification. Simulation Credibility, 31.Google Scholar
  28. Rider, W. J., Kamm, J. R., Weirs, V. G., & Cacui, D. G. (2010). Verification, validation and uncertainty quantification workflow in CASL. Albuquerque, NM: Sandia National Laboratories.Google Scholar
  29. Roache, P. (1998). Verification and validation in computational science and engineering. Hermosa Publishers.Google Scholar
  30. Roache, P. (2009). Fundamentals of verification and validation. Hermosa Publishers.Google Scholar
  31. Sagaut, P. Large eddy simulation for incompressible flows: An introduction. Springer Science & Business Media, 2006.Google Scholar
  32. Stern, F., Wilson, R. V., Coleman, H. W., & Paterson, E. G. (2001). Comprehensive approach to verification and validation of CFD simulations-part 1: Methodology and procedures. Journal Fluids Engrng., 123, 793802.Google Scholar
  33. Taylor, G. I., & Green, A. E. (1937). Mechanism of the production of small eddies from large ones. Proceedings of the Royal Society of London, 158(895), 499–521.zbMATHGoogle Scholar
  34. Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society. Series B (Methodological), 267–288.MathSciNetzbMATHGoogle Scholar
  35. Twain, M. (2015). Autobiography of Mark Twain. Vol. 3. The Mark Twain papers. University of California Press, Berkeley. A publication of the Mark Twain Project of the Bancroft Library.Google Scholar
  36. White, F. (1991). Viscous fluid flow. New York: McGraw-Hill Inc.Google Scholar

Copyright information

© This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2019

Authors and Affiliations

  1. 1.Sandia National Laboratories, Center for Computing ResearchAlbuquerqueUSA

Personalised recommendations