The Method of Manufactured Solutions for Code Verification
Verification of codes that numerically approximate solutions of partial differential equations consists in demonstrating that the code is free of coding errors and is capable, given sufficient discretization, of approaching exact mathematical solutions. This requires the evaluation of discretization errors using known benchmark solutions. The best benchmarks are exact analytical solutions with a sufficiently complex solution structure; they need not be physically realistic since verification is a purely mathematical exercise. The Method of Manufactured Solutions (MMS) provides a straightforward and general procedure for generating such solutions. For complex codes, the method utilizes symbolic manipulation, but here it is illustrated with simple examples. When used with systematic grid refinement studies, which are remarkably sensitive, MMS can produce robust code verifications with a strong completion point.
KeywordsManufactured solutions Simulation Benchmark Verification Turbulence Convergence Symbolic manipulation
I gratefully acknowledge help from C. Beisbart, L. Eça, P. Moin, W. L. Oberkampf, C. J. Roy, N. Saam, L. Shunn. and especially W. J. Rider.
- ASME. (2006). ASME V&V 10-2006. Guide for verification and validation in computational solid dynamics.Google Scholar
- ASME. (2009). ASME V&V 20-2009. Standard for verification and validation in computational fluid dynamics and heat transfer.Google Scholar
- Burg, C. O. E., & Murali, V. K. (2004). Efficient code verification using the residual formulation of the method of manufactured solutions. AIAA Paper 2004-2628, 34th AIAA Fluid Dynamics Conference, Portland, Oregon, June, 2004.Google Scholar
- Bueler, E., Brown, J. (2009). Shallow shelf approximation as a “sliding law” in a thermomechanically coupled ice sheet model. Journal of Geophysical Research, 114, F03008. https://doi.org/10.1029/2008jf001179.
- Choudhary, A., Roy, C. J., Dietiker, J.-F., Shahnam, M. & Garg, R. (2014). Code verification for multiphase flows using the method of manufactured solutions, FEDSM2014-21608. In Proceedings of the ASME 2014 4th Joint US-European Fluids Engineering Division Summer Meeting (FEDSM). Chicago, IL, August 3–7, 2014.16 M3.Google Scholar
- Eça, L. (2006). Workshop Website. http://maretec.ist.utl.pt/~maretec.daemon/html_files/CFD_workshops/Workshop_2006.htm.
- Eça, L., & Hoekstra, M. (2007a). Evaluation of numerical error estimation based on grid refinement studies with the method of manufactured solutions. Report D72-42, MARIN, May 2007.Google Scholar
- Eça, L., & Hoekstra, M. (2007b). Code verification of unsteady flow solvers with the method of manufactured solutions. Paper No. ISOPE-2007-565, International Society of Offshore and Polar Engineers.Google Scholar
- Eça, L., Hoekstra, M., Roache, P. J., & Coleman, H. (2009). Code verification, solution verification and validation: An overview of the 3rd Lisbon Workshop. AIAA Paper No. 2009-3647, 19th AIAA Computational Fluid Dynamics, San Antonio, Texas, June 2009.Google Scholar
- Fickett, W., & Rivard, C. (1974). Test problems for hydrocodes. Los Alamos, New Mexico:Los Alamos Scientific Laboratory, Report No. LA-5479.Google Scholar
- IEEE. (1991). IEEE standard glossary of software engineering terminology, IEEE Std 610.12-1990, New York, IEEE.Google Scholar
- Murali, V., Burg, C. O. E. (2002). Verification of 2D navier-stokes codes by the method of manufactured solutions. AIAA Paper 2002-3109, 32nd AIAA Fluid Dynamics Conference, St. Louis, June, 2002.Google Scholar
- Oberkampf, W. L., & Trucano, T. G. (2002). Verification and validation in computational fluid dynamics. AIAA Progress in Aerospace Sciences.Google Scholar
- Pautz, S. D. (2001). Verification of transport codes by the method of manufactured solutions: The ATTILA experience. In Proceedings of the ANS International Meeting on Mathematical Methods for Nuclear Applications, M&C 2001. Salt Lake City, Utah, Sept 2001.Google Scholar
- Pelletier, D., & Roache, P. J. (2006). Verification and validation of computational heat transfer. In W. J. Minkowycz, E. M. Sparrow, & J. Y. Murthy (Eds.), Handbook of Numerical Heat Transfer (2nd ed.). New York:Wiley.Google Scholar
- Rider, W. J. (2018). Personal communication 5/5/2018.Google Scholar
- Roache, P. J. (1993). A method for uniform reporting of grid refinement studies, ASME FED-Vol. 158. In I. Celik, C. J. Chen, P. J. Roache, & G. Scheurer (Eds.), Quantification of uncertainty in computational fluid dynamics. ASME Fluids Engineering Division Summer Meeting, Washington, DC, 20–24 June 1993, pp. 109–120.Google Scholar
- Roache, P. J. (1998a). Verification and validation in computational science and engineering. Albuquerque, NM: Hermosa Publishers.Google Scholar
- Roache, P. J. (1998b). Fundamentals of computational fluid dynamics. Albuquerque, NM: Hermosa Publishers.Google Scholar
- Roache, P. J. (2009). Fundamentals of verification and validation, Hermosa Publishers, Albuquerque, NM, Ch. 3 and Appendix C.Google Scholar
- Roache, P. J. (2012). A defense of computational physics. Albuquerque, NM: Hermosa Publishers.Google Scholar
- Roy, C. J. (2001). Grid convergence error analysis for mixed-order numerical schemes. AIAA Paper 2001–2606, June 2001 (Anaheim).Google Scholar
- Roy, C. J. (2015). Code verification: past, present and future, keynote lecture. In ASME V&V Symposium, La Vegas, NV, 13 May 2015.Google Scholar
- Roy, C. J., McWherter-Payne, M. A., & Oberkampf, W. L. (2000). Verification and validation for laminar hypersonic flowfields, AIAA 2000-2550, June 2000 (Denver).Google Scholar
- Salari, K. & Knupp, P. (2000). Code verification by the method of manufactured solutions, SAND2000-1444, Sandia National Laboratories, Albuquerque, NM 87185, June 2000.Google Scholar
- Sinclair, G. B., Beisheim, J. R., & Sezer, S. (2006). Practical convergence-divergence checks for stresses from FEA. In Proceedings of the 2006 international ANSYS users conference and exposition, 2–4 May 2006, Pittsburgh, PA.Google Scholar
- Steinberg, S., & Roache, P. J. (1986b). Grid generation: A variational and symbolic-computation approach. In Proceedings numerical grid generation in fluid dynamics conference, July 1986, Landshut, W. Germany.Google Scholar
- Wang, S. S. Y., Jia, Y., Roache, P. J., Smith, P. E., & Schmalz, R. A. Jr., (Eds.). (2009). Verification and validation of 3D free-surface flow model. ASCE/EWRI Task Committee.Google Scholar