The Use of Experimental Data in Simulation Model Validation

  • David J. Murray-SmithEmail author
Part of the Simulation Foundations, Methods and Applications book series (SFMA)


The use of experimental data for the validation of deterministic dynamic simulation models based on sets of ordinary differential equations and algebraic equations is discussed. Comparisons of model and target system data are considered using graphical methods and quantitative measures in the time and frequency domains. System identification and parameter estimation methods are emphasized, especially in terms of identifiability analysis which can provide valuable information for experiment design. In general, experiments that are suitable for system identification are also appropriate for model validation. However, there is a dilemma since models are needed for this design process. The experiment design, data collection and analysis of model validation results is, inevitably, an iterative process, and experiments designed for model validation can never be truly optimal. A model of the pulmonary gas exchange processes in humans is used to illustrate some issues of identifiability, experiment design and test input selection for model validation.


Graphical comparisons Quantitative measures Identifiability Experiment design Test inputs 


  1. Bache, R. A. (1981). Time-domain system identification applied to non-invasive estimation of cardio-pulmonary quantities. Ph.D. thesis, University of Glasgow, UK.Google Scholar
  2. Bache, R. A., Gray, W. M., & Murray-Smith, D. J. (1981). Time-domain system identification applied to non-invasive estimation of cardio-pulmonary quantities. IEE Proceedings, 128, Part D, 56–64.Google Scholar
  3. Bache, R. A., & Murray-Smith, D. J. (1983). Structural and parameter identification of two lung gas-exchange models. In G. C. Vansteenkiste & P. C. Young (Eds.), Modelling and data analysis in biotechnology and medical engineering (pp. 175–188). Amsterdam, The Netherlands: North-Holland.Google Scholar
  4. Beck, J. V., & Arnold, K. J. (1977). Parameter estimation in science and engineering. New York, NY: Wiley.zbMATHGoogle Scholar
  5. Bellman, R., & Åström, K. J. (1970). On structural identifiability. Mathematical Biosciences, 7, 329–339.CrossRefGoogle Scholar
  6. Bradley, R., Padfield, G. D., Murray-Smith, D. J., & Thomson, D. G. (1990). Validation of helicopter mathematical models. Transactions of the Institute of Measurement and Control, 12, 186–196.CrossRefGoogle Scholar
  7. Brown, F., & Godfrey, K. R. (1978). Problems of determinacy in compartmental modeling with application to bilirubin kinetics. Mathematical Biosciences, 40, 205–224.CrossRefGoogle Scholar
  8. Bryce, G. W., Foord, T. R., Murray-Smith, D. J., & Agnew, P. (1976). Hybrid simulation of water-turbine governors. In R. E. Crosbie & J. L. Hay (Eds.), Simulation councils proceedings series (Vol. 6(1), pp. 35–44). La Jolla CA: Simulation Councils Inc.Google Scholar
  9. Chatfield, C. (1996). The analysis of time series. An introduction (5th ed.). London, UK: Chapman and Hall.zbMATHGoogle Scholar
  10. Federov, V. V. (1972). Theory of optimal experiments. New York, NY: Academic Press.Google Scholar
  11. Goodwin, G. C., & Payne, R. L. (1977). Dynamic system identification: Experiment design and data analysis. New York, NY: Academic Press.zbMATHGoogle Scholar
  12. Gustavsson, I. (1972). Comparison of different methods for identification of industrial processes. Automatica, 8, 127–142.CrossRefGoogle Scholar
  13. Hahn C. E. W. & Farmery, A. D. (2003). Gas exchange modelling: no more gills please. British Journal Anaesthesia, 91(1): 2–15. Google Scholar
  14. Heylen, W., & Lammens, S. (1996). FRAC: A consistent way of comparing frequency response functions. In M. I. Friswell & J. E. Mottershead (Eds.), Proceedings of Conference on Identification in Engineering Systems, Swansea, 1996 (pp. 48–57). Swansea, UK: University of Wales.Google Scholar
  15. Hughes, I. G., & Hase, T. P. A. (2010). Measurements and their uncertainties: A practical guide to modern error analysis. Oxford, UK: Oxford University Press.zbMATHGoogle Scholar
  16. Hunter, W. G., Hill, W. J., & Henson, T. L. (1969). Designing experiments for precise estimation of βsome of the constants in a mechanistic model. Canadian Journal of Chemical Engineering, 47, 76–80.CrossRefGoogle Scholar
  17. Huynh, D. P. B., Knezevic, D. J., & Patera, A. T. (2012). Certified reduced basis model characterization: a frequentistic uncertainty framework. Computer Methods in Applied Mechanics and Engineering, 201, 13–24.MathSciNetCrossRefGoogle Scholar
  18. Jachner, S., van den Boogaart, K. G., & Petzoldt, T. (2007) Statistical methods for the qualitative assessment of dynamic models with time delay (R Package qualV). Journal of Statistical Software, 22(8).Google Scholar
  19. Kammel, G., Voigt, H. M., & Neβ, K. (2005). Development of a tool to improve the forecast accuracy of dynamic simulation models for the paper process. In: J. Kappen, J. Manninen, & R. Ritala (Eds.), Proceedings of Model Validation Workshop, 6th October 2005, Espoo, Finland. Espoo, Finland: VTT Technical Research Centre.Google Scholar
  20. Kocijan, J., Girard, A., Banko, B., & Murray-Smith, R. (2005). Dynamic system identification with dynamic processes. Mathematical and Computer Modelling of Dynamic Systems, 11, 411–424.CrossRefGoogle Scholar
  21. Ljung, L. (1999). System identification: Theory for the user (2nd ed.). Upper Saddle River, NJ: Prentice Hall.zbMATHGoogle Scholar
  22. McFarland, J., & Mahadevan, S. (2008). Multivariate significance tests and model calibration under uncertainty. Computer Methods in Applied Mechanics and Engineering, 197(29–32), 2407–2479.zbMATHGoogle Scholar
  23. Murphy, T. W. (1969). Modelling of lung gas exchange - mathematical models of the lung. Mathematical Biosciences, 5, 427–447.CrossRefGoogle Scholar
  24. Murray-Smith, D. J. (2012). Modelling and simulation of integrated systems in engineering: Issues of methodology, quality, testing and application. Cambridge, UK: Woodhead Publishing.CrossRefGoogle Scholar
  25. Murray-Smith, D. J. (2015). Testing and validation of computer simulation models: Principles, methods and applications. Cham, Switzerland: Springer.CrossRefGoogle Scholar
  26. Nelles, O. (2001). Nonlinear system identification. Berlin, Germany: Springer.CrossRefGoogle Scholar
  27. Pack, A. I. (1976). Mathematical models of lung function. Ph. D. thesis, University of Glasgow, UK.Google Scholar
  28. Priestley, M. B. (1981). Spectral analysis and time series. San Diego, CA: Academic Press.zbMATHGoogle Scholar
  29. Raol, J. R., Girija, G., & Singh, J. (2004). Modelling and parameter estimation of dynamic systems. IET Control Engineering Series No. 65. London, UK: IET.Google Scholar
  30. Rosenberg, J. R., Murray-Smith, D. J., & Rigas, A. (1982). An introduction to the application of system identification techniques to elements of the neuromuscular system. Transactions of the Institute of Measurement & Control, 4, 187–201.CrossRefGoogle Scholar
  31. Smith, M. I., Murray-Smith, D. J., & Hickman, D. (2007). Verification and validation issues in a generic model of electro-optic sensor systems. Journal of Defense Modeling and Simulation, 4, 17–27.CrossRefGoogle Scholar
  32. Söderström, T., & Stoica, P. (1989). System identification. New York, NY: Prentice-Hall.zbMATHGoogle Scholar
  33. Tischler, M. B., & Remple, R. K. (2006). Aircraft and rotorcraft system identification. USA: AIAA.Google Scholar
  34. Thompson, K. R. (2009). Implementation of gaussian process models for non-linear system identification. Ph.D. thesis, University of Glasgow.Google Scholar
  35. Tomlinson, S. P., Tilley, D. G., & Burrows, C. R. (1994). Computer simulation of the human breathing process. IEEE Engineering in Medicine and Biology Magazine, 13, 115–124.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of EngineeringUniversity of GlasgowGlasgowUK

Personalised recommendations