Skip to main content
SpringerLink
Log in

Validating behavioral models for reuse

  • Original Paper
  • Published:
Research in Engineering Design Aims and scope Submit manuscript

Abstract

When using a model to predict the behavior of a physical system of interest, engineers must be confident that, under the conditions of interest, the model is an adequate representation of the system. The process of building this confidence is called model validation. It requires that engineers have knowledge about the system and conditions of interest, properties of the model and their own tolerance for uncertainty in the predictions. To reduce time and costs, engineers often reuse preexisting models that other engineers have developed. However, if the user lacks critical parts of this knowledge, model validation can be as time consuming and costly as developing a similar model from scratch. In this article, we describe a general process for performing model validation for reused behavioral models that overcomes this problem by relying on the formalization and exchange of knowledge. We identify the critical elements of this knowledge, discuss how to represent it and demonstrate the overall process on a simple engineering example.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The terminology is not strictly uniform. The US Department of Defense publishes a definition for accreditation that corresponds to the ISO definition for certification (US DoD 2003).

References

  • Akman V, Surav M (1996) Steps toward formalizing context. AI Mag 17:55–72

    Google Scholar 

  • Akman V, Surav M (1997) The use of situation theory in context modeling. Comput Intell 13:427–438

    Article  Google Scholar 

  • Aughenbaugh JM, Paredis CJJ (2006) The value of using imprecise probabilities in engineering design. J Mech Des 128:969–979

    Article  Google Scholar 

  • Balci O, Ormsby WF (2000) Well-defined intended uses: an explicit requirement for accreditation of modeling and simulation applications. In: Proceedings of the 2000 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, San Francisco

  • Banks J, Gerstein D, Searles SP (1987) Modeling processes, validation and verification of complex systems: a survey. In: Orlando FL, Balci O (eds) Proceedings of the conference on methodology and validation. Society for Computer Simulation, pp 13–18

  • Barlas Y, Carpenter S (1990) Philosophical roots of model validation: two paradigms. Syst Dyn Rev 6:148–166

    Article  Google Scholar 

  • Basili VR, Briand LC, Melo W (1996) How reuse influences productivity in object-oriented systems. Commun ACM 39:104–116

    Article  Google Scholar 

  • Biggerstaff TJ, Richter C (1989) Reusability framework, assessment, and directions. In: Biggerstaff TJ, Perlis AJ (eds) Software reusability: volume 1: concepts and models. ACM Press, New York

  • Clemen RT (1996) Making hard decisions: an introduction to decision analysis, 2nd edn. Duxbury Press, Pacific Grove

    Google Scholar 

  • Der Kiureghian A (1989) Measures of structural safety under imperfect states of knowledge. J Struct Eng 115:1119–1139

    Google Scholar 

  • Falkenhainer B, Forbus KD (1991) Compositional modeling: finding the right model for the job. Artif Intell 51:95–143

    Article  Google Scholar 

  • Ferson S, Donald S (1998) Probability bounds analysis. In: Proceedings of the international conference on probabilistic safety assessment and management (PSAM4), 13–18 September. Springer, New York, pp 1203–1208

  • Ferson S, Ginzburg L (1996) Different methods are needed to propagate ignorance and variability. Reliabil Eng Syst Safety 54:133–144

    Article  Google Scholar 

  • Grosse IR, Milton-Benoit JM, Wileden JC (2005) Ontologies for supporting engineering analysis models. Artif Intell Eng Des Anal Manuf 19:1–18

    Google Scholar 

  • Guha RV, Lenat DB (1992) Language, representation and contexts. J Inf Process 15:340–349

    Google Scholar 

  • Guha RV, McCarthy J (2003) Varieties of contexts. In: Blackburn P, Ghidini C, Turner RM, Giunchiglia F (eds) Proceedings of the modeling and using context, 4th international and interdisciplinary conference, CONTEXT 2003. Springer, Stanford, pp 164–177

  • Herskovitz PJ (1991) A theoretical framework for simulation validation: Popper’s falsificationism. Int J Model Simul 11:56–58

    Google Scholar 

  • HLA Working Group (2000) IEEE standard for modeling and simulation (M&S) high level architecture (HLA)—framework and rules. Standard. IEEE, New York. IEEE STD 1516–2000

  • Hume D (1965) A treatise of human nature. Clarendon Press, Oxford

    Google Scholar 

  • Kleindorfer GB, Ganeshan R (1993) The philosophy of science and validation in simulation. In: Evans GW, Mollaghasemi M, Russell EC, Biles WE (eds) Proceedings of the winter simulation conference. Institute of Electrical and Electronic Engineers, pp 50–57

  • Krueger CW (1992) Software reuse. ACM Comput Surv 24:131–183

    Article  Google Scholar 

  • Lawrence DB (1999) The economic value of information. Springer, Heidelberg

    Google Scholar 

  • Ling J, Aughenbaugh JM, Paredis CJJ (2006) Managing the collection of information under uncertainty using information economics. J Mech Des 128:980–990

    Article  Google Scholar 

  • Malak RJ (2005) A framework for validating reusable behavioral models in engineering design. Masters thesis, Georgia Institute of Technology, Atlanta

  • McCarthy J (1993) Notes on formalizing context. In: Proceedings of the IJCAI-93: proceedings of the thirteenth international joint conference on artificial intelligence, Chambery, France, 28 August–3 September, Morgan Kaufmann, San Francisco, pp 555–560

  • Mocko G, Malak RJ, Paredis CJJ et al (2004) A knowledge repository for behavioral models in engineering design. In: Proceedings of the ASME computers and information in engineering conference, Salt Lake City. Paper No. CIE2004-57746

  • Morse KL, Lightner M, Little R et al (2006) Enabling simulation interoperability. IEEE Comput 39(1):115–117

    Google Scholar 

  • Naylor TH, Finger JM (1967) Verification of computer simulation models. Manag Sci 14:B92–B101

    Article  Google Scholar 

  • Nikolaidis E (2005) Types of uncertainty in design decision making. In: Nikolaidis E, Ghiocel DM, Singhal S (eds) Engineering design reliability handbook. CRC Press, New York

    Google Scholar 

  • Oberkampf WL, Trucano TG (2002) Verification and validation in computational fluid dynamics. Prog Aerosp Sci 38:209–272

    Article  Google Scholar 

  • Oberkampf WL, DeLand SM, Rutherford BM et al (2000) Estimation of total uncertainty in modeling and simulation. Sandia National Laboratories, Albuquerque. (SAND2000-0824)

    Google Scholar 

  • Pace D (2004) The referent study final report. US Defense Modeling and Simulation Office, Alexandria. (Report No. JWR-04-010)

    Google Scholar 

  • Parry GW (1996) The characterization of uncertainty in probabilistic risk assessment of complex systems. Reliabil Eng Syst Safety 54:119–126

    Article  Google Scholar 

  • Popper KR (1972) The logic of scientific discovery, 6th edn. Hutchinson, London

    Google Scholar 

  • Rae A, Robert P, Hausen H-L (1995) Software evaluation and certification: principles, practice and legal liability. McGraw-Hill, London

    Google Scholar 

  • Rekuc SJ, Aughenbaugh JM, Bruns M et al (2006) Eliminating design alternatives based on imprecise information. In: Proceedings of the society of automotive engineering world congress, Detroit, MI, 3–7 April. (Paper No. 2006-01-0272)

  • Robinson S, Nance RE, Paul RJ, et al. (2004) Simulation model reuse: definitions, benefits, and obstacles. Simul Model Pract Theor 12:479–494

    Article  Google Scholar 

  • Sargent RG (1985) An expository on verification and validation of simulation models. In: Gantz D, Blais G, Solomon S (eds) Proceedings of the 1985 winter simulation conference. Institute of Electrical and Electronics Engineers, San Francisco, pp 15–22

  • Schlesinger S, Crosbie RE, Innis GS et al (1979) Terminology for model credibility. Simulation 32:103–104

    Google Scholar 

  • Schlosser J, Paredis CJJ (2007) Managing multiple sources of epistemic uncertainty in engineering decision making. In: Proceedings of the society of automotive engineering world congress, Detroit, MI, 16–19 April. (Paper No. 2007-01-1481)

  • Shigley JE, Mischke CR (2001) Mechanical engineering design, 6th edn. McGraw-Hill, New York

    Google Scholar 

  • Simon HA (1996) The sciences of the artificial, 3rd edn. MIT Press, Cambridge

    Google Scholar 

  • Spiegel M, Reynolds PF Jr, Brogan DC (2005) A case study of model context for simulation composibility and reusability. In: Kuhl ME, Steiger NM, Armstrong FB, Joines JA (eds) Proceedings of the 2005 winter simulation conference. Institute of Electrical and Electronic Engineers, San Francisco

  • Szykman S, Sriram R, Bochenek C et al (1998) The NIST design repository project. In: Advances in soft computing—engineering design and manufacturing. Springer, London

  • United States Department of Defense (2003) DoD modeling and simulation (M&S) verification, validation and accreditation (VV&A). DoD instruction number 5000.61, GPO, Washington, DC, 24 February 2005. http://www.dtic.mil/whs/directives/corres/pdf/i500061_051303/i500061p.pdf

  • United States Nuclear Regulatory Commission (1983) PRA procedures guide. Washington, DC

  • Walley J (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall, New York

    MATH  Google Scholar 

  • Yilmaz L (2004) On the need for contextualized introspective models to improve reuse and composibility of defense simulations. J Def Model Simul 1:141–151

    Google Scholar 

Download references

Acknowledgments

The authors thank Jason Aughenbaugh, Jay Ling, Steven Rekuc, Morgan Bruns for their contributions to this work. This work is supported by the G.W.W. School of Mechanical Engineering at the Georgia Institute of Technology and NASA Ames Research Center under cooperative agreement NNA04CK40A.

Author information

Authors and Affiliations

  1. Systems Realization Laboratory, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Richard J. Malak Jr & Christiaan J. J. Paredis

Authors
  1. Richard J. Malak Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christiaan J. J. Paredis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard J. Malak Jr.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Malak, R.J., Paredis, C.J.J. Validating behavioral models for reuse. Res Eng Design 18, 111–128 (2007). https://doi.org/10.1007/s00163-007-0031-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00163-007-0031-0

Keywords

Search

Navigation