Abstract
The subject of validation and verification (V&V) of knowledge-based systems (KBS) has been one of decreasing importance in the last decade. Research and development in the field reduced drastically. One of the main reasons is the persistent software challenges and failures. These failures have been categorized in different ways. One initiative however, which most researchers in the field agree upon, is that the only way to eliminate these problems is by rigorously performing V&V. Although there have been vast improvements in the field of V&V methodology, studies indicate that KBS industry still lacks rigorous validation methods. In this paper, we review the most important validation paradigms described in literature for KBS during the years of their fame. Additionally, this article studies the significant methods, aims to reassess these methods in light of recent advances, and propose new future directions for validation of KBS.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdallah K, Mohammad T, Louis F (2007) Validation of intelligent systems: a critical study and a tool, Corus. In: Proceedings of the, international journal of soft computing, pp 191–198
Abel T, Gonzalez AJ (1997) Utilizing criteria to reduce a set of test cases for expert system validation. In: Proceedings of the 10th FLAIRS conference 1997, pp 402–406
Abel T, Knauf R, Gonzalez AJ (1996) Generation of a minimal set of test cases that is functionally equivalent to an exhaustive set, for use in knowledge-based system validation. In: Proceedings of the 9th FLAIRS conference 1996, pp 280–284
Batarseh F (2011) Incremental lifecycle validation of knowledge-based systems through commonKADS, Ph.D. Dissertation at the University of Central Florida, Spring 2011
Batarseh F, Gonzalez AJ, Knauf R (Sep 2010) Validation of knowledge-based systems through commonKADS. In: Proceedings of the 55th Internationales Wissenschaftliches Kolloquium (IWK), Ilmenau, Germany
Cook D (2010) Software processes? How bohring? Back talk in the magazine of crosstalk. J Def Softw Eng 23(1):39
Dibie-Barthelemy J, Haemmerle O, Salvat E (2006) A semantic validation of conceptual graphs. In: Proceedings of the journal of, knowledge-based systems, pp 498–511
Ginsberg A, Weiss S, Politakis P (1985) SEEK2: a generalized approach to automatic knowledge-base refinement. In: Proceedings of international joint conference on artificial intelligence (IJCAI), pp 367–374
Gonzalez AJ, Barr V (2000) Validation and verification of intelligent systems—what are they and how are they different? In: Proceedings of the journal of experimental & theoretical artificial intelligence, October-2000, pp 407–420
Gonzalez AJ, Gupta U, Chianese R (1996) Performance evaluation of a large diagnostic expert system using a heuristic test case generator. In: Proceedings of the engineering applications for artificial intelligence, vol 9, pp 275–284
Kelbasa H (2004) Context refinement—investigating the rule refinement completeness of SEEK/SEEK 2. In: Proceedings of the 15th European conference on, artificial intelligence, pp 123–154
Knauf R, Gonzalez AJ, Abel T (2002) A framework for validation of rule-based systems. IEEE Trans Syst Man Cybern 32(3):181–196
Knauf R, Tsuruta S, Gonzalez AJ (2007) Toward reducing human involvement in validation of knowledge-based systems. In: Proceedings of the IEEE transactions on systems, man and, cybernetics, vol 37, January 2007, pp 120–131
Lee S, O’Keefe R (1994) Developing a strategy for expert system validation and verification. In: IEEE 1994, proceedings of the IEEE transactions on systems, man and cybernetics, vol 24, pp 643–655
Lockwood S, Chen Z (1995) Knowledge validation of engineering expert systems. In: Proceedings of the journal of advances in software engineering, pp 97–104
Marcos M, Del Pobil D, Moisan S (2000) Model-based verification of knowledge-based systems: a case study. In: Proceedings of the IEEE journal on software, pp 163–167
Mengshoel OJ, Delab S, (1993) Knowledge validation: principles and practice. In: Proceedings of the journal of IEEE experts systems, pp 62–68
Min F, Yang M, Wang Z (2006) An intelligent validation system of simulation model. In: Proceedings of the fifth international IEEE conference on machine learning and, cybernetics, pp 1459–1464
Min F, Ma P, Yang M (2007) A knowledge base method for the validation of military simulation. In: Proceedings of the winter simulation conference-IEEE, pp 1395–1402
Morell L (1988) Use of meta knowledge in the verification of knowledge-based systems. In: Proceedings of the 1st international conference on industrial and engineering applications of artificial intelligence and, expert systems, pp 847–857
Mosquiera-Rey E, Moret-Bonillo V (2000) Validation of intelligent systems: a critical study and a tool. In: Proceedings of the journal of, expert systems with applications, pp 1–16
Mukkavilli L (2011) Publication number: US Patent 2011/0098963 A1, Context-based testing
Murrel S, Plant RA (1997) A survey of tools for the validation and verification of knowledge-based systems: 1985–1995. In: Proceedings of the, decision support systems, pp 307–323
O’Keefe RM, Balci O, Smith EP (1987) Validating expert system performance. Proc IEEE Expert 2(4):81–90
Onoyama T, Tsuruta S (2000) Validation method for intelligent systems. In: Proceedings of the, journal of experimental and theoretical artificial intelligence, pp 461–472
Preece A (2001) Evaluating verification and validation methods in knowledge engineering. In: Roy R(ed) Micro-Level Knowledge Management, Morgan–Kaufman pp 123–145
Santos E Jr, Dinh H (2004) Consistency of test case in validation of Bayesian knowledge bases. In: Proceedings of the 16th IEEE international conference on tools with artificial intelligence (ICTAI 2004)
Slafer L (2001) (Boeing Satellite Systems) Achieving software validation through simulation. In: Proceedings of the advanced dynamics international user conference (ADIUS)
Smith S, Kandel A (1990) Validation of expert systems. In: Proceedings of the third Florida artificial intelligence research symposium (FLAIRS), pp 197–201
Sommerville I (2007) Software engineering, 8th edition, Chapter 4. published by Addison Wesley
Tadji C, Laroussi T (2009) Dynamic verification of an object-rule knowledge base using colored Petri nets. In: Proceedings of the conference of systemic, cybernetics and informatics 2009, vol 4, pp 337–352
Turing A (1950) Computing machinery and intelligence. Proc Mind LIX(236):433–460
Vermesan A, Hogberg F (1999) Applicability of conventional software verification and validation to knowledge base components: a qualitative assessment. In: Proceedings of the 5th European symposium on validation and verification of knowledge based- systems-theory, tools and practice (EUROVAV ‘99), pp 343–364
Vinze A, Vogel D, Nunamaker J (1991) Performance evaluation of a knowledge-based system, a validation study. In: Proceedings of the journal of information and management. Elsevier, Amsterdam, pp 225–236
Wells S (1993) The VIVA method: a life cycle independent approach to KBS validation. In: Proceedings of the IEEE AAAI conference, pp 102–106
Wu C, Lee S (2002) KJ3-a tool assisting formal validation of knowledge-based systems. In: Proceedings of the international Journal of the human–computer studies, pp 495–525
Zlatareva N (1992) A framework for knowledge-based systems verification, validation and refinement: the VVR system. In: Proceedings of the 5th FLAIRS conference 1992, pp 10–14
Zlatareva N (1998) Knowledge refinement during development and field validation of expert systems. In: Proceedings of the 11th international FLAIRS conference, 1998, pp 467–472
Zlatareva N, Preece A (1994) State of the art in automated validation of knowledge-based systems. In: Proceedings of the journal of, expert systems with applications, pp 151–168
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Batarseh, F.A., Gonzalez, A.J. Validation of knowledge-based systems: a reassessment of the field. Artif Intell Rev 43, 485–500 (2015). https://doi.org/10.1007/s10462-013-9396-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-013-9396-9