Controlling the Application of Knowledge-Based Systems

Abstract

This chapter aims to indicate some of the potential benefits and dangers inherent in the technology of Knowledge-Based Systems (KBSs) and to suggest courses of action that would both minimize risk and promote social benefit in future developments. KBSs offer cheap and ready access to knowledge, efficient management and organization of information, conversion of information into usable knowledge to guide human action, knowledge and advice relevant to the context in which it is needed. There are, however, potential dangers if a user is unfamiliar with the assumptions underlying KBS structures of reasoning and representing knowledge: the danger of a program becoming incomprehensible or unreliable in an emergency: the possibility of placing too much faith in the abilities of KBSs and thereby losing the capacity for critical evaluation of its judgments: the inevitable uncertainty or inexactitude programmed into KBSs: the economic implications at an uneven distribution of new technology: and the likelihood of misinterpretation or technical failure when autonomous. systems are automatically activating machinery (e. g. military/defence systems). The chapter argues for the regulation of KBS construction, particularly for those systems designed for public or “non-expert” use. Suggestions include programming into each system a clear qualification of answers and requests for data, setting up a workable Code of Conduct for AI practitioners, and constant supervision of the assumptions and activities of each computerized system. Since KBSs work by consulting an internal model of the world (supplied to them by a human programmer), their decisions must be challenged or ignored when necessary and always treated with an appropriate degree of scepticism