Evidential techniques in parallel Database Mining

Abstract

Realisation of the fact that stored masses of data contain more information than what is obvious has led to a great interest in the field of Database Mining in the last couple of years. While hardware requirements for storage of these masses of data have advanced rapidly with the demand as have software methodologies for storage, manipulation and reporting of the data, little progress has been made in methods for automatically analysing the data and extracting knowledge stored implicitly within the data. This process of “reading between the lines” is called Database Mining (DM).

Clearly, the process of DM is a difficult one. This is due to the fact that methods required to achieve the goal of discovering knowledge are complex and data intensive. In this paper we explain how high performance computing can play a vital role in DM and discuss the implementation of a specific algorithm, STRIP (Strong Rule Induction in Parallel) [ANAN94b, ANAN95] developed by the authors for the discovery of Strong or “almost exact” rules from databases. STRIP is the first algorithm to be implemented within a parallel framework for Database Mining based on Evidence Theory (EDM) [ANAN94a] developed by the authors.

In an earlier paper we discussed the different levels of parallelism within STRIP and demonstrated them using a transputer network [ANAN95]. In this paper we discuss the implementation of STRIP on a cluster of Silicon Graphics Workstations connected using an ATM network.