UMiner: A Data Mining System Handling Uncertainty and Quality

Abstract

The explosive growth of data collections in the science and business applications and the need to analyze and extract useful knowledge from this data leads to a new generation of tools and techniques grouped under the term data mining [FU96]. Their objective is to deal with volumes of data and automate the data mining and knowledge discovery from large data repositories. The majorities of data mining systems produce a particular enumeration of patterns over data sets accomplishing a limited set of tasks, such as clustering, classification and rules extraction [BL96, FPSU96]. However, there are some aspects in the data mining process that are under-addressed by the current approaches in database and data mining applications. These aspects are:

• i) the revealing and handling of uncertainty in the context of data mining tasks. In traditional data mining systems database values are not overlapping and treated equally in the classification process. The different values in the database are classified in the available categories in a crisp manner i.e. they may be classified into at most one cluster. Also all the values that are classified in a cluster belong to it with the same degree of belief Thus, there is significant information included in classification results that is not exploited by the traditional classification approaches.

• ii) the evaluation of data mining results based on well-established quality criteria. Most of the clustering algorithms depend on assumptions and initial guesses in order to define the subgroups presented in a data set [TK99]. As a consequence, in most applications the final clustering scheme requires some sort of evaluation.