Lightweight Classifier-Based Outlier Detection Algorithms from Multivariate Data Stream

Abstract

Outlier detection is a preprocessing method that is useful in detecting unusual data if they are of interest to the users, or reducing irrelevant instances in machine learning if outliers are interpreted as noise. In the past decades, there are many outlier detection algorithms that have been developed and reported in the literature. Many have been implemented as software programs across a spectrum of various applications ranging from identifying special customers in marketing to fraud detection in security context. In chapter, a novel outlier detection method named lightweight analysis is proposed. Instead of using a full dataset to compute outlier values like the traditional outlier detection algorithms, the proposed concept of lightweight analysis works by examining only a certain “number of instances as a reference in order to calculate the outlier indicator, like cumulative analysis or lightweight analysis with sliding window, other than global analysis only.” These three mechanisms are then combined with the existing outlier measurements such as “interquartile, local outlier factor and Mahalanobis distance range.” In this study, the computer simulation experiments show encouraging results that support the fact that good classification accuracy is possible using reduced datasets. The outlier detection results are on par or more accurate than the existing state of the arts by using the “proposed classifier-based outlier detection (COD) method coupled with lightweight analysis.”