Extraction of Outliers from Imbalanced Sets
In this paper, we presented an outlier detection method, designed for small datasets, such as datasets in animal group behaviour research. The method was aimed at detection of global outliers in unlabelled datasets where inliers form one predominant cluster and the outliers are at distances from the centre of the cluster. Simultaneously, the number of inliers was much higher than the number of outliers. The extraction of exceptional observations (EEO) method was based on the Mahalanobis distance with one tuning parameter. We proposed a visualization method, which allows expert estimation of the tuning parameter value. The method was tested and evaluated on 44 datasets. Excellent results, fully comparable with other methods, were obtained on datasets satisfying the method requirements. For large datasets, the higher computational requirement of this method might be prohibitive. This drawback can be partially suppressed with an alternative distance measure. We proposed to use Euclidean distance in combination with standard deviation normalization as a reliable alternative.
KeywordsOutlier analysis Distance based method Global outlier Single cluster Mahalanobis distance Biology
The work was supported by the University of Pardubice (PŠ) and the Czech Science Foundation grant number 17-20286S (NM).
- 1.MATLAB: Global optimization toolbox (R2016a) (2016). https://www.mathworks.com/help/gads/index.html
- 9.Gower, J., Lubbe, S., Roux, N.: Understanding Biplots. Wiley, New York (2010)Google Scholar
- 15.Kohl, M.: Performance measures in binary classification. Int. J. Stat. Med. Res. 1(1), 79–81 (2012)Google Scholar
- 19.Xu, L., Chow, M.Y., Taylor, L.S.: Using the data mining based fuzzy classification algorithm for power distribution fault cause identification with imbalanced data. In: 2006 IEEE PES Power Systems Conference and Exposition. pp. 1228–1233, October 2006Google Scholar