Advertisement

NeoLOD: A Novel Generalized Coupled Local Outlier Detection Model Embedded Non-IID Similarity Metric

  • Fan Meng
  • Yang GaoEmail author
  • Jing Huo
  • Xiaolong Qi
  • Shichao Yi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11439)

Abstract

Traditional generalized local outlier detection model (TraLOD) unifies the abstract methods and steps for classic local outlier detection approaches that are able to capture local behavior to improve detection performance compared to global outlier detection techniques. However, TraLOD still suffers from an inherent limitation for rational data: it uses traditional (Euclidean) similarity metric to pick out the context/reference set ignoring the effect of attribute structure. i.e., it is with the fundamental assumption that attributes and attribute values are independent and identically distributed (IID). To address the issue above, this paper introduces a novel Non-IID generalized coupled local outlier detection model (NeoLOD) and its instance (NeoLOF) for identifying local outliers with strong couplings. Concretely, this paper mainly includes three aspects: (i) captures the underlying attribute relations automatically by using the Bayesian network. (ii) proposes a novel Non-IID similarity metric to capture the intra-coupling and inter-coupling between attributes and attribute values. (iii) unifies the generalized local outlier detection model by incorporating the Non-IID similarity metric and instantiates a novel NeoLOF algorithm. Results obtained from 13 data sets show the proposed similarity metric can utilize the attribute structure effectively and NeoLOF can improve the performance in local outlier detection tasks.

Keywords

Local outlier detection Non-IID Attribute structure Coupled similarity metric 

Notes

Acknowledgements

This work was supported by the National Key R&D Program of China (2017YFB0702600, 2017YFB0702601), the National Natural Science Foundation of China (61432008, U1435214, 61503178, 61806092) and Jiangsu Natural Science Foundation (BK20180326).

References

  1. 1.
    Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 1–12 (2000)Google Scholar
  2. 2.
    Ernst, M., Haesbroeck, G.: Comparison of local outlier detection techniques in spatial multivariate data. Data Min. Knowl. Discov. 31(2), 371–399 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Kriegel, H.-P., Kröger, P., Schubert, E., Zimek, A.: LoOP: local outlier probabilities. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 1649–1652. ACM (2009)Google Scholar
  4. 4.
    Zhang, K., Hutter, M., Jin, H.: A new local distance-based outlier detection approach for scattered real-world data. In: Theeramunkong, T., Kijsirikul, B., Cercone, N., Ho, T.-B. (eds.) PAKDD 2009. LNCS (LNAI), vol. 5476, pp. 813–822. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-01307-2_84CrossRefGoogle Scholar
  5. 5.
    Schubert, E., Zimek, A., Kriegel, H.-P.: Local outlier detection reconsidered: a generalized view on locality with applications to spatial, video, and network outlier detection. Data Min. Knowl. Discov. 28(1), 190–237 (2014)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Song, X., Wu, M., Jermaine, C., Ranka, S.: Conditional anomaly detection. IEEE Trans. Knowl. Data Eng. 19(5), 631–645 (2007)CrossRefGoogle Scholar
  7. 7.
    Wang, X., Davidson, I.: Discovering contexts and contextual outliers using random walks in graphs. In: 2009 Ninth IEEE International Conference on Data Mining, ICDM 2009, pp. 1034–1039. IEEE (2009)Google Scholar
  8. 8.
    Zheng, G., Brantley, S.L., Lauvaux, T., Li, Z.: Contextual spatial outlier detection with metric learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2161–2170. ACM (2017)Google Scholar
  9. 9.
    Jian, S., Cao, L., Lu, K., Gao, H.: Unsupervised coupled metric similarity for non-IID categorical data. IEEE Trans. Knowl. Data Eng. 30, 1810–1823 (2018)CrossRefGoogle Scholar
  10. 10.
    Zhu, C., Cao, L., Liu, Q., Yin, J., Kumar, V.: Heterogeneous metric learning of categorical data with hierarchical couplings. IEEE Trans. Knowl. Data Eng. 30, 1254–1267 (2018)CrossRefGoogle Scholar
  11. 11.
    Chen, L., Liu, H., Pang, G., Cao, L.: Learning homophily couplings from non-IID data for joint feature selection and noise-resilient outlier detection. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-2017, pp. 2585–2591 (2017)Google Scholar
  12. 12.
    Pang, G., Cao, L., Chen, L., Liu, H.: Learning homophily couplings from non-IID data for joint feature selection and noise-resilient outlier detection. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 2585–2591. AAAI Press (2017)Google Scholar
  13. 13.
    Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65(1), 31–78 (2006)CrossRefGoogle Scholar
  14. 14.
    Wang, C., Dong, X., Zhou, F., Cao, L., Chi, C.H.: Coupled attribute similarity learning on categorical data. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 781–797 (2015)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Ienco, D., Pensa, R.G., Meo, R.: From context to distance: learning dissimilarity for categorical data clustering. ACM Trans. Knowl. Discov. Data 6(1), 1–25 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Fan Meng
    • 1
  • Yang Gao
    • 1
    Email author
  • Jing Huo
    • 1
  • Xiaolong Qi
    • 1
  • Shichao Yi
    • 1
  1. 1.State Key Lab for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations