Advertisement

HOT: Hypergraph-Based Outlier Test for Categorical Data

  • Li Wei
  • Weining Qian
  • Aoying Zhou
  • Wen Jin
  • Jeffrey X. Yu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2637)

Abstract

As a widely used data mining technique, outlier detection is a process which aims at finding anomalies with good explanations. Most existing methods are designed for numeric data. They will have problems with real-life applications that contain categorical data. In this paper, we introduce a novel outlier mining method based on a hypergraph model. Since hypergraphs precisely capture the distribution characteristics in data subspaces, this method is effective in identifying anomalies in dense subspaces and presents good interpretations for the local outlierness. By selecting the most relevant subspaces, the problem of “curse of dimensionality” in very large databases can also be ameliorated. Furthermore, the connectivity property is used to replace the distance metrics, so that the distance-based computation is not needed anymore, which enhances the robustness for handling missing-value data. The fact, that connectivity computation facilitates the aggregation operations supported by most SQL-compatible database systems, makes the mining process much efficient. Finally, experiments and analysis show that our method can find outliers in categorical data with good performance and quality.

Keywords

Outlier Detection Frequent Itemsets Subspace Density Local Outlier Outlying Attribute 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    C. Aggarwal and P. Yu. Outlier detection for high dimensional data. In Proc. of SIGMOD’2001, pages 37–47, 2001.Google Scholar
  2. 2.
    R. Agrawal and R. Srikant. Fast algorithms for mining association rules in large databases. In Proc. of VLDB’94, pages 487–499, 1994.Google Scholar
  3. 3.
    V. Barnett and T. Lewis. Outliers In Statistical Data. John Wiley, Reading, New York, 1994.zbMATHGoogle Scholar
  4. 4.
    M. Breunig, H.-P. Kriegel, R. Ng, and J. Sander. Optics-of: Identifying local outliers. In Proc. of PKDD’99, pages 262–270, 1999.Google Scholar
  5. 5.
    M. Breunig, H.-P. Kriegel, R. Ng, and J. Sander. Lof: Identifying density-based local outliers. In Proc. of SIGMOD’2000, pages 93–104, 2000.Google Scholar
  6. 6.
    M. Ester, H.-P. Kriegel, J. Sander, and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. of KDD’96, pages 226–231, 1996.Google Scholar
  7. 7.
    S. Guha, R. Rastogi, and K. Shim. Cure: An efficient clustering algorithm for large databases. In Proc. of SIGMOD’98, pages 73–84, 1998.Google Scholar
  8. 8.
    D. Hawkins. Identification of Outliers. Chapman and Hall, Reading, London, 1980.zbMATHGoogle Scholar
  9. 9.
    F. Hussain, H. Liu, C. L. Tan, and M. Dash. Discretization: An enabling technique. Technical Report TRC6/99, National University of Singapore, School of Computing, 1999.Google Scholar
  10. 10.
    W. Jin, A. K. Tung, and J. Han. Mining top-n local outliers in large databases. In Proc. of KDD’2001, pages 293–298, 2001.Google Scholar
  11. 11.
    E. Knorr and R. Ng. Algorithms for mining distance-based outliers in large datasets. In Proc. of VLDB’98, pages 392–403, 1998.Google Scholar
  12. 12.
    E. Knorr and R. Ng. Finding intensional knowledge of distance-based outliers. In Proc. of VLDB’99, pages 211–222, 1999.Google Scholar
  13. 13.
    G. Merz and P. Murphy. Uci repository of machine learning databases. Technical Report, University of California, Department of Information and Computer Science: http://www.ics.uci.edu/mlearn/MLRepository.html, 1996.
  14. 14.
    F. Preparata and M. Shamos. Computational Geometry: an Introduction. Springer-Verlag, Reading, New York, 1988.Google Scholar
  15. 15.
    S. Ramaswamy, R. Rastogi, and K. Shim. Efficient algorithms for mining outliers from large data sets. In Proc. of SIGMOD’2000, pages 427–438, 2000.Google Scholar
  16. 16.
    I. Ruts and P. Rousseeuw. Computing depth contours of bivariate point clouds. Journal of Computational Statistics and data Analysis, 23:153–168, 1996.zbMATHCrossRefGoogle Scholar
  17. 17.
    S. Shekhar, C.-T. Lu, and P. Zhang. Detecting graph-based spatial outliers: Algorithms and applications (a summary of results). In Proc. of KDD’2001, 2001.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Li Wei
    • 1
  • Weining Qian
    • 1
  • Aoying Zhou
    • 1
  • Wen Jin
    • 2
  • Jeffrey X. Yu
    • 3
  1. 1.Department of Computer Science and EngineeringFudan UniversityChina
  2. 2.Department of Computer ScienceSimon Fraser UniversityCanada
  3. 3.Department of Systems Engineering and Engineering ManagementThe Chinese University of Hong KongChina

Personalised recommendations