Advertisement

Mining Maximal Correlated Member Clusters in High Dimensional Database

  • Lizheng Jiang
  • Dongqing Yang
  • Shiwei Tang
  • Xiuli Ma
  • Dehui Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3918)

Abstract

Mining high dimensional data is an urgent problem of great practical importance. Although some data mining models such as frequent patterns and clusters have been proven to be very successful for analyzing very large data sets, they have some limitations. Frequent patterns are inadequate to describe the quantitative correlations among nominal members. Traditional cluster models ignore distances of some pairs of members, so a pair of members in one big cluster may be far away. As a combination and complementary of both techniques, we propose the Maximal-Correlated-Member-Cluster (MCMC) model in this paper. The MCMC model is based on a statistical measure reflecting the relationship of nominal variables, and every pair of members in one cluster satisfy unified constraints. Moreover, in order to improve algorithm’s efficiency, we introduce pruning techniques to reduce the search space. In the first phase, a Tri-correlation inequation is used to eliminate unrelated member pairs, and in the second phase, an Inverse-Order-Enumeration-Tree (IOET) method is designed to share common computations. Experiments over both synthetic datasets and real life datasets are performed to examine our algorithm’s performance. The results show that our algorithm has much higher efficiency than the naïve algorithm, and this model can discover meaningful correlated patterns in high dimensional database.

Keywords

Association Rule Frequent Pattern High Dimensional Data Association Rule Mining Member Pair 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules between Sets of Items in Large Databases. In: Proc. of 1993 Int. Conf. on Management of Data (SIGMOD 1993), pp. 207–216 (1993)Google Scholar
  2. 2.
    Agrawal, R., Srikant, R.: Fast Algorithms for Mining Association Rules in Large Databases. In: Proc. of 1994 Int. Conf. Very Large Data Bases (VLDB 1994), pp. 487–499 (1994)Google Scholar
  3. 3.
    Bayardo, R.J.: Efficiently mining long patterns from databases. In: Proc. of 1998 Int. Conf. on Management of Data (SIGMOD 1998), pp. 85–93 (1998)Google Scholar
  4. 4.
    Han, J., Pei, J., Yin, Y.: Mining Frequent Patterns Without Candidate Generation. In: Proc. of 2000 Int. Conf. on Management of Data (SIGMOD 2000), pp. 1–12 (2000)Google Scholar
  5. 5.
    Jiang, D., Pei, J., Zhang, A.: Mining Coherent Gene Clusters from Gene-Sample-Time Microarray Data. In: Proc. of Int. Conf. Knowledge Discovery and Data Mining (KDD 2004), pp. 430–439 (2004)Google Scholar
  6. 6.
    Liu, J., Wang, W.: OP-Cluster: Clustering by Tendency in High Dimensional Space. In: Proc. of the 3rd IEEE Int. Conf. on Data Mining (ICDM 2003), pp. 187–194 (2003)Google Scholar
  7. 7.
    Pei, J., Dong, G., Zou, W., Han, J.: Mining Condensed Frequent Pattern Bases. Knowledge and Information Systems 6(5), 570–594 (2004)CrossRefGoogle Scholar
  8. 8.
    Pei, J., Han, J., Wang, W.: Mining sequential patterns with constraints in large databases. In: Proc. of ACM Conf. on Information and Knowledge Management (CIKM 2002), pp. 18–25 (2002)Google Scholar
  9. 9.
    Wang, H., Wang, W., Yang, J., Yu, P.S.: Clustering by Pattern Similarity in Large. Data Sets. In: Proc. of 2002 Int. Conf. on Management of Data (SIGMOD 2002), pp. 418–427 (2002)Google Scholar
  10. 10.
    Xiong, H., Shekhar, S., Tan, P.N., Kumar, V.: Exploiting a Support-based Upper Bound of Pearson’s Correlation Coefficient for Efficiently Identifying Strongly Correlated Pairs. In: Proc. of 2004 Int. Conf. Knowledge Discovery and Data Mining (KDD 2004), pp. 334–343 (2004)Google Scholar
  11. 11.
    Zaki, M.J.: Generating non-redundant association rules. In: Proc. of 2000 Int. Conf. Knowledge Discovery and Data Mining (KDD 2000), pp. 34–43 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Lizheng Jiang
    • 1
  • Dongqing Yang
    • 1
  • Shiwei Tang
    • 1
    • 2
  • Xiuli Ma
    • 2
  • Dehui Zhang
    • 2
  1. 1.School of Electronics Engineering and Computer SciencePeking UniversityBeijingChina
  2. 2.National Laboratory on Machine Perception, School of Electronics Engineering and Computer SciencePeking UniversityBeijingChina

Personalised recommendations