Mining Maximal Correlated Member Clusters in High Dimensional Database
Mining high dimensional data is an urgent problem of great practical importance. Although some data mining models such as frequent patterns and clusters have been proven to be very successful for analyzing very large data sets, they have some limitations. Frequent patterns are inadequate to describe the quantitative correlations among nominal members. Traditional cluster models ignore distances of some pairs of members, so a pair of members in one big cluster may be far away. As a combination and complementary of both techniques, we propose the Maximal-Correlated-Member-Cluster (MCMC) model in this paper. The MCMC model is based on a statistical measure reflecting the relationship of nominal variables, and every pair of members in one cluster satisfy unified constraints. Moreover, in order to improve algorithm’s efficiency, we introduce pruning techniques to reduce the search space. In the first phase, a Tri-correlation inequation is used to eliminate unrelated member pairs, and in the second phase, an Inverse-Order-Enumeration-Tree (IOET) method is designed to share common computations. Experiments over both synthetic datasets and real life datasets are performed to examine our algorithm’s performance. The results show that our algorithm has much higher efficiency than the naïve algorithm, and this model can discover meaningful correlated patterns in high dimensional database.
KeywordsAssociation Rule Frequent Pattern High Dimensional Data Association Rule Mining Member Pair
Unable to display preview. Download preview PDF.
- 1.Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules between Sets of Items in Large Databases. In: Proc. of 1993 Int. Conf. on Management of Data (SIGMOD 1993), pp. 207–216 (1993)Google Scholar
- 2.Agrawal, R., Srikant, R.: Fast Algorithms for Mining Association Rules in Large Databases. In: Proc. of 1994 Int. Conf. Very Large Data Bases (VLDB 1994), pp. 487–499 (1994)Google Scholar
- 3.Bayardo, R.J.: Efficiently mining long patterns from databases. In: Proc. of 1998 Int. Conf. on Management of Data (SIGMOD 1998), pp. 85–93 (1998)Google Scholar
- 4.Han, J., Pei, J., Yin, Y.: Mining Frequent Patterns Without Candidate Generation. In: Proc. of 2000 Int. Conf. on Management of Data (SIGMOD 2000), pp. 1–12 (2000)Google Scholar
- 5.Jiang, D., Pei, J., Zhang, A.: Mining Coherent Gene Clusters from Gene-Sample-Time Microarray Data. In: Proc. of Int. Conf. Knowledge Discovery and Data Mining (KDD 2004), pp. 430–439 (2004)Google Scholar
- 6.Liu, J., Wang, W.: OP-Cluster: Clustering by Tendency in High Dimensional Space. In: Proc. of the 3rd IEEE Int. Conf. on Data Mining (ICDM 2003), pp. 187–194 (2003)Google Scholar
- 8.Pei, J., Han, J., Wang, W.: Mining sequential patterns with constraints in large databases. In: Proc. of ACM Conf. on Information and Knowledge Management (CIKM 2002), pp. 18–25 (2002)Google Scholar
- 9.Wang, H., Wang, W., Yang, J., Yu, P.S.: Clustering by Pattern Similarity in Large. Data Sets. In: Proc. of 2002 Int. Conf. on Management of Data (SIGMOD 2002), pp. 418–427 (2002)Google Scholar
- 10.Xiong, H., Shekhar, S., Tan, P.N., Kumar, V.: Exploiting a Support-based Upper Bound of Pearson’s Correlation Coefficient for Efficiently Identifying Strongly Correlated Pairs. In: Proc. of 2004 Int. Conf. Knowledge Discovery and Data Mining (KDD 2004), pp. 334–343 (2004)Google Scholar
- 11.Zaki, M.J.: Generating non-redundant association rules. In: Proc. of 2000 Int. Conf. Knowledge Discovery and Data Mining (KDD 2000), pp. 34–43 (2000)Google Scholar