Abstract
Mining high dimensional data is an urgent problem of great practical importance. Although some data mining models such as frequent patterns and clusters have been proven to be very successful for analyzing very large data sets, they have some limitations. Frequent patterns are inadequate to describe the quantitative correlations among nominal members. Traditional cluster models ignore distances of some pairs of members, so a pair of members in one big cluster may be far away. As a combination and complementary of both techniques, we propose the Maximal-Correlated-Member-Cluster (MCMC) model in this paper. The MCMC model is based on a statistical measure reflecting the relationship of nominal variables, and every pair of members in one cluster satisfy unified constraints. Moreover, in order to improve algorithm’s efficiency, we introduce pruning techniques to reduce the search space. In the first phase, a Tri-correlation inequation is used to eliminate unrelated member pairs, and in the second phase, an Inverse-Order-Enumeration-Tree (IOET) method is designed to share common computations. Experiments over both synthetic datasets and real life datasets are performed to examine our algorithm’s performance. The results show that our algorithm has much higher efficiency than the naïve algorithm, and this model can discover meaningful correlated patterns in high dimensional database.
This work is Supported by the National Natural Science Foundation of China under Grant No.60473072 and Grant No. 60473051.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules between Sets of Items in Large Databases. In: Proc. of 1993 Int. Conf. on Management of Data (SIGMOD 1993), pp. 207–216 (1993)
Agrawal, R., Srikant, R.: Fast Algorithms for Mining Association Rules in Large Databases. In: Proc. of 1994 Int. Conf. Very Large Data Bases (VLDB 1994), pp. 487–499 (1994)
Bayardo, R.J.: Efficiently mining long patterns from databases. In: Proc. of 1998 Int. Conf. on Management of Data (SIGMOD 1998), pp. 85–93 (1998)
Han, J., Pei, J., Yin, Y.: Mining Frequent Patterns Without Candidate Generation. In: Proc. of 2000 Int. Conf. on Management of Data (SIGMOD 2000), pp. 1–12 (2000)
Jiang, D., Pei, J., Zhang, A.: Mining Coherent Gene Clusters from Gene-Sample-Time Microarray Data. In: Proc. of Int. Conf. Knowledge Discovery and Data Mining (KDD 2004), pp. 430–439 (2004)
Liu, J., Wang, W.: OP-Cluster: Clustering by Tendency in High Dimensional Space. In: Proc. of the 3rd IEEE Int. Conf. on Data Mining (ICDM 2003), pp. 187–194 (2003)
Pei, J., Dong, G., Zou, W., Han, J.: Mining Condensed Frequent Pattern Bases. Knowledge and Information Systems 6(5), 570–594 (2004)
Pei, J., Han, J., Wang, W.: Mining sequential patterns with constraints in large databases. In: Proc. of ACM Conf. on Information and Knowledge Management (CIKM 2002), pp. 18–25 (2002)
Wang, H., Wang, W., Yang, J., Yu, P.S.: Clustering by Pattern Similarity in Large. Data Sets. In: Proc. of 2002 Int. Conf. on Management of Data (SIGMOD 2002), pp. 418–427 (2002)
Xiong, H., Shekhar, S., Tan, P.N., Kumar, V.: Exploiting a Support-based Upper Bound of Pearson’s Correlation Coefficient for Efficiently Identifying Strongly Correlated Pairs. In: Proc. of 2004 Int. Conf. Knowledge Discovery and Data Mining (KDD 2004), pp. 334–343 (2004)
Zaki, M.J.: Generating non-redundant association rules. In: Proc. of 2000 Int. Conf. Knowledge Discovery and Data Mining (KDD 2000), pp. 34–43 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jiang, L., Yang, D., Tang, S., Ma, X., Zhang, D. (2006). Mining Maximal Correlated Member Clusters in High Dimensional Database. In: Ng, WK., Kitsuregawa, M., Li, J., Chang, K. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2006. Lecture Notes in Computer Science(), vol 3918. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11731139_19
Download citation
DOI: https://doi.org/10.1007/11731139_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33206-0
Online ISBN: 978-3-540-33207-7
eBook Packages: Computer ScienceComputer Science (R0)