Clustering Large Datasets Using Data Stream Clustering Techniques
Unsupervised identification of groups in large data sets is important for many machine learning and knowledge discovery applications. Conventional clustering approaches (k-means, hierarchical clustering, etc.) typically do not scale well for very large data sets. In recent years, data stream clustering algorithms have been proposed which can deal efficiently with potentially unbounded streams of data. This paper is the first to investigate the use of data stream clustering algorithms as light-weight alternatives to conventional algorithms on large non-streaming data. We will discuss important issue including order dependence and report the results of an initial study using several synthetic and real-world data sets.
KeywordsData Stream Data Stream Cluster Conventional Cluster Algorithm Data Stream Cluster Algorithm Data Stream Algorithm
This work is supported in part by the U.S. National Science Foundation as a research experience for undergraduates (REU) under contract number IIS-0948893 and by the National Institutes of Health under contract number R21HG005912.
- Aggarwal, C. (2007). Data streams: Models and algorithms. Advances in database systems (Vol. 31). New York: Springer.Google Scholar
- Aggarwal, C. C., Han, J., Wang, J., & Yu, P. S. (2003). A framework for clustering evolving data streams. In Proceedings of the 29th International Conference on Very Large Data Bases (VLDB ’03) (Vol. 29, pp. 81–92). VLDB Endowment.Google Scholar
- Bifet, A., Holmes, G., Kirkby, R., & Pfahringer, B. (2010). MOA: Massive online analysis. Journal of Machine Learning Research, 99, 1601–1604.Google Scholar
- Cao, F., Ester, M., Qian, W., & Zhou, A. (2006). Density-based clustering over an evolving data stream with noise. In Proceedings of the 2006 SIAM International Conference on Data Mining (pp. 328–339). Philadelphia: SIAM.Google Scholar
- Zhao, W., Ma, H., & He, Q. (2009) Parallel k-means clustering based on MapReduce. In Proceedings of the 1st International Conference on Cloud Computing, CloudCom ’09 (pp. 674–679). Berlin: Springer.Google Scholar