Abstract
This paper presents a Parallel Data Mining Platform (PDMP), aiming at rapidly developing parallel data mining applications on cluster system. This platform consists of parallel data mining algorithm library, data warehouse, field knowledge base and platform middleware. The middleware is the kernel of the platform, which comprises data processing component, task manager, data manager, GUI, and so on. Taking advantage of cluster system, the middleware provides a convenient developing environment and effective bottom supports for implementation of parallel data mining algorithms. So far, the parallel data mining algorithm library has possessed several parallel algorithms, such as classification, clustering, association rule mining, sequence pattern mining and so on. With a register mechanism, the parallel data mining algorithm library is easy to be extended by users. Experiment results with the use of the PDMP show that there is a substantial performance improvement due to the cooperation of the middleware and corresponding parallel strategy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jiawei Han and Micheline Kamber: (1994) Data Mining: Concepts and Techniques. NY: Morgan Kaufman, pp.15.
Du Zhihui:(2001) High performance computing parallel programming technology-MPI parallel program design. Beijing Tsinghua University Press.
Albert Y. Zomaya, Tarek El-Ghazawi and Ophir Frieder. 1999 “Parallel and Distributed Computing for Data Mining”. IEEE Concurrency, pp.11–14, volume 1092–3063
J.S. Park, M.S. Chen, and P. S. Yu.: Efficient parallel data mining of association rules. 4th International Conference on information and Knowledge Management, Baltimore, Maryland,Novermber 1995.
Agrawal and J. Shafer.: Parallel mining of association rules. IEEE Transactions on knowledge and Data Engineering, 8(6): 962–969, June 1996
John Darlington, Moustafa M. Ghanem, etc.: Performance models for coordinating parallel data classification. In Proceedings of the Seventh International Parallel Computing Workshop (PCW-97), Canberra, Australia, September 1997.
Kilian Stoffel and Abdelkader Belkoniene.: Parallel k/h-means Clustering for large datasets. In Proceeding of Europar-99, Lecture Notes in Computer Science (LNCS) Volume 1685, pages 1451–1454. Spring Verlag, August 1999.
Takahiko Shintani and Masaru Kitsuregawa, “Mining Algorithms for Sequential Patterns in Parallel: Hash Based Approach”. Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp.283–294. 1998.
Rajkumar Buyya: High Performance ClusterComputing: Architectures and Systems Volume 1. People post press. 2002. P3838.
R Lottiaux, C Morin.: File mapping in shared virtual memory using a parallel file system. In: Proc of Parallel Computing’ 99. Delft, Netherlands, 1999. pp.606–614.
M. J. Zaki, S. Parthasarathy, and W. Li.: A localized algorithm for parallel association mining. 9th Annual ACM Symposium on Parallel Algorithms and Architectures, Newport, Rhode Island, June 1997.
YAN Sheng-xiang, WU Shao-chun, WU Geng-feng.: A Data-Vertical-Partitioning Based Parallel Decision Tree Classification Algorithm. Application Research of Computers. To be published in 2004.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, S., Wu, G., Yu, Z., Ban, H. (2005). A Platform for Parallel Data Mining on Cluster System. In: Zhang, W., Tong, W., Chen, Z., Glowinski, R. (eds) Current Trends in High Performance Computing and Its Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-27912-1_15
Download citation
DOI: https://doi.org/10.1007/3-540-27912-1_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25785-1
Online ISBN: 978-3-540-27912-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)