Abstract
Data mining has attracted extensive research for several decades. As an important task of data mining, classification plays an important role in information retrieval, web searching, CRM, etc. Most of the present classification techniques are serial, which become impractical for large dataset. The computing resource is under-utilized and the executing time is not waitable. Provided the program mode of MapReduce, we propose the parallel implementation methods of several classification algorithms, such as k-nearest neighbors, naive bayesian model and decision tree, etc. Preparatory experiments show that the proposed parallel methods can not only process large dataset, but also can be extended to execute on a cluster, which can significantly improve the efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1997)
Weiss, S.M., Kulikowski, C.A.: Computer Systems that Learn: Classification and Prediction Methods from Statistics. In: Neural Nets, Machine Learning, and Expert Systems. Morgan Kaufmann, San Francisco (1991)
Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine learning, neural and statistical classification. Ellis Horwood, New York (1994)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley and Sons, Chichester (2001)
Cover, T., Hart, P.E.: Nearest neighbor pattern classification. IEEE Trans. Information Theory 13, 21–27 (1967)
Dasarathy, B.V.: Nearest Neighbor (NN) Norms: NN Pattern Classification Techiniques. IEEE Computer Society Press, Los Alamitos (1991)
Weiss, A.: Computing in the Clouds. netWorker 11(4), 16–25 (2007)
Buyya, R., Yeo, C.S., Venugopal, S.: Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities. In: Proc. of the 10th IEEE International Conference on High Performance Computing and Communications, China (2008)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: Proc. of the 6th Symposium on Operating System Design and Implementation, USA (2004)
Elsayed, T., Jimmy, L., Douglas, W.O.: Pairwise Document Similarity in Large Collections with MapReduce. In: Proc. of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies, pp. 262–268 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
He, Q., Zhuang, F., Li, J., Shi, Z. (2010). Parallel Implementation of Classification Algorithms Based on MapReduce. In: Yu, J., Greco, S., Lingras, P., Wang, G., Skowron, A. (eds) Rough Set and Knowledge Technology. RSKT 2010. Lecture Notes in Computer Science(), vol 6401. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16248-0_89
Download citation
DOI: https://doi.org/10.1007/978-3-642-16248-0_89
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16247-3
Online ISBN: 978-3-642-16248-0
eBook Packages: Computer ScienceComputer Science (R0)