Abstract
Hadoop is a distributed system infrastructure of cloud computing. Based on the characteristics of ant-based clustering algorithm, the paper implements the parallelization of this algorithm using MapReduce on Hadoop. The Map function calculates the average similarity of the object with its neighborhood objects. The Reduce function processes the objects with the Map outputs and updates related information of both ants and the objects to get ready for the next job. Results on the Hadoop clusters show that our method can significantly improve the computational efficiency with the premise of maintaining clustering accuracy.
This work is partially supported by the National Science Foundation of China (Nos. 61170111, 61003142 and 61152001) and the Fundamental Research Funds for the Central Universities (No. SWJTU11ZT08).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: Operating Systems Design and Implementation, pp. 137–149 (2004)
Apache Hadoop. Hadoop, http://hadoop.apache.org
Ghemawat, S., Gobioff, H., Leung, S.: The Google file system. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 29–43. ACM, Bolton Landing (2003)
Borthakur, D.: The Hadoop Distributed File System: Architecture and Design. The Apache Software Foundation, http://hadoop.apache.org
Wei, J., Ravi, V.T., Agrawal, G.: Comparing map-reduce and FREERIDE for data-intensive applications. In: IEEE International Conference on Cluster Computing and Workshops. CLUSTER 2009, pp. 1–10 (2009)
Smith, A.E.: Swarm intelligence: from natural to artificial systems. IEEE Transactions on Evolutionary Computation 4, 192–193 (2000)
Yang, Y., Kamel, M.: Clustering Ensemble Using Swarm Intelligence. In: IEEE Swarm Intelligence Symposium, pp. 65–71 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, Y., Ni, X., Wang, H., Zhao, Y. (2012). Parallel Implementation of Ant-Based Clustering Algorithm Based on Hadoop. In: Tan, Y., Shi, Y., Ji, Z. (eds) Advances in Swarm Intelligence. ICSI 2012. Lecture Notes in Computer Science, vol 7331. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30976-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-30976-2_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30975-5
Online ISBN: 978-3-642-30976-2
eBook Packages: Computer ScienceComputer Science (R0)