Abstract
Scheduling is NP-hard problem in Hadoop, because scheduling algorithm must use available resources to complete assignments in the shortest time. This paper proposes an improved Genetic-Particle Swarm Optimization (IG-PSO) algorithm to solve scheduling problems. Traditional PSO algorithm is easy to fall into local optimum solution, so novel improved Genetic-Particle Swarm Optimization (IG-PSO) algorithm introduced GA’s mutation and crossover to overcome the shortcoming and increase the ability of global optimization. Compared with traditional PSO and GA, the experiment simulation shows that IG-PSO algorithm can escape from local optimal solution and find a better global optimal solution. Because the position of PSO particle falls into local optimal solution, GA uses mutation and crossover to diversify particles, which make the particle escape out of local optima.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Luis, M.V., Luis, R.-M., Juan, C., Maik, L.: A break in the clouds: towards a cloud GAfinition. SIGCOMM Comput. Commun. Rev. 39(1), 50–55 (2008)
Michael, A., Armando, F., Rean, G., et al.: A view of cloud computing. Commun. ACM. 53(4), 50–58 (2010)
Dean, J., Ghemawat, S., et al.: MapReduce: simplified data processing on large clusters. In: Sixth Symposium on Operating System Design and Implementation, San Francisco, pp. 1–13 (2004)
Ali, M.M., Fatti, L.P.: A differential free point generation scheme in the differential evolution algorithm. J. Global Optim. 35, 551–572 (2006). MapReduce. Morgan and Claypool Publishers, 2010
Chaobo, H., Yong, T., Zhenxiong, Y., Kai, Z., Guohua, C.: SRSH: a social recommender system based on Hadoop. Int. J. Multimedia Ubiquit. Eng. 9(6), 141–152 (2014)
Aluffi-Pentini, F., Parisi, V., Zirilli, F.: Global optimization and stochastic differential equations. J. Optim. Theor. Appl. 47(1), 1–16 (1985)
Wang, G.Z., Salles, M.V., Sowell, B., Wang, X., Cao, T., Gamers, A.: Behavioral simulations in MapReduce. In: PVLDB2010, Singapore, pp. 952–963 (2010)
Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, pp. 1942–1948 (1995)
Clerc, M., Kennedy, J.: The particle swarm explosion, stability, and convergence in a multidimensional complex space. IEEE Trans. Evol. Comput. 6(1), 58–73 (2002)
Rui, Z., Kalivarapu, V., Winer, E., Olive, J., Bhattacharya, S.: Particle swarm optimization-based source seeking. IEEE Trans. Autom. Sci. Eng. 12(3), 865–875 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Xu, J., Tang, Y. (2015). Research of Improved Particle Swarm Optimization Based on Genetic Algorithm for Hadoop Task Scheduling Problem. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9532. Springer, Cham. https://doi.org/10.1007/978-3-319-27161-3_76
Download citation
DOI: https://doi.org/10.1007/978-3-319-27161-3_76
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27160-6
Online ISBN: 978-3-319-27161-3
eBook Packages: Computer ScienceComputer Science (R0)