Advertisement

Resource and Cost Aware Glowworm Mapreduce Optimization Based Big Data Processing in Geo Distributed Data Center

  • 5 Accesses

Abstract

Handling large data in geographically distributed information centers with resource and cost optimization is a key challenge. With several approaches being designed, handling a large volume of data in multiple datacenters in an inappropriate manner yet is considered to be a time-consuming process. To address these issues, a Multivariate Metaphor based Metaheuristic Glowworm Swarm Map-Reduce Optimization (MM-MGSMO) technique is presented. Here, with search space and large data volume as input for geo-distributed datacenters, glowworm (i.e. virtual machine) population is initialized. With each glowworm possessing a certain amount of luciferin (i.e. objective function), multiple objective functions (i.e. bandwidth, storage capacity, energy and computation cost) are defined for each virtual machine. Next, the glowworm position is updated according to the neighboring factor by means of probability. Followed by this, MapReduce function identifies the optimal virtual machine and accordingly allocation is performed, therefore improving data allocation efficiency. Besides, the workload is assigned across datacenters, reduction in computation cost and storage capacity is guaranteed. Experimental evaluation of MM-MGSMO approach with existing methods attained improved performances with factors such as data allocation efficiency, false-positive rate, storage capacity compared with other cutting edge technologies such as Joint optimization algorithm and Game theory-based dynamic resource allocation strategy.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Gu, L., Zeng, D., Li, P., & Guo, S. (2014). Cost minimization for big data processing in geo-distributed data centers. IEEE Transactions on Emerging Topics in Computing,2(3), 314–323.

  2. 2.

    Yuan, X., Min, G., Yang, L. T., Ding, Y., & Fang, Q. (2017). A game theory-based dynamic resource allocation strategy in geo-distributed datacenter clouds. Future Generation Computer Systems,76, 63–72.

  3. 3.

    Li, P., Guo, S., Miyazaki, T., Liao, X., Jin, H., Zomaya, A. Y., et al. (2017). Traffic-aware geo-distributed big data analytics with predictable job completion time. IEEE Transactions on Parallel and Distributed Systems,28(6), 1785–1796.

  4. 4.

    Xiao, W., Bao, W., Zhu, X., & Liu, L. (2017). Cost-aware big data processing across geo-distributed datacenters. IEEE Transactions on Parallel and Distributed Systems,28(11), 3114–3127.

  5. 5.

    Chalack, V. A., Razavi, S. N., & Gudakahriz, S. J. (2017). Resource allocation in cloud environment using approaches based particle swarm optimization. International Journal of Computer Applications Technology and Research.,6(2), 87–90.

  6. 6.

    Manasrah, A. M., & Ba Ali, H. (2018). Workflow scheduling using hybrid GA-PSO algorithm in cloud computing. Wireless Communications and Mobile Computing,2018, 1–16.

  7. 7.

    Hashem, I. A. T., Anuar, N. B., Marjani, M., Gani, A., Sangaiah, A. K., & Sakariyah, A. K. (2018). Multi-objective scheduling of MapReduce jobs in big data processing. Multimedia Tools and Applications,77(8), 9979–9994.

  8. 8.

    Palmieri, F., Fiore, U., Ricciardi, S., & Castiglione, A. (2016). GRASP-based resource re-optimization for effective big data access in federated clouds. Future Generation Computer Systems,54, 168–179.

  9. 9.

    Rajesh, M., & Singaravel, G. (2014). I/O workload in virtualized data center using hypervisor. International Journal on Recent and Innovation Trends in Computing and Communication,2(8), 2256–2260.

  10. 10.

    Chen, W., Paik, I., & Li, Z. (2017). Cost-aware streaming workflow allocation on geo-distributed data centers. IEEE Transactions on Computers,66(2), 256–271.

  11. 11.

    Zeng, X., Garg, S. K., Wen, Z., Strazdins, P., Zomaya, A. Y., & Ranjan, R. (2018). Cost efficient scheduling of MapReduce applications on public clouds. Journal of Computational Science,26, 375–388.

  12. 12.

    Sun, D., Yan, H., Gao, S., Liu, X., & Buyya, R. (2018). Rethinking elastic online scheduling of big data streaming applications over high-velocity continuous data streams. The Journal of Supercomputing, Springer,74(2), 615–636.

  13. 13.

    Simic, V., Stojanovic, B., & Ivanovic, M. (2019). Optimizing the performance of optimization in the cloudenvironment–an intelligent auto-scaling approach. Future Generation Computer Systems,101, 909–920.

  14. 14.

    Gawali, M. B., & Shinde, S. K. (2018). Task scheduling and resource allocation in cloud computing using a heuristic approach. Journal of Cloud Computing,7(4), 1–16.

  15. 15.

    Sharkh, M. A., Shami, A., & Ouda, A. (2017). Optimal and suboptimal resource allocation techniques in cloud computing data centers. Journal of Cloud Computing,6(6), 1–17.

  16. 16.

    Rawas, S., & Zekri, A. (2018). Location-aware energy-efficient workload allocation in geo distributed cloud environment. Journal of Computer Science,14(3), 334–350.

  17. 17.

    Ziafat, H., & Babamir, S. M. (2018). Optimal selection of VMs for resource task scheduling in geographically distributed clouds using fuzzy c-mean and MOLP. Journal of Software: Practice and Experience,48(10), 1820–1846.

  18. 18.

    Ficco, M., Esposito, C., Palmieri, F., & Castiglion, A. (2018). A coral-reefs and game theory-based approach for optimizing elastic cloud resource allocation. Future Generation Computer Systems,78, 343–352.

  19. 19.

    Zheng, W., Qin, Y., Bugingo, E., Zhang, D., & Chen, J. (2018). Cost optimization for deadline-aware scheduling of big-data processing jobs on clouds. Future Generation Computer Systems,82, 244–255.

  20. 20.

    Zhang, N., Yang, X., Zhang, M., Sun, Y., & Lon, K. (2018). A genetic algorithm-based task scheduling for cloud resource crowd-funding model. International Journal of Communication Systems,31(1), 1–10.

  21. 21.

    Forestiero, A., Mastroianni, C., Meo, M., Papuzzo, G., & Sheikhalishahi, M. (2017). Hierarchical approach for efficient workload management in geo-distributed data centers. IEEE Transactions on Green Communications and Networking,1(1), 97–111.

Download references

Author information

Correspondence to S. Nithyanantham.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nithyanantham, S., Singaravel, G. Resource and Cost Aware Glowworm Mapreduce Optimization Based Big Data Processing in Geo Distributed Data Center. Wireless Pers Commun (2020) doi:10.1007/s11277-020-07050-6

Download citation

Keywords

  • Big data processing
  • Geo-distributed data center
  • Glowworm Swarm optimization
  • MapReduce function