Skip to main content

Advertisement

SpringerLink
Log in

Resource and Cost Aware Glowworm Mapreduce Optimization Based Big Data Processing in Geo Distributed Data Center

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Handling large data in geographically distributed information centers with resource and cost optimization is a key challenge. With several approaches being designed, handling a large volume of data in multiple datacenters in an inappropriate manner yet is considered to be a time-consuming process. To address these issues, a Multivariate Metaphor based Metaheuristic Glowworm Swarm Map-Reduce Optimization (MM-MGSMO) technique is presented. Here, with search space and large data volume as input for geo-distributed datacenters, glowworm (i.e. virtual machine) population is initialized. With each glowworm possessing a certain amount of luciferin (i.e. objective function), multiple objective functions (i.e. bandwidth, storage capacity, energy and computation cost) are defined for each virtual machine. Next, the glowworm position is updated according to the neighboring factor by means of probability. Followed by this, MapReduce function identifies the optimal virtual machine and accordingly allocation is performed, therefore improving data allocation efficiency. Besides, the workload is assigned across datacenters, reduction in computation cost and storage capacity is guaranteed. Experimental evaluation of MM-MGSMO approach with existing methods attained improved performances with factors such as data allocation efficiency, false-positive rate, storage capacity compared with other cutting edge technologies such as Joint optimization algorithm and Game theory-based dynamic resource allocation strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Gu, L., Zeng, D., Li, P., & Guo, S. (2014). Cost minimization for big data processing in geo-distributed data centers. IEEE Transactions on Emerging Topics in Computing, 2(3), 314–323.

    Article  Google Scholar 

  2. Yuan, X., Min, G., Yang, L. T., Ding, Y., & Fang, Q. (2017). A game theory-based dynamic resource allocation strategy in geo-distributed datacenter clouds. Future Generation Computer Systems, 76, 63–72.

    Article  Google Scholar 

  3. Li, P., Guo, S., Miyazaki, T., Liao, X., Jin, H., Zomaya, A. Y., et al. (2017). Traffic-aware geo-distributed big data analytics with predictable job completion time. IEEE Transactions on Parallel and Distributed Systems, 28(6), 1785–1796.

    Article  Google Scholar 

  4. Xiao, W., Bao, W., Zhu, X., & Liu, L. (2017). Cost-aware big data processing across geo-distributed datacenters. IEEE Transactions on Parallel and Distributed Systems, 28(11), 3114–3127.

    Article  Google Scholar 

  5. Chalack, V. A., Razavi, S. N., & Gudakahriz, S. J. (2017). Resource allocation in cloud environment using approaches based particle swarm optimization. International Journal of Computer Applications Technology and Research., 6(2), 87–90.

    Article  Google Scholar 

  6. Manasrah, A. M., & Ba Ali, H. (2018). Workflow scheduling using hybrid GA-PSO algorithm in cloud computing. Wireless Communications and Mobile Computing, 2018, 1–16.

    Article  Google Scholar 

  7. Hashem, I. A. T., Anuar, N. B., Marjani, M., Gani, A., Sangaiah, A. K., & Sakariyah, A. K. (2018). Multi-objective scheduling of MapReduce jobs in big data processing. Multimedia Tools and Applications, 77(8), 9979–9994.

    Article  Google Scholar 

  8. Palmieri, F., Fiore, U., Ricciardi, S., & Castiglione, A. (2016). GRASP-based resource re-optimization for effective big data access in federated clouds. Future Generation Computer Systems, 54, 168–179.

    Article  Google Scholar 

  9. Rajesh, M., & Singaravel, G. (2014). I/O workload in virtualized data center using hypervisor. International Journal on Recent and Innovation Trends in Computing and Communication, 2(8), 2256–2260.

    Google Scholar 

  10. Chen, W., Paik, I., & Li, Z. (2017). Cost-aware streaming workflow allocation on geo-distributed data centers. IEEE Transactions on Computers, 66(2), 256–271.

    MathSciNet  MATH  Google Scholar 

  11. Zeng, X., Garg, S. K., Wen, Z., Strazdins, P., Zomaya, A. Y., & Ranjan, R. (2018). Cost efficient scheduling of MapReduce applications on public clouds. Journal of Computational Science, 26, 375–388.

    Article  Google Scholar 

  12. Sun, D., Yan, H., Gao, S., Liu, X., & Buyya, R. (2018). Rethinking elastic online scheduling of big data streaming applications over high-velocity continuous data streams. The Journal of Supercomputing, Springer, 74(2), 615–636.

    Article  Google Scholar 

  13. Simic, V., Stojanovic, B., & Ivanovic, M. (2019). Optimizing the performance of optimization in the cloudenvironment–an intelligent auto-scaling approach. Future Generation Computer Systems, 101, 909–920.

    Article  Google Scholar 

  14. Gawali, M. B., & Shinde, S. K. (2018). Task scheduling and resource allocation in cloud computing using a heuristic approach. Journal of Cloud Computing, 7(4), 1–16.

    Google Scholar 

  15. Sharkh, M. A., Shami, A., & Ouda, A. (2017). Optimal and suboptimal resource allocation techniques in cloud computing data centers. Journal of Cloud Computing, 6(6), 1–17.

    Google Scholar 

  16. Rawas, S., & Zekri, A. (2018). Location-aware energy-efficient workload allocation in geo distributed cloud environment. Journal of Computer Science, 14(3), 334–350.

    Article  Google Scholar 

  17. Ziafat, H., & Babamir, S. M. (2018). Optimal selection of VMs for resource task scheduling in geographically distributed clouds using fuzzy c-mean and MOLP. Journal of Software: Practice and Experience, 48(10), 1820–1846.

    Google Scholar 

  18. Ficco, M., Esposito, C., Palmieri, F., & Castiglion, A. (2018). A coral-reefs and game theory-based approach for optimizing elastic cloud resource allocation. Future Generation Computer Systems, 78, 343–352.

    Article  Google Scholar 

  19. Zheng, W., Qin, Y., Bugingo, E., Zhang, D., & Chen, J. (2018). Cost optimization for deadline-aware scheduling of big-data processing jobs on clouds. Future Generation Computer Systems, 82, 244–255.

    Article  Google Scholar 

  20. Zhang, N., Yang, X., Zhang, M., Sun, Y., & Lon, K. (2018). A genetic algorithm-based task scheduling for cloud resource crowd-funding model. International Journal of Communication Systems, 31(1), 1–10.

    Article  Google Scholar 

  21. Forestiero, A., Mastroianni, C., Meo, M., Papuzzo, G., & Sheikhalishahi, M. (2017). Hierarchical approach for efficient workload management in geo-distributed data centers. IEEE Transactions on Green Communications and Networking, 1(1), 97–111.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, AL-Amen Engineering College, Erode, Tamilnadu, 638 104, India

    S. Nithyanantham

  2. Department of Information Technology, K.S.R. College of Engineering (Autonomous), Tiruchengode, Namakkal District, Tamilnadu, 637 215, India

    G. Singaravel

Authors
  1. S. Nithyanantham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Singaravel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Nithyanantham.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nithyanantham, S., Singaravel, G. Resource and Cost Aware Glowworm Mapreduce Optimization Based Big Data Processing in Geo Distributed Data Center. Wireless Pers Commun 117, 2831–2852 (2021). https://doi.org/10.1007/s11277-020-07050-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-020-07050-6

Keywords

Search

Navigation