Skip to main content
SpringerLink
Log in

A Load Distribution Based Resource Allocation Strategy for Bag of Tasks (BoT) in Computational Grid Environment

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

In the ever-evolving landscape of computational grid systems, the meticulous selection of resources tailored to specific tasks is a formidable challenge. This paper introduces an efficient load distribution strategy known as Load Distribution Based Resource Allocation (LDRA), one of the foremost goals is to allocate resources to gain enhanced resource utilization and also try to achieve least possible execution time to fulfil the need for grid systems. A comprehensive performance evaluation unfolds to elevate grid efficiency, pitting LDRA against existing heuristics using the ETC Simulation Benchmark. The study expands further on the real-world dataset from the Gaia Cluster Configurations (https://hpc.uni.lu/systems/gaia/) to verify its significance in the real environment. The LDRA algorithm emerges with superior performance when compared to state-of-the-art such as Max–Min, Opportunistic Load Balancing, AlgHybrid_LB, and Resource Aware Load Balancing for resource utilization, makespan, flowtime, and energy efficiency in the majority of the cases in experimental evaluation. In some cases, the experimental results show that LDRA’s usage of the grid resources is remarkable, reaching over 99% in four cases and approaching 98% in two cases of the ETC simulation benchmark. These accomplishments are further mirrored in the evaluation against real datasets, where LDRA’s performance among peers is nothing short of exemplary in the cases under study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availability

We selected the Gaia cluster log of the University of Luxemburg by considering the criteria mentioned in the previous section. The details of both the selected cluster logs are given at https://hpc.uni.lu/systems/gaia/.

The reason behind the selection of each dataset is also stated. The details of The University of Luxemburg Gaia Cluster log are as follows:

• System: University of Luxemburg Gaia Cluster

• Duration: 22nd May 2014 to 19th August 2014

• Jobs: 51,987

This log contains three months’ worth of data from the Gaia cluster at the University of Luxemburg. It is used mainly by biologists working with massive data problems and engineering people working with physical simulations. The workload data includes CPU and memory usage; further, I/O activity is in a separate file as I/O is not accommodated by the standard workload format. The workload log from the Gaia cluster system was graciously provided by Joseph Emeras at https://www.cs.huji.ac.il/labs/parallel/workload/lunilugaia/ (accessed on 10/30/2020).

The Gaia cluster is one of the four clusters operated by the ULHPC (University of Luxembourg HPC Center). Initially released in 2011, Gaia is now a heterogeneous cluster that has been upgraded several times. It currently features 151 nodes, manufactured by Bull and Dell, with a total of 2004 cores. Several nodes (20) feature NVidia Tesla-class GPGPUs accelerators.

Code Availability

This work is part of a Ph.D. thesis of the first author, Sophiya Sheikh. Now we are working on its extended version. Therefore, we can’t make this code public. However, it can be made available on request.

References

  1. title = https://hpc.uni.lu/systems/gaia/, url = https://hpc.uni.lu/systems/gaia/

  2. Foster, I., Kesselman, C., & Tuecke, S. (2001). The anatomy of the grid: Enabling scalable virtual organizations. The International Journal of High-Performance Computing Applications, 15(3), 200–222.

    Article  Google Scholar 

  3. Shah, R., Veeravalli, B., & Misra, M. (2007). On the design of adaptive and decentralized load balancing algorithms with load estimation for computational grid environments. IEEE Transactions on Parallel and Distributed Systems, 18(12), 1675–1686.

    Article  Google Scholar 

  4. Maheswaran, M., Ali, S., Siegel, H. J., Hensgen, D., & Freund, R. F. (1999). Dynamic mapping of a class of independent tasks onto heterogeneous computing systems. Journal of Parallel and Distributed Computing, 59(2), 107–131.

    Article  Google Scholar 

  5. Armstrong Jr, R. K. (1997). Investigation of effect of different run-time distributions on SmartNet performance. Naval Postgraduate School Monterey CA.

  6. Patni, J. C., Aswal, M. S., Pal, O. P., & Gupta, A. (2011, April). Load balancing strategies for grid computing. In 2011 3rd international conference on electronics computer technology (Vol. 3, pp. 239–243). IEEE.

  7. Qureshi, M. B., Dehnavi, M. M., Min-Allah, N., Qureshi, M. S., Hussain, H., Rentifis, I., Tziritas, N., Loukopoulos, T., Khan, S. U., Xu, C. Z., & Zomaya, A. Y. (2014). Survey on grid resource allocation mechanisms. Journal of Grid Computing., 12(2), 399–441.

    Article  Google Scholar 

  8. Reda, N. M., Tawfik, A., Marzok, M. A., & Khamis, S. M. (2015). Sort-mid tasks scheduling algorithm in grid computing. Journal of Advanced Research, 6(6), 987–993.

    Article  Google Scholar 

  9. Balasangameshwara, J., & Raju, N. (2012). A hybrid policy for fault-tolerant load balancing in grid computing environments. Journal of Network and Computer Applications, 35(1), 412–422.

    Article  Google Scholar 

  10. Braun, T. D., Siegel, H. J., Beck, N., Bölöni, L. L., Maheswaran, M., Reuther, A. I., Robertson, J. P., Theys, M. D., Yao, B., Hensgen, D., & Freund, R. F. (2001). A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. Journal of Parallel and Distributed Computing, 61(6), 810–837.

    Article  Google Scholar 

  11. Salimi, R., Motameni, H., & Omranpour, H. (2012, December). Task scheduling with Load balancing for computational grid using NSGA II with fuzzy mutation. In 2012 2nd IEEE international conference on parallel, distributed and grid computing (pp. 79–84). IEEE.

  12. Alam, M., Haidri, R. A., & Shahid, M. (2020). Resource-aware load balancing model for batch of tasks (BoT) with best fit migration policy on heterogeneous distributed computing systems. International Journal of Pervasive Computing and Communications.

  13. Cao, J., Spooner, D. P., Jarvis, S. A., & Nudd, G. R. (2005). Grid load balancing using intelligent agents. Future Generation Computer Systems, 21(1), 135–149.

    Article  Google Scholar 

  14. Feng, Y., Li, D., Wu, H., & Zhang, Y. (2000, May). A dynamic load balancing algorithm based on distributed database system. In Proceedings fourth international conference/exhibition on high performance computing in the Asia-Pacific Region (Vol. 2, pp. 949–952). IEEE.

  15. Patel, D. K., Tripathy, D., & Tripathy, C. R. (2016). Survey of load balancing techniques for grid. Journal of Network and Computer Applications, 65, 103–119.

    Article  Google Scholar 

  16. Wu, T., Ye, N., & Zhang, D. (2005). Comparison of distributed methods for resource allocation. International Journal of Production Research., 43(3), 515–536.

    Article  Google Scholar 

  17. Sheikh, S., Nagaraju, A., & Shahid, M. (2021). A fault-tolerant hybrid resource allocation model for dynamic computational grid. Journal of Computational Science, 48, 101268.

    Article  Google Scholar 

  18. Sheikh, S., Nagaraju, A., & Shahid, M. (2019). A Parallelized Dynamic Task Scheduling for Batch of Task in a computational grid. International Journal of Computers and Applications, 41(1), 39–53.

    Article  Google Scholar 

  19. Sheikh, S., & Nagaraju, A. (2020). Dynamic task scheduling with advance reservation of resources to minimize turnaround time for computational grid. International Journal of Information Technology, 1–9.

  20. Xiao, P., & Hu, Z. (2008). A novel QoS-based co-allocation model in computational grid. In IEEE GLOBECOM 2008–2008 IEEE Global Telecommunications Conference (pp. 1–5). IEEE.

  21. Cheng, C.-T., & Li, Z.-J. (2006). Parallel algorithm for grid resource allocation based on nash equilibrium. Paper presented at the Machine Learning and Cybernetics, 2006 International Conference on. IEEE. pp. 4383–4388.

  22. Zhi-jie, L., & Cun-rui, W. (2012). Resource allocation optimization based on load forecast in computational grid. International Journal of Engineering Research and Applications (IJERA), 2(3), 1353–1358.

    Google Scholar 

  23. Sajid, M., Raza, Z., & Shahid, M. (2016). Energy-efficient scheduling algorithms for batch-of-tasks (BoT) applications on heterogeneous computing systems. Concurrency and Computation: Practice and Experience, 28(9), 2644–2669.

    Article  Google Scholar 

  24. Saravanakumar, E., & Prathima, G. (2010). A novel load balancing algorithm for computational grid. Paper presented at the Innovative Computing Technologies (ICICT), 2010 International Conference on. IEEE. pp. 1–6.

  25. Wang, L., Wang, T. G., & Luo, Y. (2011). Improved non-dominated sorting genetic algorithm (NSGA)- II in multi-objective optimization studies of wind turbine blades. Applied Mathematics and Mechanics, 32(6), 739–748.

    Article  Google Scholar 

  26. Hao, Y., Liu, G., & Wen, N. (2012). An enhanced load balancing mechanism based on deadline control on GridSim. Future Generation Computer Systems, 28(4), 657–665.

    Article  Google Scholar 

  27. Patel, D. K., Tripathy, D., & Tripathy, C. (2016). An improved load-balancing mechanism based on deadline failure recovery on GridSim. Engineering with Computers, 32(2), 173–188.

  28. Salimi, R., Motameni, H., & Omranpour, H. (2014). Task scheduling using NSGA II with fuzzy adaptive operators for computational grids. Journal of Parallel and Distributed Computing, 74(5), 2333–2350.

    Article  Google Scholar 

  29. Braun, T. D., Siegel, H. J., Beck, N., Boloni, L. L., Maheswaran, M., Reuther, A. I., Robertson, J. P., Theys, M. D., Yao, B., Hensgen, D., et al. (2001). Task scheduling with Load balancing for computational grid using NSGA II with fuzzy mutation. Journal of Parallel and Distributed Computing, 61(6), 810–837.

    Article  Google Scholar 

  30. Sathish, K., & Reddy, A. R. M. (2008). Enhanced ant algorithm-based load balanced task scheduling in grid computing. IJCSNS, 8(10), 219.

    Google Scholar 

  31. Sajid, M., Raza, Z., & Shahid, M. (2018). Hybrid bio-inspired scheduling algorithms for batch of tasks on heterogeneous computing system. International Journal of Bio-Inspired Computation, 11(3), 135–148.

    Article  Google Scholar 

  32. Xhafa, F., Barolli, L., & Durresi, A. (2007). Immediate mode scheduling of independent jobs in computational grids. In 21st international conference on advanced information networking and applications (AINA 07) (pp. 970–977). IEEE.

  33. Sheikh, S., Nagaraju, A., & Shahid, M. (2018). Dynamic load balancing with advanced reservation of resources for computational grid. Progress in computing, analytics and networking (pp. 501–510). Springer.

    Chapter  Google Scholar 

  34. title = https://www.cs.huji.ac.il/labs/parallel/workload/l unilu gaia/ (Accessed on 10/30/2020), url = https://www.cs.huji.ac.il/labs/parallel/workload/l unilu gaia/

  35. title = https://www.cs.huji.ac.il/labs/parallel/workload/logs.html (Accessed on 10/30/2020), url = https://www.cs.huji.ac.il/labs/parallel/workload/logs.html

  36. Chapin, S. J., Cirne, W., Feitelson, D. G., Jones, J. P., Leutenegger, S. T., Schwiegelshohn, U., Smith, W., & Talby, D. (1999, April). Benchmarks and standards for the evaluation of parallel job schedulers. In Workshop on job scheduling strategies for parallel processing (pp. 67–90). Springer.

Download references

Funding

No funding is supported by any agency.

Author information

Authors and Affiliations

  1. School of Computer Application, Lovely Professional University, Phagwara, Punjab, 144411, India

    Sophiya Sheikh

  2. Department of Commerce, Aligarh Muslim University, Aligarh, Uttar Pradesh, 202001, India

    Mohammad Shahid

  3. Department of Data Science, FLAME University, Pune, Maharastra, 412115, India

    Manas Sambare

  4. Department of Computer Science and Information Technology, Khwaja Moinuddin Chishti Language University, Lucknow, Uttar Pradesh, India

    Raza Abbas Haidri

  5. Department of Electronics and Communication, University of Allahabad, Allahabad, Uttar Pradesh, India

    Shiv Prakash

Authors
  1. Sophiya Sheikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Shahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manas Sambare
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raza Abbas Haidri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shiv Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Main Idea, Methodology, [SS and MS]; Implementation, data acquisition, and Data Analysis [SS]; All authors participated in Writing- Original draft preparation, data analysis, and writing review & editing. All the authors reviewed the manuscript.

Corresponding author

Correspondence to Mohammad Shahid.

Ethics declarations

Competing Interests/Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. There is no conflict of interest between the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sheikh, S., Shahid, M., Sambare, M. et al. A Load Distribution Based Resource Allocation Strategy for Bag of Tasks (BoT) in Computational Grid Environment. Wireless Pers Commun 135, 47–80 (2024). https://doi.org/10.1007/s11277-024-10951-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-024-10951-5

Keywords

Search

Navigation