Skip to main content
SpringerLink
Log in

Passive Network Awareness as a Means for Improved Grid Scheduling

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Grids enable sharing resources of heterogeneous nature and administration. In such distributed systems, the network is usually taken for granted which is potentially problematic due to the complexity and unpredictability of public networks that typically underlie grids. This article introduces GridMAP, a mechanism for considering the network state for enhancing grid scheduling. Network measurements are collected in a passive manner from a user-centric vantage point. This mechanism has been evaluated on a production e-science grid infrastructure, with results showing the ability of GridMAP to improve grid scheduling with minimal network, computational and deployment overheads.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Akella, A., Seshan, S., Shaikh, A.: An empirical valuation of wide-area internet bottlenecks. In: Proceedings of ACM SIGCOMM IMC, pp. 101–114 (2003)

  2. Cisco Visual Networking Index: Global mobile data traffic forecast update, 20122017. Tech. rep., Cisco Press (2013)

  3. Lee, C., Lee, D.K., Yi, Y., Moon, S.: Operating a Network Link at 100 %. In: Proc. PAM, pp. 1–10 (2011)

  4. Foster, I., Kesselman, C.: Globus: a metacomputing infrastructure toolkit. The International Journal of Supercomputer Applications and High Performance Computing 11(2), 115–128 (1997)

    Article  Google Scholar 

  5. Berman, F.: The grid: blueprint for a new computing infrastructure, chap. High-Performance Schedulers, pp. 279–309. Morgan Kaufmann (1998)

  6. Batista, D.M., da Fonseca, N.L.S.: A survey of self-adaptive grids. IEEE Commun. Mag. 48(7), 94–100 (2010)

    Article  Google Scholar 

  7. Qureshi, M.B., Dehnavi, M.M., Min-Allah, N., Qureshi, M.S., Hussain, H., Rentifis, I., Tziritas, N., Loukopoulos, T., Khan, S.U., Xu, C.Z., et al.: Survey on grid resource allocation mechanisms. J. Grid. Comput., 1–43 (2014)

  8. Agarwal, A., Kumar, P.: Multidimensional QoS Oriented Task Scheduling in Grid Environments. International Journal of Grid Computing & Applications (IJGCA) 2(1), 28–37 (2011)

    Article  MathSciNet  Google Scholar 

  9. Tomás, L., Caminero, A.C., Carrión, C., Caminero, B.: Network-aware meta-scheduling in advance with autonomous self-tuning system. Future Gener. Comp. Sys. 27(5), 486–497 (2011)

    Article  Google Scholar 

  10. Chinnaiah, V., Somasundaram, T.S.: A grid resource brokering strategy based on resource and network performance in grid. Future Gener. Comp. Sys. 28(3), 491–499 (2012)

    Article  Google Scholar 

  11. McClatchey, R., Anjum, A., Stockinger, H., Ali, A., Willers, I., Thomas, M.: Data intensive and network aware (DIANA) grid scheduling. J Grid Comput 5(1), 43–64 (2007)

    Article  Google Scholar 

  12. Pezaros, D., Sifalakis, M., Schmid, S., Hutchison, D.: Dynamic link measurements using active components. In: Proceedings of IFIP conference on active networks (2004)

  13. He, Q., Dovrolis, C., Ammar, M.: On the predictability of large transfer TCP throughput. Comput. Netw. 51(14), 3959–3977 (2007)

    Article  MATH  Google Scholar 

  14. Lowekamp, B.B.: Combining active and passive network measurements to build scalable monitoring systems on the grid. ACM SIGMETRICS Performance Evaluation Review 30(4), 19–26 (2003)

    Article  Google Scholar 

  15. Wolski, R.: Dynamically forecasting network performance using the network weather service. Clust. Comput. 1(1), 119–132 (1998)

    Article  Google Scholar 

  16. Vazhkudai, S., Schopf, J.M., Foster, I.T.: Predicting the performance of wide area data transfers. In: Proceedings of IPDPS, p. 270 (2002)

  17. Yousaf, M.M., Welzl, M., Junaid, M.M.: Fog in the network weather service: a case for novel approaches. In: Proceedings of conference on networks for grid applied, pp. 1–6 (2007)

  18. Goga, O., Teixeira, R.: Speed measurements of residential internet access. In: Proceedings PAM, pp. 168–178 (2012)

  19. Huang, A.C., Steenkiste, P.: Network-sensitive service discovery. J. Grid. Comput. 1(3), 309–326 (2003)

    Article  Google Scholar 

  20. Madhyastha, H.V., Isdal, T., Piatek, M., Dixon, C., Anderson, T., Krishnamurthy, A., Venkataramani, A.: iPlane: an information plane for distributed services. In: Proceedings of USENIX OSDI, pp. 367–380 (2006)

  21. Hullár, B., Laki, S., Stéger, J., Csabai, I., Vattay, G.: SONoMA: a service oriented network measurement architecture. Tech. Rep. TR-ELTE-CNL-2010/1. Eötvös Loránd University, Hungary (2010)

    Google Scholar 

  22. Wenwei, L., Dafang, Z., Jinmin, Y., Gaogang, X.: On evaluating the differences of TCP and ICMP in network measurement. Comput. Commun. 30(2), 428–439 (2007)

    Article  Google Scholar 

  23. Paxson, V.E.: Measurements and analysis of end-to-end internet dynamics. Ph.D. thesis, University of California at Berkeley (1998)

  24. Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: WOW: self-organizing wide area overlay networks of virtual workstations. In: Proceedings of IEEE HPDC, pp. 30–42 (2006)

  25. Raicu, I., Dumitrescu, C., Ripeanu, M., Foster, I.: The design, performance, and use of diperf: An automated distributed performance evaluation framework. J. Grid. Comput. 4(3), 287–309 (2006)

    Article  MATH  Google Scholar 

  26. Zhang, X., Freschl, J.L., Schopf, J.M.: Scalability analysis of three monitoring and information systems: MDS2, R-GMA, and Hawkeye. J Parallel Distrib. Comput. 67(8), 883–902 (2007)

    Article  MATH  Google Scholar 

  27. Fitzgerald, S., Foster, I., Kesselman, C., von Laszewski, G., Smith, W., Tuecke, S.: A directory service for configuring high-performance distributed computations. In: Proceedings of IEEE HPDC, pp. 365–375 (1997)

  28. Cooke, A.W., Gray, A.J., Nutt, W., Magowan, J., Oevers, M., Taylor, P., Cordenonsi, R., Byrom, R., Cornwall, L., Djaoui, A., et al.: The relational grid monitoring architecture: mediating information about the grid. J. Grid. Comput. 2(4), 323–339 (2004)

    Article  Google Scholar 

  29. Tierney, B., Aydt, R., Gunter, D., Smith, W., Swany, M., Taylor, V., Wolski, R.: A grid monitoring architecture. Tech. rep., Global Grid Forum (2002)

  30. Zanikolas, S., Sakellariou, R.: A taxonomy of grid monitoring systems. Future Gener. Comp. Sys. 21(1), 163–188 (2005). doi:10.1016/j.future.2004.07.002

    Article  Google Scholar 

  31. Lee, S.: eTOP: End-to-end Performance Measurement Infrastructure on KOREN and APAN Links., http://master.apan.net/meetings/busan03/measurement/koren.ppt (2003)

  32. Sacerdoti, F.D., Katz, M.J., Massie, M.L., Culler, D.E.: Wide area cluster monitoring with ganglia. In: Proceedings of IEEE CLUSTER, pp. 289–298 (2003)

  33. Al-Ali, R.J., Amin, K., Von Laszewski, G., Rana, O.F., Walker, D.W., Hategan, M., Zaluzec, N.: Analysis and provision of qos for distributed grid applications. J. Grid. Comput. 2(2), 163–182 (2004)

    Article  Google Scholar 

  34. Elkhatib, Y.: Monitoring, analysing and predicting network performance in grids. Ph.D. thesis, Lancaster University (2011)

  35. Elkhatib, Y., Edwards, C.: A survey-based study of grid traffic. In: Proceedings of conference on networks for grid applications (2007)

  36. Savage, S.: Sting: A TCP-based network measurement tool. In: Proceedings of USENIX symposium on internet technologies and systems, vol. 2 (1999)

  37. Horneffer, M.: Assessing Internet Performance Metrics Using Large-Scale TCP-SYN based Measurements. In: Proceedings of PAM (2000)

  38. Jiang, H., Dovrolis, C.: Passive estimation of TCP round-trip times. ACM SIGCOMM Comput. Commun. Rev. 32(3), 75–88 (2002)

    Article  Google Scholar 

  39. Elkhatib, Y., Edwards, C., Mackay, M., Tyson, G.: Providing grid schedulers with passive network measurements. In: Proceedings of ICCCN (2009)

  40. Network Measurement Working Group. http://nmwg.internet2.edu

  41. Akima, H.: A new method of interpolation and smooth curve fitting based on local procedures. J. ACM 17(4), 589–602 (1970)

    Article  MATH  Google Scholar 

  42. White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., Joglekar, A.: An integrated experimental environment for distributed systems and networks. In: Proceedings of USENIX OSDI, pp. 255–270 (2002)

  43. Marco, C., Fabio, C., Alvise, D., Antonia, G., Francesco, G., Alessandro, M., Moreno, M., Salvatore, M., Fabrizio, P., Luca, P., et al.: The gLite workload management system. In: Advances in grid and pervasive computing, pp. 256–268. Springer (2009)

  44. AutoDock, the scripps research institute. http://autodock.scripps.edu/

  45. Case, D.A., Cheatham, T.E., Darden, T., Gohlke, H., Luo, R., Merz, K.M., Onufriev, A., Simmerling, C., Wang, B., Woods, R.J.: The amber biomolecular simulation programs. J. Comput. Chem. 26(16), 1668–1688 (2005)

    Article  Google Scholar 

  46. Jacq, N., Salzemann, J., Jacq, F., Legré, Y., Medernach, E., Montagnat, J., Maaß, A., Reichstadt, M., Schwichtenberg, H., Sridhar, M., Kasam, V., Zimmermann, M., Hofmann, M., Breton, V.: Grid-enabled virtual screening against malaria. J Grid Comp 6(1), 29–43 (2008)

    Article  Google Scholar 

  47. Kasam, V., Salzemann, J., Jacq, N., Maaß, A., Breton, V.: Large scale deployment of molecular docking application on computational grid infrastructures for combating malaria. In: Proceedings CCGrid, pp. 691–700 (2007)

Download references

Author information

Authors and Affiliations

  1. The School of Computing, Communications, Lancaster University, Lancaster, LA1 4WA, UK

    Yehia Elkhatib & Chris Edwards

Authors
  1. Yehia Elkhatib
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chris Edwards
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yehia Elkhatib.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elkhatib, Y., Edwards, C. Passive Network Awareness as a Means for Improved Grid Scheduling. J Grid Computing 13, 275–291 (2015). https://doi.org/10.1007/s10723-015-9332-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-015-9332-0

Keywords

Search

Navigation