Skip to main content
SpringerLink
Log in

Resource Aware Adaptive Scheduler for Heterogeneous Workload with Task Based Job Sampling

  • Conference paper
  • First Online:
Innovations in Bio-Inspired Computing and Applications

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 424))

  • 942 Accesses

Abstract

Resource aware adaptive scheduling for Mapreduce jobs aims at improving resource utilization across machines. Mapreduce schedulers mainly have fixed number of execution slot on each tasktracker that represents the capacity of cluster. Here a method of dynamically adjusting the number of slots on tasktracker based on task completion gaol is implemented to maximize the resource utilization. A method of task based job sampling is used to get job profile information that inturn used to adjust the slots dynamically. Accuracy of our estimations where assessed based on completion time goal and actual execution time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Apache Hadoop. http://hadoop.apache.org

  2. Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: SOSP (2009)

    Google Scholar 

  3. The Hadoop Capacity Scheduler. http://hadoop.apache.org/common/docs/r0.19.2/capacitynscheduler.html

  4. Polo, J., Carrera, D., Becerra, Y., Torres, J., Ayguade, E., Steinder, M., Whalley, I.: Performance Management of Accelerated MapReduce Workloads in Heterogeneous Clusters

    Google Scholar 

  5. Polo, J., Castillo, C., Carrera, D., Becerra, Y., Whalley, I., Steinder, M., Torres, J., Ayguade, E.: Resource-aware adaptive scheduling for MapReduce clusters. In: Middleware, ser. Lecture Notes in Computer Science, vol. 7049, pp. 187–207. Springer (2011)

    Google Scholar 

  6. Wei, L.: Resource Aware Scheduling for Hadoop, unpublished

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Rajagiri School of Engineering and Technology, Cochin, Kerala, India

    Athira V. Panicker & G. Jisha

Authors
  1. Athira V. Panicker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Jisha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Jisha .

Editor information

Editors and Affiliations

  1. Dep. of Computer Science, VŠB – Technical Univ. of Ostrava, Ostrava, Czech Republic

    Václav Snášel

  2. (MIR Labs), Scientific Net Innov & Res Excel, Auburn, Washington, USA

    Ajit Abraham

  3. Faculty of Elec. Eng. & Comp. Sci., VÅ B - Technical University of Ostrava, Ostrava-Poruba, Czech Republic

    Pavel Krömer

  4. Department of Paper Technology, Indian Institute of Technology Roorkee, Roorkee, Uttarakhand, India

    Millie Pant

  5. Fac of Info & Comm, Comp Inte & Tech Lab, Universiti Teknikal Malaysia Melaka, Durian Tunggal, Malaysia

    Azah Kamilah Muda

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Panicker, A.V., Jisha, G. (2016). Resource Aware Adaptive Scheduler for Heterogeneous Workload with Task Based Job Sampling. In: Snášel, V., Abraham, A., Krömer, P., Pant, M., Muda, A. (eds) Innovations in Bio-Inspired Computing and Applications. Advances in Intelligent Systems and Computing, vol 424. Springer, Cham. https://doi.org/10.1007/978-3-319-28031-8_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28031-8_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28030-1

  • Online ISBN: 978-3-319-28031-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation