Skip to main content
SpringerLink
Log in

Setting up Clusters of Computing Units to Process Several Data Streams Efficiently

  • Conference paper
  • First Online:
Parallel Processing and Applied Mathematics (PPAM 2013)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8385))

  • 1329 Accesses

Abstract

Let us consider an upper bounded number of data streams to be processed by a Divisible Load application. The total workload is unknown and the available speeds for communicating and computing may be poorly a priori estimated. This paper presents a resource selection method that aims at maximizing the throughput of this processing. From a set of processing units linked by a network, this method consists in forming an optimal set of master-workers clusters. Results of simulations are presented to assess the efficiency of this method experimentally. Before focusing on the proposed resource selection method, the paper comes back on the adaptive scheduling method on which it relies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Lee, C., Hamdi, M.: Parallel image processing application in a network of workstation. Parallel Comput. 21, 137–160 (1995)

    Article  MATH  Google Scholar 

  2. Altılar, D.T., Paker, Y.: An optimal scheduling algorithm for stream based parallel video processing. In: Yazıcı, A., Şener, C. (eds.) ISCIS 2003. LNCS, vol. 2869, pp. 731–738. Springer, Heidelberg (2003)

    Google Scholar 

  3. Robertazzi, T.G.: Ten reasons to use divisible load theory. IEEE Comput. 36(5), 63–68 (2003)

    Article  Google Scholar 

  4. Altilar, D., Paker, Y.: An optimal scheduling algorithm for parallel video processing. In: Proceedings of the International Conference on Multimedia Computing and Systems. IEEE Computing Society Press (1998)

    Google Scholar 

  5. Dong, L., Bharadwaj, V., Ko, C.C.: Efficient movie retrieval strategies for movie-on-demand multimedia services on distributed networks. Multimedia Tools Appl. 20(2), 99–133 (2003)

    Article  Google Scholar 

  6. Beaumont, O., Casanova, H., Legrand, A., Robert, Y., Yang, Y.: Scheduling divisible loads on star and tree networks: results and open problems. IEEE Trans. Parallel Distrib. Syst. 16(3), 207–218 (2005)

    Article  Google Scholar 

  7. Drozdowski, M., Wolniewicz, P.: Optimizing divisible load scheduling on heterogeneous stars with limited memory. Eur. J. Oper. Res. 172(2), 545–559 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  8. Rosenberg, A.L., Chiang, R.C.: Toward understanding heterogeneity in computing. In: Proceeding of the 24th International Parallel and Distributed Processing Symposium (IPDPS’10), vol. 1, pp. 1–10. IEEE Computing Society Press, April 2010

    Google Scholar 

  9. Beaumont, O., Marchal, L., Robert, Y.: Scheduling divisible loads with return messages on heterogeneous master-worker platforms. In: Bader, D.A., Parashar, M., Sridhar, V., Prasanna, V.K. (eds.) HiPC 2005. LNCS, vol. 3769, pp. 498–507. Springer, Heidelberg (2005)

    Google Scholar 

  10. Saif, T., Parashar, M.: Understanding the behavior and performance of non-blocking communications in MPI. In: Danelutto, M., Vanneschi, M., Laforenza, D. (eds.) Euro-Par 2004. LNCS, vol. 3149, pp. 173–182. Springer, Heidelberg (2004)

    Google Scholar 

  11. Bharadwaj, V., Ghose, D., Mani, V., Robertazzi, T.: Scheduling divisible loads in parallel and distributed systems. IEEE Computing Society Press, Los Almitos (1996)

    Google Scholar 

  12. Drozdowski, M.: Selected problems of scheduling tasks in multiprocessor computing systems. Ph.D. thesis, Instytut Informatyki Politechnika Poznanska, Poznan (1997)

    Google Scholar 

  13. Bharadwaj, V., Ghose, D., Mani, V.: Multi-installment load distribution in tree networks with delays. IEEE Trans. Aerosp. Electron. Syst. 31(2), 555–567 (1995)

    Article  Google Scholar 

  14. Yang, Y., Casanova, H.: Extensions to the multi-installment algorithm: affine costs and output data transfers. Technical Report CS2003-0754, Dept. of Computer Science and Engineering, University of California, San Diego (2003)

    Google Scholar 

  15. Millot, D., Parrot, C.: Scheduling on unspecified heterogeneous distributed resources. In: Proceedings of the 25th International Symposium on Parallel and Distributed Processing Workshops (IPDPSW’11), vol. 1, pp. 45–56. IEEE Computing Society Press, May 2011

    Google Scholar 

  16. Millot, D., Parrot, C.: Fundamental results on the AS4DR scheduler. Technical Report RR-11005-INF, TELECOM sudParis, Évry, France (2011)

    Google Scholar 

  17. Millot, D., Parrot, C.: Some tests of adaptivity for the AS4DR scheduler. In: Proceedings of the 41th International Conference on Parallel Processing (ICPP’12), pp. 323–331. IEEE Computing Society Press, September 2012

    Google Scholar 

  18. Pisinger, D.: An exact algorithm for large multiple knapsack problems. Eur. J. Oper. Res. 114(3), 528–541 (1999)

    Article  MATH  Google Scholar 

  19. Casanova, H., Legrand, A., Quinson, M.: SimGrid: a generic framework for large-scale distributed experiments. In: Proceedings of the 10th International Conference on Computer Modeling and Simulation (ICCMS’10), pp. 126–131. IEEE Computing Society Press, March 2008

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Telecom SudParis, Institut Mines-Telecom, Évry, France

    Daniel Millot & Christian Parrot

Authors
  1. Daniel Millot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Parrot
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Parrot .

Editor information

Editors and Affiliations

  1. Institute of Computer and Information Science, Czestochowa University of Technology, Czestochowa, Poland

    Roman Wyrzykowski

  2. University of Tennessee, Department of Computer Science, Knoxville, Tennessee, USA

    Jack Dongarra

  3. Institute of Computer and Information Science, Czestochowa University of Technology, Czestochowa, Poland

    Konrad Karczewski

  4. Technical University of Denmark Informatics and Mathematical Modelling, Kongens Lyngby, Denmark

    Jerzy Waśniewski

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Millot, D., Parrot, C. (2014). Setting up Clusters of Computing Units to Process Several Data Streams Efficiently. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds) Parallel Processing and Applied Mathematics. PPAM 2013. Lecture Notes in Computer Science(), vol 8385. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55195-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-55195-6_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-55194-9

  • Online ISBN: 978-3-642-55195-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation