Queueing Systems

, Volume 83, Issue 3–4, pp 261–291 | Cite as

Stochastic bounds in Fork–Join queueing systems under full and partial mapping

  • Amr RizkEmail author
  • Felix Poloczek
  • Florin Ciucu


In a Fork–Join (FJ) queueing system, an upstream fork station splits incoming jobs into N tasks to be further processed by N parallel servers, each with its own queue; the response time of one job is determined, at a downstream join station, by the maximum of the corresponding tasks’ response times. This queueing system is useful to the modeling of multi-service systems subject to synchronization constraints, such as MapReduce clusters or multipath routing. Despite their apparent simplicity, FJ systems are hard to analyze. This paper provides the first computable stochastic bounds on the waiting and response time distributions in FJ systems under full (bijective) and partial (injective) mapping of tasks to servers. We consider four practical scenarios by combining (1a) renewal and (1b) non-renewal arrivals, and (2a) non-blocking and (2b) blocking servers. In the case of non-blocking servers, we prove that delays scale as \(\mathcal {O}(\log N)\), a law which is known for first moments under renewal input only. In the case of blocking servers, we prove that the same factor of \(\log N\) dictates the stability region of the system. Simulation results indicate that our bounds are tight, especially at high utilizations, in all four scenarios. A remarkable insight gained from our results is that, at moderate to high utilizations, multipath routing “makes sense” from a queueing perspective for two paths only, i.e., response times drop the most when \(N=2\); the technical explanation is that the resequencing (delay) price starts to quickly dominate the tempting gain due to multipath transmissions.


Fork–Join queue Performance evaluation Parallel systems MapReduce Multipath 

Mathematics Subject Classification

90B22 68M20 60K25 90B18 


  1. 1.
    Abate, J., Choudhury, G.L., Whitt, W.: Exponential approximations for tail probabilities in queues, I: waiting times. Oper. Res. 43, 885–901 (1995)CrossRefGoogle Scholar
  2. 2.
    Amazon Elastic Compute Cloud EC2.
  3. 3.
    Babu, S.: Towards automatic optimization of MapReduce programs. In: Proceedings of ACM SoCC, pp. 137–142 (2010)Google Scholar
  4. 4.
    Baccelli, F., Gelenbe, E., Plateau, B.: An end-to-end approach to the resequencing problem. J. ACM 31(3), 474–485 (1984)CrossRefGoogle Scholar
  5. 5.
    Baccelli, F., Makowski, A.M., Shwartz, A.: The Fork–Join queue and related systems with synchronization constraints: stochastic ordering and computable bounds. Adv. Appl. Probab. 21(3), 629–660 (1989)CrossRefGoogle Scholar
  6. 6.
    Balsamo, S., Donatiello, L., Van Dijk, N.M.: Bound performance models of heterogeneous parallel processing systems. IEEE Trans. Parallel Distrib. Syst. 9(10), 1041–1056 (1998)CrossRefGoogle Scholar
  7. 7.
    Billingsley, P.: Probability and Measure, 3rd edn. Wiley, New York (1995)Google Scholar
  8. 8.
    Boxma, O., Koole, G., Liu, Z.: Queueing-theoretic solution methods for models of parallel and distributed systems. In: Proceedings of Performance Evaluation of Parallel and Distributed Systems. CWI Tract 105, pp. 1–24 (1994)Google Scholar
  9. 9.
    Buffet, E., Duffield, N.G.: Exponential upper bounds via martingales for multiplexers with Markovian arrivals. J. Appl. Probab. 31(4), 1049–1060 (1994)CrossRefGoogle Scholar
  10. 10.
    Chang, C.S.: Performance Guarantees in Communication Networks. Springer, New York (2000)CrossRefGoogle Scholar
  11. 11.
    Chen, Y., Alspaugh, S., Katz, R.: Interactive analytical processing in big data systems: a cross-industry study of mapreduce workloads. Proc. VLDB Endow. 5(12), 1802–1813 (2012)CrossRefGoogle Scholar
  12. 12.
    Ciucu, F., Poloczek, F., Schmitt, J.: Sharp per-flow delay bounds for bursty arrivals: the case of FIFO, SP, and EDF scheduling. In: Proceedings of IEEE INFOCOM, pp. 1896–1904 (2014)Google Scholar
  13. 13.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  14. 14.
    Duffield, N.: Exponential bounds for queues with Markovian arrivals. Queueing Syst. 17(3–4), 413–430 (1994)CrossRefGoogle Scholar
  15. 15.
    Flatto, L., Hahn, S.: Two parallel queues created by arrivals with two demands I. SIAM J. Appl. Math. 44(5), 1041–1053 (1984)CrossRefGoogle Scholar
  16. 16.
    Ganesh, A., O’Connell, N., Wischik, D.: Big Queues. No. 1838 in Lecture Notes in Mathematics. Springer, New York (2004)Google Scholar
  17. 17.
    Gibbens, R.J.: Traffic characterisation and effective bandwidths for broadband network traces. J. R. Stat. Soc. Ser. B. Stat. Methodol. 4, 169–179 (1996)Google Scholar
  18. 18.
    Han, Y., Makowski, A.: Resequencing delays under multipath routing—asymptotics in a simple queueing model. In: Proceedings of IEEE INFOCOM, pp. 1–12 (2006)Google Scholar
  19. 19.
    Harrus, G., Plateau, B.: Queueing analysis of a reordering issue. IEEE Trans. Softw. Eng. 8(2), 113–123 (1982)CrossRefGoogle Scholar
  20. 20.
    Jiang, Y., Liu, Y.: Stochastic Network Calculus. Springer, New York (2008)Google Scholar
  21. 21.
    Joshi, G., Liu, Y., Soljanin, E.: Coding for fast content download. In: Proceedings of the Allerton Conference on Communication, Control, and Computing, pp. 326–333 (2012)Google Scholar
  22. 22.
    Joshi, G., Liu, Y., Soljanin, E.: On the delay-storage trade-off in content download from coded distributed storage systems. IEEE J. Sel. Areas Commun. 32(5), 989–997 (2014)CrossRefGoogle Scholar
  23. 23.
    Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of data center traffic: measurements & analysis. In: Proceedings of ACM IMC, pp. 202–208 (2009)Google Scholar
  24. 24.
    Kavulya, S., Tan, J., Gandhi, R., Narasimhan, P.: An analysis of traces from a production MapReduce cluster. In: Proceedings of IEEE/ACM CCGRID, pp. 94–103 (2010)Google Scholar
  25. 25.
    Kemper, B., Mandjes, M.: Mean sojourn times in two-queue Fork–Join systems: bounds and approximations. OR Spectr. 34(3), 723–742 (2012)CrossRefGoogle Scholar
  26. 26.
    Kesidis, G., Urgaonkar, B., Shan, Y., Kamarava, S., Liebeherr, J.: Network calculus for parallel processing. In: Proceedings of the ACM MAMA Workshop (2015)Google Scholar
  27. 27.
    Kingman, J.F.C.: Inequalities in the theory of queues. J. R. Stat. Soc. Ser. B. Stat. Methodol. 32(1), 102–110 (1970)Google Scholar
  28. 28.
    Ko, S.S., Serfozo, R.F.: Sojourn times in G/M/1 Fork–Join networks. Naval Res. Logist. 55(5), 432–443 (2008)CrossRefGoogle Scholar
  29. 29.
    Lebrecht, A.S., Knottenbelt, W.J.: Response time approximations in Fork–Join queues. In: Proceedings of UKPEW (2007)Google Scholar
  30. 30.
    Lu, H., Pang, G.: Gaussian limits for a Fork–Join network with nonexchangeable synchronization in heavy traffic. Math. Oper. Res. 41(2), 560–595 (2016)CrossRefGoogle Scholar
  31. 31.
    Nelson, R., Tantawi, A.: Approximate analysis of Fork/Join synchronization in parallel queues. IEEE Trans. Comput. 37(6), 739–743 (1988)CrossRefGoogle Scholar
  32. 32.
    Pike, R., Dorward, S., Griesemer, R., Quinlan, S.: Interpreting the data: parallel analysis with Sawzall. Sci. Program. 13(4), 277–298 (2005)Google Scholar
  33. 33.
    Polato, I., Ré, R., Goldman, A., Kon, F.: A comprehensive view of Hadoop research—a systematic literature review. J. Netw. Comput. Appl. 46, 1–25 (2014)CrossRefGoogle Scholar
  34. 34.
    Poloczek, F., Ciucu, F.: Scheduling analysis with martingales. Perform. Eval. 79, 56–72 (2014)CrossRefGoogle Scholar
  35. 35.
    Raiciu, C., Barre, S., Pluntke, C., Greenhalgh, A., Wischik, D., Handley, M.: Improving datacenter performance and robustness with multipath TCP. SIGCOMM Comput. Commun. Rev. 41(4), 266–277 (2011)CrossRefGoogle Scholar
  36. 36.
    Rényi, A.: On the theory of order statistics. Acta Math. Hung. 4(3—-4), 191–231 (1953)CrossRefGoogle Scholar
  37. 37.
    Tan, J., Meng, X., Zhang, L.: Delay tails in MapReduce scheduling. SIGMETRICS Perform. Eval. Rev. 40(1), 5–16 (2012)CrossRefGoogle Scholar
  38. 38.
    Tan, J., Wang, Y., Yu, W., Zhang, L.: Non-work-conserving effects in MapReduce: diffusion limit and criticality. SIGMETRICS Perform. Eval. Rev. 42(1), 181–192 (2014)CrossRefGoogle Scholar
  39. 39.
    Varki, E.: Mean value technique for closed Fork–Join networks. SIGMETRICS Perform. Eval. Rev. 27(1), 103–112 (1999)CrossRefGoogle Scholar
  40. 40.
    Varma, S., Makowski, A.M.: Interpolation approximations for symmetric Fork–Join queues. Perform. Eval. 20(1–3), 245–265 (1994)CrossRefGoogle Scholar
  41. 41.
    Vianna, E., Comarela, G., Pontes, T., Almeida, J., Almeida, V., Wilkinson, K., Kuno, H., Dayal, U.: Analytical performance models for MapReduce workloads. Int. J. Parallel Program. 41(4), 495–525 (2013)CrossRefGoogle Scholar
  42. 42.
    White, T.: Hadoop: The Definitive Guide, 1st edn. O’Reilly Media, Inc., Sebastopol (2009)Google Scholar
  43. 43.
    Xia, Y., Tse, D.: On the large deviation of resequencing queue size: 2-M/M/1 case. IEEE Trans. Inf. Theory 54(9), 4107–4118 (2008)CrossRefGoogle Scholar
  44. 44.
    Zaharia, M., Konwinski, A., Joseph, A.D., Katz, R., Stoica, I.: Improving MapReduce performance in heterogeneous environments. In: Proceedings of USENIX OSDI, pp. 29–42 (2008)Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.University of Massachusetts AmherstAmherstUSA
  2. 2.University of WarwickCoventryUK
  3. 3.TU BerlinBerlinGermany

Personalised recommendations