High-Level and Productive Stream Parallelism for Dedup, Ferret, and Bzip2

  • 200 Accesses

  • 4 Citations


Parallel programming has been a challenging task for application programmers. Stream processing is an application domain present in several scientific, enterprise, and financial areas that lack suitable abstractions to exploit parallelism. Our goal is to assess the feasibility of state-of-the-art frameworks/libraries (Pthreads, TBB, and FastFlow) and the SPar domain-specific language for real-world streaming applications (Dedup, Ferret, and Bzip2) targeting multi-core architectures. SPar was specially designed to provide high-level and productive stream parallelism abstractions, supporting programmers with standard C++-11 annotations. For the experiments, we implemented three streaming applications. We discussed SPar’s programmability advantages compared to the frameworks in terms of productivity and structured parallel programming. The results demonstrate that SPar improves productivity and provides the necessary features to achieve similar performances compared to the state-of-the-art.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.


  1. 1.

    Aldinucci, M., Danelutto, M., Kilpatrick, P., Meneghin, M., Torquati, M.: Accelerating code on multi-cores with FastFlow. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011 Parallel Processing. Lecture Notes in Computer Science, vol. 6853, pp. 170–181. Springer, Berlin (2011)

  2. 2.

    Aldinucci, M., Danelutto, M., Kilpatrick, P., Torquati, M.: FastFlow: high-level and efficient streaming on multi-core. In: Pllana, S., Xhafa, F. (eds.) Programming Multi-core and Many-core Computing Systems, Parallel and Distributed Computing, vol. 1, p. 14. Wiley, New York (2014)

  3. 3.

    Bienia, C., Kumar, S., Singh, J.P., Li, K.: The PARSEC benchmark suite: characterization and architectural implications. In: 17th International Conference on Parallel Architectures and Compilation Techniques. PACT ’08, pp. 72–81. ACM, Toronto, Ontario, Canada (2008)

  4. 4.

    Chasapis, D., Marc, C., Moretó, M., Vidal, R., Ayguadé, E., Labarta, J., Valero, M.: PARSECSs: evaluating the impact of task parallelism in the PARSEC benchmark suite. ACM Trans. Archit. Code Optim. (TACO) 12(4), 41:1–41:22 (2016)

  5. 5.

    Cole, M.: Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, Cambridge (1989)

  6. 6.

    Danelutto, M., de Matteis, T., de Sensi, D., Mencagli, G., Torquati, M.: P3ARSEC: Towards parallel patterns benchmarking. In: 32nd Annual ACM Symposium on Applied Computing, SAC ’17. ACM, Marrakech, Morocco (2017)

  7. 7.

    del Rio Astorga, D., Dolz, M.F., Fernández, J., García, J.D.: A generic parallel pattern interface for stream and data processing. Concurr. Comput. Pract. Exp. 29, e4175 (2017)

  8. 8.

    Gilchrist, J.: Parallel compression with BZIP2. In: 16th IASTED International Conference on Parallel and Distributed Computing and Systems. PDCS’ 04, pp. 559–564. ACTA Press, MIT, Cambridge, USA (2004)

  9. 9.

    Griebler, D.: Domain-Specific Language and Support Tool for High-Level Stream Parallelism. Ph.D. thesis, Faculdade de Informática - PUCRS, Porto Alegre, Brazil (2016)

  10. 10.

    Griebler, D., Danelutto, M., Torquati, M., Fernandes, L.G.: An embedded C++ domain-specific language for stream parallelism. Parallel computing: on the road to exascale. In: Proceedings of the International Conference on Parallel Computing, ParCo’15, pp. 317–326. IOS Press, Edinburgh, Scotland, UK (2015)

  11. 11.

    Griebler, D., Danelutto, M., Torquati, M., Fernandes, L.G.: SPar: a DSL for high-level and productive stream parallelism. Parallel Process. Lett. 27(01), 20 (2017)

  12. 12.

    ISO/IEC.: Information Technology—Programming Languages—C++. Technical report, International Standard, Geneva, Switzerland (2014).

  13. 13.

    Laird, L.M., Brennan, M.C.: Software Measurement and Estimation: A Practical Approach, 1st edn. Wiley-IEEE Computer Society Pr, Hoboken (2006)

  14. 14.

    Lee, I.T.A., Leiserson, C.E., Schardl, T.B., Zhang, Z., Sukha, J.: On-the-fly pipeline parallelism. ACM Trans. Parallel Comput. 2(3), 17:1–17:42 (2015)

  15. 15.

    Mattson, T.G., Sanders, B.A., Massingill, B.L.: Patterns for Parallel Programming. Addison-Wesley, Boston (2005)

  16. 16.

    McCool, M., Robison, A.D., Reinders, J.: Structured Parallel Programming: Patterns for Efficient Computation. Morgan Kaufmann, Burlington (2012)

  17. 17.

    Navarro, A., Asenjo, R., Tabik, S., Cascaval, C.: Analytical modeling of pipeline parallelism. In: 18th International Conference on Parallel Architectures and Compilation Techniques. PACT ’09, pp. 281–290. IEEE, Washington, DC, USA (2009)

  18. 18.

    Reed, E.C., Chen, N., Johnson, R.E.: Expressing pipeline parallelism using TBB constructs: a case study on what works and what doesn’t. In: ACM SIGPLAN Conference on Systems. Programming, Languages and Applications: Software for Humanity (SPLASH), SPLASH ’11 Workshops, pp. 133–138. ACM, Portland, Oregon, USA (2011)

  19. 19.

    Reinders, J.: Intel Threading Building Blocks. O’Reilly, Newton (2007)

  20. 20.

    Seward, J.: A Program and Library for Data Compression. (2017)

  21. 21.

    Soulé, R., Gordon, M.I., Amarasinghe, S., Grimm, R., Hirzel, M.: Dynamic expressivity with static optimization for streaming languages. In: 7th ACM International Conference on Distributed Event-based Systems. DEBS ’13, pp. 159–170. ACM, Arlington, Texas, USA (2013)

  22. 22.

    Sujeeth, A.K., Brown, K.J., Lee, H., Rompf, T., Chafi, H., Odersky, M., Olukotun, K.: Delite: a compiler architecture for performance-oriented embedded domain-specific languages. ACM Trans. Embed. Comput. Syst. (TECS) 13(4), 25 (2014)

  23. 23.

    Thies, W.: Language and Compiler Support for Stream Programs. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts (2009)

  24. 24.

    Zhang, D., Li, Q.J., Rabbah, R., Amarasinghe, S.: A lightweight streaming layer for multicore execution. SIGARCH Comput. Archit. News 36(2), 18–27 (2008)

Download references


Authors thank the partial financial support from the following Brazilian research institutions: CAPES and FAPERGS. Moreover, this work has been partially supported by the EU H2020-ICT-2014-1 Project RePhrase (No. 644235).

Author information

Correspondence to Dalvan Griebler.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Griebler, D., Hoffmann, R.B., Danelutto, M. et al. High-Level and Productive Stream Parallelism for Dedup, Ferret, and Bzip2. Int J Parallel Prog 47, 253–271 (2019).

Download citation


  • High-level parallelism
  • Parallel programming
  • Stream processing
  • Parallel patterns
  • Pipeline parallelism
  • Streaming applications