High-Level and Productive Stream Parallelism for Dedup, Ferret, and Bzip2

  • Dalvan Griebler
  • Renato B. Hoffmann
  • Marco Danelutto
  • Luiz G. Fernandes
Part of the following topical collections:
  1. Special Issue on High-Level Parallel Programming


Parallel programming has been a challenging task for application programmers. Stream processing is an application domain present in several scientific, enterprise, and financial areas that lack suitable abstractions to exploit parallelism. Our goal is to assess the feasibility of state-of-the-art frameworks/libraries (Pthreads, TBB, and FastFlow) and the SPar domain-specific language for real-world streaming applications (Dedup, Ferret, and Bzip2) targeting multi-core architectures. SPar was specially designed to provide high-level and productive stream parallelism abstractions, supporting programmers with standard C++-11 annotations. For the experiments, we implemented three streaming applications. We discussed SPar’s programmability advantages compared to the frameworks in terms of productivity and structured parallel programming. The results demonstrate that SPar improves productivity and provides the necessary features to achieve similar performances compared to the state-of-the-art.


High-level parallelism Parallel programming Stream processing Parallel patterns Pipeline parallelism Streaming applications 



Authors thank the partial financial support from the following Brazilian research institutions: CAPES and FAPERGS. Moreover, this work has been partially supported by the EU H2020-ICT-2014-1 Project RePhrase (No. 644235).


  1. 1.
    Aldinucci, M., Danelutto, M., Kilpatrick, P., Meneghin, M., Torquati, M.: Accelerating code on multi-cores with FastFlow. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011 Parallel Processing. Lecture Notes in Computer Science, vol. 6853, pp. 170–181. Springer, Berlin (2011)CrossRefGoogle Scholar
  2. 2.
    Aldinucci, M., Danelutto, M., Kilpatrick, P., Torquati, M.: FastFlow: high-level and efficient streaming on multi-core. In: Pllana, S., Xhafa, F. (eds.) Programming Multi-core and Many-core Computing Systems, Parallel and Distributed Computing, vol. 1, p. 14. Wiley, New York (2014)Google Scholar
  3. 3.
    Bienia, C., Kumar, S., Singh, J.P., Li, K.: The PARSEC benchmark suite: characterization and architectural implications. In: 17th International Conference on Parallel Architectures and Compilation Techniques. PACT ’08, pp. 72–81. ACM, Toronto, Ontario, Canada (2008)Google Scholar
  4. 4.
    Chasapis, D., Marc, C., Moretó, M., Vidal, R., Ayguadé, E., Labarta, J., Valero, M.: PARSECSs: evaluating the impact of task parallelism in the PARSEC benchmark suite. ACM Trans. Archit. Code Optim. (TACO) 12(4), 41:1–41:22 (2016)Google Scholar
  5. 5.
    Cole, M.: Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, Cambridge (1989)zbMATHGoogle Scholar
  6. 6.
    Danelutto, M., de Matteis, T., de Sensi, D., Mencagli, G., Torquati, M.: P3ARSEC: Towards parallel patterns benchmarking. In: 32nd Annual ACM Symposium on Applied Computing, SAC ’17. ACM, Marrakech, Morocco (2017)Google Scholar
  7. 7.
    del Rio Astorga, D., Dolz, M.F., Fernández, J., García, J.D.: A generic parallel pattern interface for stream and data processing. Concurr. Comput. Pract. Exp. 29, e4175 (2017)CrossRefGoogle Scholar
  8. 8.
    Gilchrist, J.: Parallel compression with BZIP2. In: 16th IASTED International Conference on Parallel and Distributed Computing and Systems. PDCS’ 04, pp. 559–564. ACTA Press, MIT, Cambridge, USA (2004)Google Scholar
  9. 9.
    Griebler, D.: Domain-Specific Language and Support Tool for High-Level Stream Parallelism. Ph.D. thesis, Faculdade de Informática - PUCRS, Porto Alegre, Brazil (2016)Google Scholar
  10. 10.
    Griebler, D., Danelutto, M., Torquati, M., Fernandes, L.G.: An embedded C++ domain-specific language for stream parallelism. Parallel computing: on the road to exascale. In: Proceedings of the International Conference on Parallel Computing, ParCo’15, pp. 317–326. IOS Press, Edinburgh, Scotland, UK (2015)Google Scholar
  11. 11.
    Griebler, D., Danelutto, M., Torquati, M., Fernandes, L.G.: SPar: a DSL for high-level and productive stream parallelism. Parallel Process. Lett. 27(01), 20 (2017)MathSciNetCrossRefGoogle Scholar
  12. 12.
    ISO/IEC.: Information Technology—Programming Languages—C++. Technical report, International Standard, Geneva, Switzerland (2014).
  13. 13.
    Laird, L.M., Brennan, M.C.: Software Measurement and Estimation: A Practical Approach, 1st edn. Wiley-IEEE Computer Society Pr, Hoboken (2006)CrossRefGoogle Scholar
  14. 14.
    Lee, I.T.A., Leiserson, C.E., Schardl, T.B., Zhang, Z., Sukha, J.: On-the-fly pipeline parallelism. ACM Trans. Parallel Comput. 2(3), 17:1–17:42 (2015)CrossRefGoogle Scholar
  15. 15.
    Mattson, T.G., Sanders, B.A., Massingill, B.L.: Patterns for Parallel Programming. Addison-Wesley, Boston (2005)zbMATHGoogle Scholar
  16. 16.
    McCool, M., Robison, A.D., Reinders, J.: Structured Parallel Programming: Patterns for Efficient Computation. Morgan Kaufmann, Burlington (2012)Google Scholar
  17. 17.
    Navarro, A., Asenjo, R., Tabik, S., Cascaval, C.: Analytical modeling of pipeline parallelism. In: 18th International Conference on Parallel Architectures and Compilation Techniques. PACT ’09, pp. 281–290. IEEE, Washington, DC, USA (2009)Google Scholar
  18. 18.
    Reed, E.C., Chen, N., Johnson, R.E.: Expressing pipeline parallelism using TBB constructs: a case study on what works and what doesn’t. In: ACM SIGPLAN Conference on Systems. Programming, Languages and Applications: Software for Humanity (SPLASH), SPLASH ’11 Workshops, pp. 133–138. ACM, Portland, Oregon, USA (2011)Google Scholar
  19. 19.
    Reinders, J.: Intel Threading Building Blocks. O’Reilly, Newton (2007)Google Scholar
  20. 20.
    Seward, J.: A Program and Library for Data Compression. (2017)
  21. 21.
    Soulé, R., Gordon, M.I., Amarasinghe, S., Grimm, R., Hirzel, M.: Dynamic expressivity with static optimization for streaming languages. In: 7th ACM International Conference on Distributed Event-based Systems. DEBS ’13, pp. 159–170. ACM, Arlington, Texas, USA (2013)Google Scholar
  22. 22.
    Sujeeth, A.K., Brown, K.J., Lee, H., Rompf, T., Chafi, H., Odersky, M., Olukotun, K.: Delite: a compiler architecture for performance-oriented embedded domain-specific languages. ACM Trans. Embed. Comput. Syst. (TECS) 13(4), 25 (2014)Google Scholar
  23. 23.
    Thies, W.: Language and Compiler Support for Stream Programs. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts (2009)Google Scholar
  24. 24.
    Zhang, D., Li, Q.J., Rabbah, R., Amarasinghe, S.: A lightweight streaming layer for multicore execution. SIGARCH Comput. Archit. News 36(2), 18–27 (2008)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of InformaticsPontifical Catholic University of Rio Grande do SulPorto AlegreBrazil
  2. 2.Computer Science DepartmentUniversity of PisaPisaItaly

Personalised recommendations