Towards Semi-automated Parallelization of Data Stream Processing

  • Martin Kruliš
  • David Bednárek
  • Zbyněk Falt
  • Jakub Yaghob
  • Filip Zavoral
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 616)

Abstract

Current hardware development trends exhibit clear inclination towards parallelism. Multicore CPUs as well as many-core architectures such as GPUs or Xeon Phi devices are widely present in both high-end servers and common desktop PCs. In order to utilize the computational power of these parallel platforms, the applications must be designed in a way that intensively exploits parallel processing. In our work, we propose techniques that simplify the application decomposition process in data streaming systems. The data streaming paradigm may be applied in many data-intensive applications, e.g., database management systems or scientific data processing. In order to employ these techniques, we have developed a data streaming language called Bobolang that simplifies the design of the application. This approach allows the programmer to write strictly serial operators in a traditional language and then interconnect these operators in an execution plan, that presents opportunities for automated parallel processing.

References

  1. 1.
    Aldinucci, M., Danelutto, M., Kilpatrick, P., Meneghin, M., Torquati, M.: Accelerating code on multi-cores with fastflow. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011 Parallel Processing. Lecture Notes in Computer Science, vol. 6853, pp. 170–181. Springer, Berlin (2011)Google Scholar
  2. 2.
    Bednarek, D., Dokulil, J., Yaghob, J., Zavoral, F.: Bobox: parallelization framework for data processing. In: Advances in Information Technology and Applied Computing (2012)Google Scholar
  3. 3.
    Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M., Hanrahan, P.: Brook for GPUs: stream computing on graphics hardware. ACM Trans. Graph. 23, 777–786 (2004)CrossRefGoogle Scholar
  4. 4.
    Consel, C., Hamdi, H., Réveillère, L., Singaravelu, L., Yu, H., Pu, C.: Spidle: a DSL approach to specifying streaming applications. In: Proceedings of the 2nd International Conference on Generative Programming and Component Engineering, pp. 1–17. Springer, New York, NY, USA (2003)Google Scholar
  5. 5.
    Cordes, D., Heinig, A., Marwedel, P., Mallik, A,: Automatic extraction of pipeline parallelism for embedded software using linear programming. In: Parallel and Distributed Systems (ICPADS), 2011 IEEE 17th International Conference on, pp. 699–706 (2011)Google Scholar
  6. 6.
    Das, A., Dally, W.J., Mattson, P., Compiling for stream processing. In: Proceedings of the 15th International Conference on Parallel Architectures and Compilation Techniques, pp. 33–42. ACM, New York, NY, USA (2006)Google Scholar
  7. 7.
    Falt, Z., Bednárek, D., Kruliš, M., Yaghob, J., Zavoral, F.: Bobolang: a language for parallel streaming applications. In: Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, pp. 311–314. ACM (2014)Google Scholar
  8. 8.
    Franklin, M., Tyson, E., Buckley, J., Crowley, P., Maschmeyer, J.: Auto-pipe and the X language: a pipeline design tool and description language. In: 20th International Parallel and Distributed Processing Symposium. IEEE (2006)Google Scholar
  9. 9.
    Jagannathan, R., Dodd, C., Agi, I.: Glu: a high-level system for granular data-parallel programming. Concurrency—Pract. Expe. 9(1), 63–83 (1997)CrossRefGoogle Scholar
  10. 10.
    Kapasi, U.J., Dally, W.J., Rixner, S., Owens, J.D., Khailany, B.: Programmable stream processors. IEEE Comput. 36, 282–288 (2003)CrossRefGoogle Scholar
  11. 11.
    Mark, W.R., Steven, R., Kurt, G., Mark, A., Kilgard, J.: Cg: a system for programming graphics hardware in a C-like language. ACM Trans. Graph. 22, 896–907 (2003)CrossRefGoogle Scholar
  12. 12.
    Muller, S.C., Alonso, G., Amara, A., Csillaghy, A.: Pydron: semi-automatic parallelization for multi-core and the cloud. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), pp. 645–659. USENIX Association (2014)Google Scholar
  13. 13.
    Reinders, J.: Intel threading building blocks. O’Reilly, Sebastopol (2007)Google Scholar
  14. 14.
    Thies, W., Karczmarek, M., Amarasinghe, S.: StreamIt: a language for streaming applications. In: Compiler Construction, pp 179–196. Springer (2002)Google Scholar
  15. 15.
    Vandierendonck, H., Rul, S., De Bosschere, K.: The paralax infrastructure: automatic parallelization with a helping hand. In: Parallel Architectures and Compilation Techniques, 19th International Conference, Proceedings, pp. 389–400. Association for Computing Machinery (ACM) (2010)Google Scholar
  16. 16.
    Zhang, D., Li, Z.Z., Song, H., Liu, L.: A programming model for an embedded media processing architecture. In: Embedded Computer Systems: Architectures, Modeling, and Simulation, pp. 251–261. Springer (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Martin Kruliš
    • 1
  • David Bednárek
    • 1
  • Zbyněk Falt
    • 1
  • Jakub Yaghob
    • 1
  • Filip Zavoral
    • 1
  1. 1.Charles University in PraguePragueCzech Republic

Personalised recommendations