Abstract
Current hardware development trends exhibit clear inclination towards parallelism. Multicore CPUs as well as many-core architectures such as GPUs or Xeon Phi devices are widely present in both high-end servers and common desktop PCs. In order to utilize the computational power of these parallel platforms, the applications must be designed in a way that intensively exploits parallel processing. In our work, we propose techniques that simplify the application decomposition process in data streaming systems. The data streaming paradigm may be applied in many data-intensive applications, e.g., database management systems or scientific data processing. In order to employ these techniques, we have developed a data streaming language called Bobolang that simplifies the design of the application. This approach allows the programmer to write strictly serial operators in a traditional language and then interconnect these operators in an execution plan, that presents opportunities for automated parallel processing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Field-programmable gate array.
References
Aldinucci, M., Danelutto, M., Kilpatrick, P., Meneghin, M., Torquati, M.: Accelerating code on multi-cores with fastflow. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011 Parallel Processing. Lecture Notes in Computer Science, vol. 6853, pp. 170–181. Springer, Berlin (2011)
Bednarek, D., Dokulil, J., Yaghob, J., Zavoral, F.: Bobox: parallelization framework for data processing. In: Advances in Information Technology and Applied Computing (2012)
Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M., Hanrahan, P.: Brook for GPUs: stream computing on graphics hardware. ACM Trans. Graph. 23, 777–786 (2004)
Consel, C., Hamdi, H., Réveillère, L., Singaravelu, L., Yu, H., Pu, C.: Spidle: a DSL approach to specifying streaming applications. In: Proceedings of the 2nd International Conference on Generative Programming and Component Engineering, pp. 1–17. Springer, New York, NY, USA (2003)
Cordes, D., Heinig, A., Marwedel, P., Mallik, A,: Automatic extraction of pipeline parallelism for embedded software using linear programming. In: Parallel and Distributed Systems (ICPADS), 2011 IEEE 17th International Conference on, pp. 699–706 (2011)
Das, A., Dally, W.J., Mattson, P., Compiling for stream processing. In: Proceedings of the 15th International Conference on Parallel Architectures and Compilation Techniques, pp. 33–42. ACM, New York, NY, USA (2006)
Falt, Z., Bednárek, D., Kruliš, M., Yaghob, J., Zavoral, F.: Bobolang: a language for parallel streaming applications. In: Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, pp. 311–314. ACM (2014)
Franklin, M., Tyson, E., Buckley, J., Crowley, P., Maschmeyer, J.: Auto-pipe and the X language: a pipeline design tool and description language. In: 20th International Parallel and Distributed Processing Symposium. IEEE (2006)
Jagannathan, R., Dodd, C., Agi, I.: Glu: a high-level system for granular data-parallel programming. Concurrency—Pract. Expe. 9(1), 63–83 (1997)
Kapasi, U.J., Dally, W.J., Rixner, S., Owens, J.D., Khailany, B.: Programmable stream processors. IEEE Comput. 36, 282–288 (2003)
Mark, W.R., Steven, R., Kurt, G., Mark, A., Kilgard, J.: Cg: a system for programming graphics hardware in a C-like language. ACM Trans. Graph. 22, 896–907 (2003)
Muller, S.C., Alonso, G., Amara, A., Csillaghy, A.: Pydron: semi-automatic parallelization for multi-core and the cloud. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), pp. 645–659. USENIX Association (2014)
Reinders, J.: Intel threading building blocks. O’Reilly, Sebastopol (2007)
Thies, W., Karczmarek, M., Amarasinghe, S.: StreamIt: a language for streaming applications. In: Compiler Construction, pp 179–196. Springer (2002)
Vandierendonck, H., Rul, S., De Bosschere, K.: The paralax infrastructure: automatic parallelization with a helping hand. In: Parallel Architectures and Compilation Techniques, 19th International Conference, Proceedings, pp. 389–400. Association for Computing Machinery (ACM) (2010)
Zhang, D., Li, Z.Z., Song, H., Liu, L.: A programming model for an embedded media processing architecture. In: Embedded Computer Systems: Architectures, Modeling, and Simulation, pp. 251–261. Springer (2005)
Acknowledgments
This work was supported by the Czech Science Foundation (GACR) projects P103-14-14292P and P103-13-08195S and by Specific Research SVV-2015-260222.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Kruliš, M., Bednárek, D., Falt, Z., Yaghob, J., Zavoral, F. (2016). Towards Semi-automated Parallelization of Data Stream Processing. In: Novais, P., Camacho, D., Analide, C., El Fallah Seghrouchni, A., Badica, C. (eds) Intelligent Distributed Computing IX. Studies in Computational Intelligence, vol 616. Springer, Cham. https://doi.org/10.1007/978-3-319-25017-5_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-25017-5_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25015-1
Online ISBN: 978-3-319-25017-5
eBook Packages: EngineeringEngineering (R0)