Advertisement

Creating Distributed Execution Plans with BobolangNG

  • David Bednárek
  • Martin Kruliš
  • Jakub Yaghob
  • Filip Zavoral
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10048)

Abstract

Execution plans constitute the traditional interface between DBMS front-ends and back-ends; similar networks of interconnected operators are found also outside database systems. Tasks like adapting execution plans for distributed or heterogeneous runtime environments require a plan transformation mechanism which is simple enough to produce predictable results while general enough to express advanced communication schemes required for instance in skew-resistant partitioning. In this paper, we describe the BobolangNG language designed to express execution plans as well as their transformations, based on hierarchical models known from many environments but enhanced with a novel compile-time mechanism of component multiplication. Compared to approaches based on general graph rewriting, the plan transformation in BobolangNG is not iterative; therefore the consequences and limitations of the process are easier to understand and the development of distribution strategies and experimenting with distributed plans are easier and safer.

Keywords

Execution plan Distributed computing Partitioning Distributed database Datalog Modeling language 

References

  1. 1.
    Bruno, N., Kwon, Y., Wu, M.C.: Advanced join strategies for large-scale distributed computation. Proc. VLDB Endow. 7(13), 1484–1495 (2014). http://dx.doi.org/10.14778/2733004.2733020 CrossRefGoogle Scholar
  2. 2.
    Falt, Z., Bednárek, D., Kruliš, M., Yaghob, J., Zavoral, F.: Bobolang: a language for parallel streaming applications. In: Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing, pp. 311–314. ACM (2014)Google Scholar
  3. 3.
    Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. In: ACM SIGOPS Operating Systems Review, vol. 41, pp. 59–72. ACM (2007)Google Scholar
  4. 4.
    Ke, Q., Isard, M., Yu, Y.: Optimus: a dynamic rewriting framework for data-parallel execution plans. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 15–28. ACM (2013)Google Scholar
  5. 5.
    Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: distributed stream computing platform. In: Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW), pp. 170–177. IEEE (2010)Google Scholar
  6. 6.
    Levandoski, J.J., Eldawy, A., Mokbel, M.F., Khalefa, M.E.: Flexible and extensible preference evaluation in database systems. ACM Trans. Database Syst. (TODS) 38(3), 17 (2013)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Murray, D.G., McSherry, F., Isaacs, R., Isard, M., Barham, P., Abadi, M.: Naiad: a timely dataflow system. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 439–455. ACM (2013)Google Scholar
  8. 8.
    Murray, D.G., Schwarzkopf, M., Smowton, C., Smith, S., Madhavapeddy, A., Hand, S.: Ciel: a universal execution engine for distributed data-flow computing. In: Proceedings of 8th ACM/USENIX Symposium on Networked Systems Design and Implementation, pp. 113–126 (2011)Google Scholar
  9. 9.
    Murray, D.G., Hand, S.: Scripting the cloud with skywriting. HotCloud 10, 12 (2010)Google Scholar
  10. 10.
    Nagendra, M., Candan, K.S.: Efficient processing of skyline-join queries over multiple data sources. ACM Trans. Database Syst. (TODS) 40(2), 10 (2015)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Oscar, B., Sam, R., Ian, O., Jimmy, L.: Summingbird: a framework for integrating batch and online mapreduce computations. Proc. VLDB Endow. 7(13), 1441–1451 (2014)CrossRefGoogle Scholar
  12. 12.
    Gulisano, V., Jimenez-Peris, R., Patino-Martinez, M., Valduriez, P.: Streamcloud: a large scale data streaming system. In: Proceedings of the 30th International Conference on Distributed Computing Systems (ICDCS), pp. 126–137. IEEE (2010)Google Scholar
  13. 13.
    Van Emden, M.H., Kowalski, R.A.: The semantics of predicate logic as a programming language. J. ACM 23(4), 733–742 (1976). http://doi.acm.org/10.1145/321978.321991 MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • David Bednárek
    • 1
  • Martin Kruliš
    • 1
  • Jakub Yaghob
    • 1
  • Filip Zavoral
    • 1
  1. 1.Parallel Architectures/Algorithms/Applications Research Group, Faculty of Mathematics and PhysicsCharles UniversityPragueCzech Republic

Personalised recommendations