Datalyzer: Streaming Data Applications Made Easy

  • Mario González-Jiménez
  • Juan de LaraEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10845)


Nowadays, streaming data are continuously generated from thousands of sources, including social networks, mobile apps, sensors, e-commerce transactions, and many more. Hence, it becomes very useful to build applications able to process these data, with the purpose of filtering interesting parts, monitor their run-time evolution, persist valuable chunks, trigger events upon certain conditions are met and provide analytics. While several frameworks and systems have emerged to create this kind of applications, these systems tend to be low-level, based on complicated APIs, challenging to install and configure for end-users, and requiring from high performant hardware for their execution. Our goal is to lower the entry level to develop, deploy and run streaming applications.

To accomplish this goal, we propose Datalyzer, an approach to create streaming data applications on the cloud based on a visual language. This way, Datalyzer provides a facility to describe streaming data sources in an open way, and a visual language to describe the execution flow of the streaming application. Datalyzer is based on model-based development principles, where code is generated automatically, and then compiled, deployed and executed on the cloud. As a proof of concept, we describe a case study in enterprise systems, and how it can be built using our prototype tool.


Streaming data Data transformation Data monitoring Cloud-based development environments Model-based development Code generation 



Work partially funded by the Spanish MINECO (TIN2014-52129-R) and the R&D programme of the Madrid Region (S2013/ICE-3006).


  1. 1.
    Brambilla, M., Cabot, J., Wimmer, M.: Model-Driven Software Engineering in Practice. Synthesis Lectures on Software Engineering, 2nd edn. Morgan & Claypool Publishers, San Rafael (2017)Google Scholar
  2. 2.
    de Assunção, M.D., Veith, A.D.S., Buyya, R.: Distributed data stream processing and edge computing: a survey on resource elasticity and future directions. J. Netw. Comput. Appl. 103, 1–17 (2018)CrossRefGoogle Scholar
  3. 3.
    Dindar, N., Tatbul, N., Miller, R.J., Haas, L.M., Botan, I.: Modeling the execution semantics of stream processing engines with SECRET. VLDB J. 22(4), 421–446 (2013)CrossRefGoogle Scholar
  4. 4.
    Harth, A., Knoblock, C.A., Stadtmüller, S., Studer, R., Szekely, P.A.: On-the-fly integration of static and dynamic sources. In: COLD. CEUR Workshop Proceedings, vol. 1034 (2013)Google Scholar
  5. 5.
    Hirzel, M., et al.: IBM streams processing language: analyzing big data in motion. IBM J. Res. Dev. 57(3/4), 7 (2013)CrossRefGoogle Scholar
  6. 6.
    Luckham, D.C.: The power of events - an introduction to complex event processing in distributed enterprise systems. ACM (2005)Google Scholar
  7. 7.
    Rettig, L., Khayati, M., Cudré-Mauroux, P., Piórkowski, M.: Online anomaly detection over big data streams. In: 2015 IEEE International Conference on Big Data, pp. 1113–1122. IEEE (2015)Google Scholar
  8. 8.
    Stephens, R.: A survey of stream processing. Acta Inf. 34(7), 491–541 (1997)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Tatbul, N.: Streaming data integration: challenges and opportunities. In: IEEE 26th International Conference on Data Engineering Workshops (ICDEW 2010), pp. 155–158 (2010)Google Scholar
  10. 10.
    Thies, W., Karczmarek, M., Amarasinghe, S.: StreamIt: a language for streaming applications. In: Horspool, R.N. (ed.) CC 2002. LNCS, vol. 2304, pp. 179–196. Springer, Heidelberg (2002). Scholar
  11. 11.
    W3C. XML Path Language (XPath) 3.1.
  12. 12.
    Zhuang, Z., Feng, T., Pan, Y., Ramachandra, H., Sridharan, B.: Effective multi-stream joining in Apache Samza framework. In: 2016 IEEE International Conference on Big Data, pp. 267–274. IEEE Computer Society (2016). See also

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Modelling and Software Engineering Research GroupMadridSpain
  2. 2.Universidad Autónoma de MadridMadridSpain

Personalised recommendations