Skip to main content

Towards a Framework for Data Stream Processing in the Fog

Abstract

In volatile data streams as encountered in the Internet of Things (IoT), the data volume to be processed changes permanently. Hence, to ensure timely data processing, there is a need to reconfigure the computational resources used for processing data streams. Up to now, mostly cloud-based computational resources have been utilized for this. However, cloud data centers are usually located far away from IoT data sources, which leads to an increase in latency since data needs to be sent from the data sources to the cloud and back. With the advent of fog computing, it is possible to perform data processing in the cloud as well as at the edge of the network, i. e., by exploiting the computational resources offered by networked devices. This leads to decreased latency and a lower communication overhead. Despite this, there is currently a lack of approaches to data stream processing which explicitly exploit the computational resources available in the fog.

Within this paper, we consider the usage of fog-based computational resources for the purposes of data stream processing in the IoT. For this, we introduce a representative application scenario in the field of Industry 4.0 and present a framework for stream processing in the fog.

References

  1. Al-Fuqaha AI, Guizani M, Mohammadi M, Mohammed Aledhari M, Ayyash M (2015) Internet of things: A survey on enabling technologies, protocols, and applications. IEEE Commun Surv Tutor 17(4):2347–2376

    Article  Google Scholar 

  2. Andrade H, Gedik B, Turaga D (2014) Fundamentals of Stream Processing. Cambridge University Press

  3. Atzori L, Iera A, Morabito G (2010) The internet of things: A survey. Comput Networks 54:2787–2805

    Article  MATH  Google Scholar 

  4. Bonomi F, Milito R, Natarajan P, Zhu J: Fog computing: A platform for internet of things and analytics. In: Bessis N, Dobre C (eds) Big Data and Internet of Things: A Roadmap for Smart Environments, Studies in Computational Intelligence, vol 546. Springer, pp 169–186

  5. Cardellini V, Lo Presti F, Nardelli M, Russo Russo G (2018) Decentralized self-adaptation for elastic data stream processing. Future Gener Comp Sy 87:171–185

    Article  MATH  Google Scholar 

  6. Chen N, Chen Y, You Y, Ling H, Liang P, Zimmermann R: Dynamic Urban Surveillance Video Stream Processing Using Fog Computing. In: 2016 IEEE Second International Conference on Multimedia Big Data. IEEE, pp 105–112

  7. Cortés R, Bonnaire X, Marin O, Sens P (2015) Stream Processing of Healthcare Sensor Data: Studying User Traces to Identify Challenges from a Big Data Perspective. In: 4th International Workshop on Body Area Sensor Networks, Procedia Computer Science, vol 52. Elsevier, pp 1004–1009

  8. Dastjerdi AV, Gupta H, Calheiros RN, Ghosh SK, Buyya R (2016) Fog computing: Principles, architectures, and applications. In: Buyya R, Dastjerdi AV (eds) Internet of Things: Principles and Paradigms, chap 4. Morgan Kaufmann, pp 61–75

  9. Dautov R, Distefano S, Bruneo D, Longo F, Merlino G, Puliafito A (2018) Data processing in cyber-physical-social systems through edge computing. IEEE Access 6:29822–29835

    Article  Google Scholar 

  10. Heinze T, Roediger L, Meister A, Ji Y, Jerzak Z, Fetzer C (2015) Online parameter optimization for elastic data stream processing. In: Sixth ACM Symposium on Cloud Computing. ACM, pp 276–287

  11. Hießl T, Karagiannis V, Hochreiner C, Schulte S, Nardelli M (2019) Optimal placement of stream processing operators in the fog (forthcoming). In: 3rd IEEE International Conference on Fog and Edge Computing. IEEE

  12. Hochreiner C, Schulte S, Dustdar S, Lécué F (2015) Elastic stream processing for distributed environments. IEEE Internet Comput 19:54–59

    Article  Google Scholar 

  13. Hochreiner C, Vögler M, Schulte S, Dustdar S (2017) Cost-efficient enactment of stream processing topologies. PeerJ Comput Sci 3:e141

    Article  Google Scholar 

  14. Hochreiner C, Vögler M, Waibel P, Dustdar S (2016) VISP: An Ecosystem for Elastic Data Stream Processing for the Internet of Things. In: 20th International Enterprise Distributed Object Computing Conference. IEEE, pp 19–29

  15. Jeschke S, Brecher C, Meisen T, Özdemir D, Eschert T (2017) Industrial internet of things and cyber manufacturing systems. In: Jeschke S, Brecher C, Song H, Rawat DB (eds) Industrial Internet of Things: Cybermanufacturing Systems. Springer, pp 3–19

  16. Kolozali S, Bermúdez-Edo M, Puschmann D, Ganz F, Barnaghi PM (2014) A Knowledge-Based Approach for Real-Time IoT Data Stream Annotation and Processing. In: 2014 IEEE International Conference on Internet of Things. IEEE, pp 215–222

  17. Lee EA (2010) CPS Foundations. In: 47th Design Automation Conference. IEEE, pp 737–742

  18. OpenFog Consortium (2018) IEEE Standard for Adoption of OpenFog Reference Architecture for Fog Computing. IEEE Std 1934-2018

  19. Ottenwälder B, Koldehofe B, Rothermel K, Ramachandran U (2013) MigCEP: Operator Migration for Mobility Driven Distributed Complex Event Processing. In: 7th ACM International Conference on Distributed Event-Based Systems. ACM, pp 183–194

  20. Perera C, Zaslavsky AB, Christen P, Georgakopoulos D (2014) Context aware computing for the internet of things: A survey. IEEE Commun Surv Tutor 16(1):414–454

    Article  Google Scholar 

  21. Puiu D, Barnaghi PM, Toenjes R, Kuemper D, Ali MI, Mileo A, Parreira JX, Fischer M, Kolozali S, Farajidavar N, Gao F, Iggena T, Pham T, Nechifor C, Puschmann D, Fernandes J (2016) CityPulse: Large scale data analytics framework for smart cities. IEEE Access 4:1086–1108

    Article  Google Scholar 

  22. Rajkumar R, Lee I, Sha L, Stankovic J (2010) Cyber-Physical Systems: The Next Computing Revolution. In: 47th Design Automation Conference. IEEE, pp 731–736

  23. Renart E, Diaz-Montes J, Parahsar M (2017) Data-driven Stream Processing at the Edge. In: IEEE 1st International Conference on Fog and Edge Computing. IEEE, pp 31–40

  24. Sajjad HP, Danniswara K, Al-Shishtawy A, Vlassov V (2016) SpanEdge: Towards Unifying Stream Processing over Central and Near-the-Edge Data Centers. In: IEEE/ACM Symposium on Edge Computing. IEEE, pp 168–178

  25. Stojmenovic I, Wen S (2014) The Fog Computing Paradigm: Scenarios and Security Issues. In: 2014 Federated Conference on Computer Science and Information Systems. IEEE, pp 1–8

  26. Yang S (2017) IoT stream processing and analytics in the fog. IEEE Commun Mag 55:21–27

    Article  Google Scholar 

  27. Yassine A, Singh S, Hossain MS, Muhammad G (2019) IoT big data analytics for smart homes with fog and cloud computing. Future Gener Comp Sy 91:563–573

    Article  Google Scholar 

  28. Yi S, Li C, Li Q (2015) A Survey of Fog Computing: Concepts, Applications and Issues. In: Workshop on Mobile Big Data. ACM, pp 37–42

Download references

Funding

Open access funding provided by TU Wien (TUW).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Stefan Schulte.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hießl, T., Hochreiner, C. & Schulte, S. Towards a Framework for Data Stream Processing in the Fog. Informatik Spektrum 42, 256–265 (2019). https://doi.org/10.1007/s00287-019-01192-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00287-019-01192-z