Skip to main content
SpringerLink
Log in

On Data Stream Processing in IoT Applications

  • Conference paper
  • First Online:
Internet of Things, Smart Spaces, and Next Generation Networks and Systems (NEW2AN 2018, ruSMART 2018)

Part of the book series: Lecture Notes in Computer Science ((LNCCN,volume 11118))

Abstract

This article is devoted to the issues of streaming data processing in the Internet of Things applications. Stream processing is a natural fit for the Internet of Things applications. Most of the data models on the Internet Things are exactly the data streams. Accordingly, most applications (business applications) are oriented to processing real-time data streams (e.g., search for anomalies, provide billing features, etc.). The paper considers the architecture of data processing systems, classifies stream processing patterns. Much attention is paid to time management in stream processing systems. The review is conducted from the standpoint of the contents of the master’s course on stream data processing in the Internet of Things and Industrial Internet of Things applications. Also, the paper considers the specific application models and streaming data architecture for the Internet of Things applications as well basic data analysis algorithms that are used in such systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Garofalakis, M., Gehrke, J., Rastogi, R. (eds.): Data Stream Management: Processing High-Speed Data Streams. Springer, Heidelberg (2016)

    Google Scholar 

  2. EU FP7 CityPulse. http://www.ict-citypulse.eu. Accessed 24 May 2018

  3. Tönjes, R., et al.: Real time iot stream processing and large-scale data analytics for smart city applications. Poster session, European Conference on Networks and Communications (2014)

    Google Scholar 

  4. Namiot, D., Ventspils, M.S.S., Daradkeh, Y.I.: On Internet of Things education. In: 2017 20th Conference of Open Innovations Association (FRUCT), pp. 309–315. IEEE, April 2017

    Google Scholar 

  5. Rose, D.: Enchanted Objects: Design, Human Desire, and the Internet of Things. Simon and Schuster, New York (2014)

    Google Scholar 

  6. Namiot, D., Sneps-Sneppe, M.: On Internet of Things and big data in university courses. Int. J. Embed. Real-Time Commun. Syst. (IJERTCS) 8(1), 18–30 (2017)

    Article  Google Scholar 

  7. Namiot, D., Sneps-Sneppe, M.: On data persistence models for mobile crowdsensing applications. In: Kalinichenko, L., Kuznetsov, S., Manolopoulos, Y. (eds.) Data Analytics and Management in Data Intensive Domains. DAMDID/RCDL 2016. Communications in Computer and Information Science, vol. 706, pp. 192–204. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57135-5_14

    Google Scholar 

  8. Lambda architecture. http://lambda-architecture.net. Accessed 24 May 2018

  9. Kappa Architecture. http://milinda.pathirage.org/kappa-architecture.com. Accessed 24 May 2018

  10. Questioning the Lambda Architecture. https://www.oreilly.com/ideas/questioning-the-lambda-architecture. Accessed 24 May 2018

  11. Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache flink: stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 36(4), 28–38 (2015)

    Google Scholar 

  12. Akidau, T., et al.: The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proc. VLDB Endowment 8(12), 1792–1803 (2015)

    Article  Google Scholar 

  13. The world beyond batch: Streaming 101. https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101. Accessed 24 May 2018

  14. Zaharia, M., et al.: Fast and interactive analytics over Hadoop data with Spark. USENIX Login 37(4), 45–51 (2012)

    MathSciNet  Google Scholar 

  15. Apache Flink Windows. https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/windows.html. Accessed May 2018

  16. Sneps-Sneppe, M., Namiot, D.: About M2M standards and their possible extensions. In: 2012 2nd Baltic Congress on Future Internet Communications (BCFIC), pp. 187–193. IEEE, April 2012

    Google Scholar 

  17. Namiot, D.: On big data stream processing. Int. J. Open Inf. Technol. 3(8), 48–51 (2015)

    Google Scholar 

  18. Golab, L., Özsu, M.T.: Issues in data stream management. ACM SIGMOD Rec. 32(2), 5–14 (2003)

    Article  Google Scholar 

  19. Motwani, R., et al.: Query processing, resource management, and approximation in a data stream management system. In: CIDR, January 2003

    Google Scholar 

  20. Gama, J., Gaber, M.M. (eds.): Learning from Data Streams: Processing Techniques in Sensor Networks. Springer, Heidelberg (2007)

    Google Scholar 

  21. Kreps, J., Narkhede, N., Rao, J.: Kafka: a distributed messaging system for log processing. In: Proceedings of the NetDB, pp. 1–7, June 2011

    Google Scholar 

  22. Gartner says the Internet of Things will transform the data center. http://www.gartner.com/newsroom/id/2684616. Accessed 24 May 2018

  23. Standardization Activities of oneM2M. https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201408gls.html. Accessed 24 May 2018

  24. Gaber, M.M., Zaslavsky, A., Krishnaswamy, S.: Mining data streams: a review. ACM SIGMOD Rec. 34(2), 18–26 (2005)

    Article  Google Scholar 

  25. Rivetti, N., Busnel, Y., Querzoni, L.: Load-aware shedding in stream processing systems. In: Proceedings of the 10th ACM International Conference on Distributed and Event-Based Systems, pp. 61–68. ACM, June 2016

    Google Scholar 

  26. Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 1–16. ACM, June 2002

    Google Scholar 

  27. Larsen, K.G., Nelson, J., Nguyên, H.L., Thorup, M.: Heavy hitters via cluster-preserving clustering. In: 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), pp. 61–70. IEEE, October 2016

    Google Scholar 

  28. Papadimitriou, S., Sun, J., Faloutsos, C.: Streaming pattern discovery in multiple time-series. In: Proceedings of the 31st International Conference on Very Large Data Bases, pp. 697–708. VLDB Endowment, August 2005

    Google Scholar 

  29. Big Data Processing with Apache Spark - Part 3: Spark Streaming. https://www.infoq.com/articles/apache-spark-streaming. Accessed 24 May 2018

  30. Hirzel, M., et al.: IBM streams processing language: analyzing big data in motion. IBM J. Res. Dev. 57(3/4), 7:1–7:11 (2013)

    Article  Google Scholar 

Download references

Acknowledgement

We would like to thank the reviewers of the EUCNC conference for critical comments on the first versions of this work. Also we are grateful to the employees of the Laboratory of Open Information Technologies of the Lomonosov Moscow State University and Professor V.A. Sukhomlin for valuable discussions.

Author information

Authors and Affiliations

  1. Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, Moscow, Russia

    Dmitry Namiot

  2. Ventspils International Radio Astronomy Centre, Ventspils University College, Ventspils, Latvia

    Manfred Sneps-Sneppe & Romass Pauliks

Authors
  1. Dmitry Namiot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manfred Sneps-Sneppe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Romass Pauliks
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dmitry Namiot .

Editor information

Editors and Affiliations

  1. Tampere University of Technology, Tampere, Finland

    Olga Galinina

  2. Tampere University of Technology, Tampere, Finland

    Sergey Andreev

  3. FRUCT Oy, Helsinki, Finland

    Sergey Balandin

  4. Tampere University of Technology, Tampere, Finland

    Yevgeni Koucheryavy

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Namiot, D., Sneps-Sneppe, M., Pauliks, R. (2018). On Data Stream Processing in IoT Applications. In: Galinina, O., Andreev, S., Balandin, S., Koucheryavy, Y. (eds) Internet of Things, Smart Spaces, and Next Generation Networks and Systems. NEW2AN ruSMART 2018 2018. Lecture Notes in Computer Science(), vol 11118. Springer, Cham. https://doi.org/10.1007/978-3-030-01168-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01168-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01167-3

  • Online ISBN: 978-3-030-01168-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation