Mobile Distributed Complex Event Processing—Ubi Sumus? Quo Vadimus?

  • Fabrice Starks
  • Vera Goebel
  • Stein Kristiansen
  • Thomas Plagemann
Chapter
First Online:
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 10)

Abstract

One important class of applications for the Internet of Things is related to the need to gain timely and continuous situational awareness, like smart cities, automated traffic control, or emergency and rescue operations. Events happening in the real-world need to be detected in real-time based on sensor data and other data sources. Complex Event Processing (CEP) is a technology to detect complex (or composite) events in data streams and has been successfully applied in high volume and high velocity applications like stock market analysis. However, these application domains faced only the challenge of high performance, while the Internet of Things and Mobile Big Data introduce a new set of challenges caused by mobility. This chapter aims to explain these challenges and give an overview on how they are solved respectively how far state-of-the-art research has advanced to be useful to solve Mobile Big Data problems. At the infrastructure level the main challenge is to trade performance against resource consumption; and operator placement is the most dominant mechanism to address these problems. At the application and consumer level, mobile queries pose a new set of challenges for CEP. These are related to continuously changing positions of consumers and data sources, and the need to adapt the query processing to these changes. Finally, proper methods and tools for systematical testing and reproducible performance evaluation for mobile distributed CEP are needed but not yet available.

This is a preview of subscription content, log in to check access.

References

  1. 1.
    Golab, L., Özsu, M.T.: Issues in data stream management. SIGMOD Rec. 32(2), 5–14 (2003). https://doi.org/10.1145/776985.776986
  2. 2.
    http://sqlstream.com/intro. Accessed 30 Dec 2016
  3. 3.
    Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., Widom, J.: Stream: The stanford data stream management system. Technical Report 2004-20, Stanford InfoLab (2004). http://ilpubs.stanford.edu:8090/641/
  4. 4.
    Abadi, D.J., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., Zdonik, S.: Aurora: a new model and architecture for data stream management. VLDB J. 12(2), 120–139 (2003). https://doi.org/10.1007/s00778-003-0095-z
  5. 5.
    Stegmaier, B., Kuntschke, R., Kemper, A.: Streamglobe: adaptive query processing and optimization in streaming p2p environments. In: Proceeedings of the 1st International Workshop on Data Management for Sensor Networks: In Conjunction with VLDB 2004, DMSN ’04, pp. 88–97. ACM, New York, NY, USA (2004). https://doi.org/10.1145/1052199.1052214
  6. 6.
    http://www.espertech.com/esper/. Accessed 30 Dec 2016
  7. 7.
    Kazemitabar, S.J., Demiryurek, U., Ali, M., Akdogan, A., Shahabi, C.: Geospatial stream query processing using microsoft sql server streaminsight. Proc. VLDB Endow. 3(1–2), 1537–1540 (2010). https://doi.org/10.14778/1920841.1921032
  8. 8.
    http://evam.com/platform/. Accessed 30 Dec 2016
  9. 9.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proceedings of the Twenty-first ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS ’02, pp. 1–16. ACM, New York, NY, USA (2002). https://doi.org/10.1145/543613.543615
  10. 10.
    Luckham, D.C.: The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley Longman Publishing Co., Inc, Boston, MA, USA (2001)Google Scholar
  11. 11.
    Cugola, G., Margara, A.: Processing flows of information: from data stream to complex event processing. ACM Comput. Surv. 44(3), 15:1–15:62 (2012). 10.1145/2187671.2187677. http://doi.org/10.1145/2187671.2187677
  12. 12.
    Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.M.: The many faces of publish/subscribe. ACM Comput. Surv. 35(2), 114–131 (2003). https://doi.org/10.1145/857076.857078
  13. 13.
    Etzion, O., Niblett, P.: Event Processing in Action, 1st edn. Manning Publications Co., Greenwich, CT, USA (2010)Google Scholar
  14. 14.
    Koldehofe, B., Ottenwälder, B., Rothermel, K., Ramachandran, U.: Moving range queries in distributed complex event processing. In: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, DEBS ’12, pp. 201–212. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2335484.2335507
  15. 15.
    Zhang, B., Mor, N., Kolb, J., Chan, D.S., Lutz, K., Allman, E., Wawrzynek, J., Lee, E., Kubiatowicz, J.: The cloud is not enough: saving iot from the cloud. In: 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15). USENIX Association, Santa Clara, CA (2015). https://www.usenix.org/conference/hotcloud15/workshop-program/presentation/zhang
  16. 16.
    Rizou, S., Durr, F., Rothermel, K.: Solving the multi-operator placement problem in large-scale operator networks. In: 2010 Proceedings of 19th International Conference on Computer Communications and Networks (ICCCN), pp. 1–6. IEEE (2010)Google Scholar
  17. 17.
    Rizou, S., Durr, F., Rothermel, K.: Providing qos guarantees in large-scale operator networks. In: 2010 12th IEEE International Conference on High Performance Computing and Communications (HPCC), pp. 337–345 (2010). https://doi.org/10.1109/HPCC.2010.53
  18. 18.
    Cardellini, V., Grassi, V., Lo Presti, F., Nardelli, M.: Optimal operator placement for distributed stream processing applications. In: Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems, DEBS ’16, pp. 69–80. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2933267.2933312
  19. 19.
    Lakshmanan, G.T., Li, Y., Strom, R.: Placement strategies for internet-scale data stream systems. IEEE Internet Comput. 12(6), 50–60 (2008). https://doi.org/10.1109/MIC.2008.129
  20. 20.
    Rizou, S., Diirr, F., Rothermel, K.: Fulfilling end-to-end latency constraints in large-scale streaming environments. In: 30th IEEE International Performance Computing and Communications Conference, pp. 1–8 (2011). https://doi.org/10.1109/PCCC.2011.6108086
  21. 21.
    Cipriani, N., Lbbe, C., Moosbrugger, A.: Exploiting constraints to build a flexible and extensible data stream processing middleware. In: 2010 IEEE International Symposium on Parallel Distributed Processing, Workshops and Ph.D. Forum (IPDPSW), pp. 1–8 (2010). https://doi.org/10.1109/IPDPSW.2010.5470847
  22. 22.
    Anastasi, G., Conti, M., Francesco, M.D., Passarella, A.: Energy conservation in wireless sensor networks: asurvey. Ad Hoc Netw. 7(3), 537–568 (2009). https://doi.org/10.1016/j.adhoc.2008.06.003, http://www.sciencedirect.com/science/article/pii/S1570870508000954
  23. 23.
    Lu, Z., Wen, Y., Fan, R., Tan, S.L., Biswas, J.: Toward efficient distributed algorithms for in-network binary operator tree placement in wireless sensor networks. IEEE J. Sel. Areas Commun. 31(4), 743–755 (2013)CrossRefGoogle Scholar
  24. 24.
    Bonfils, B.J., Bonnet, P.: Adaptive and decentralized operator placement for in-network query processing. Telecommun. Syst. 26(2–4), 389–409 (2004)CrossRefMATHGoogle Scholar
  25. 25.
    Chatzimilioudis, G., Cuzzocrea, A., Gunopulos, D., Mamoulis, N.: A novel distributed framework for optimizing query routing trees in wireless sensor networks via optimal operator placement. J. Comput. Syst. Sci. 79(3), 349–368 (2013)CrossRefMathSciNetMATHGoogle Scholar
  26. 26.
    Chatzimilioudis, G., Hakkoymaz, H., Mamoulis, N., Gunopulos, D.: Operator placement for snapshot multi-predicate queries in wireless sensor networks. In: 2009 Tenth International Conference on Mobile Data Management: Systems, Services and Middleware, pp. 21–30. IEEE (2009)Google Scholar
  27. 27.
    Chatzimilioudis, G., Mamoulis, N., Gunopulos, D.: A distributed technique for dynamic operator placement in wireless sensor networks. In: 2010 Eleventh International Conference on Mobile Data Management, pp. 167–176. IEEE (2010)Google Scholar
  28. 28.
    Ying, L., Liu, Z., Towsley, D., Xia, C.H.: Distributed operator placement and data caching in large-scale sensor networks. In: INFOCOM 2008. The 27th Conference on Computer Communications. IEEE (2008)Google Scholar
  29. 29.
    Ottenwälder, B., Koldehofe, B., Rothermel, K., Ramachandran, U.: Migcep: operator migration for mobility driven distributed complex event processing. In: Proceedings of the 7th ACM International Conference on Distributed Event-based Systems, pp. 183–194. ACM (2013)Google Scholar
  30. 30.
    Pietzuch, P., Ledlie, J., Shneidman, J., Roussopoulos, M., Welsh, M., Seltzer, M.: Network-aware operator placement for stream-processing systems. In: 22nd International Conference on Data Engineering (ICDE’06), pp. 49–49. IEEE (2006)Google Scholar
  31. 31.
    Lu, Z., Wen, Y.: Distributed and asynchronous solution to operator placement in large wireless sensor networks. In: Proceedings of the 2012 8th International Conference on Mobile Ad-hoc and Sensor Networks, MSN ’12, pp. 100–107. IEEE Computer Society, Washington, DC, USA (2012). https://doi.org/10.1109/MSN.2012.23
  32. 32.
    Starks, F., Plagemann, T.P.: Operator placement for efficient distributed complex event processing in manets. In: 2015 IEEE 11th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 83–90 (2015). http://doi.org/10.1109/WiMOB.2015.7347944
  33. 33.
    Jain, N., Biswas, R., Nandiraju, N., Agrawal, D.P.: Energy aware routing for spatio-temporal queries in sensor networks. In: IEEE Wireless Communications and Networking Conference, 2005, vol. 3, pp. 1860–1866 (2005). https://doi.org/10.1109/WCNC.2005.1424795
  34. 34.
    Pathak, A., Prasanna, V.K.: Energy-efficient task mapping for data-driven sensor network macroprogramming. IEEE Trans. Comput. 59(7), 955–968 (2010). https://doi.org/10.1109/TC.2009.168
  35. 35.
    Srivastava, U., Munagala, K., Widom, J.: Operator placement for in-network stream query processing. In: Proceedings of the Twenty-fourth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS ’05, pp. 250–258. ACM, New York, NY, USA (2005). https://doi.org/10.1145/1065167.1065199
  36. 36.
    Drugan, O., Plagemann, T., Munthe-Kaas, E.: Dynamic clustering in sparse MANETs. Computer Communications 59, 84–97 (2015). https://doi.org/10.1016/j.comcom.2014.12.005, http://www.sciencedirect.com/science/article/pii/S0140366414003703
  37. 37.
    Abrams, Z., Liu, J.: Greedy is good: on service tree placement for in-network stream processing. In: 26th IEEE International Conference on Distributed Computing Systems (ICDCS’06), pp. 72–72 (2006). https://doi.org/10.1109/ICDCS.2006.45
  38. 38.
    Oikonomou, K., Stavrakakis, I., Xydias, A.: Scalable service migration in general topologies. In: Proceedings of the 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks, WOWMOM ’08, pp. 1–6. IEEE Computer Society, Washington, DC, USA (2008). https://doi.org/10.1109/WOWMOM.2008.4594891
  39. 39.
    Ilarri, S., Mena, E., Illarramendi, A.: Location-dependent query processing: where we are and where we are heading. ACM Comput. Surv. 42(3), 12:1–12:73 (2010). https://doi.org/10.1145/1670679.1670682
  40. 40.
    Mokbel, M.F., Aref, W.G.: Sole: scalable on-line execution of continuous queries on spatio-temporal data streams. VLDB J. 17(5), 971–995 (2008). https://doi.org/10.1007/s00778-007-0046-1
  41. 41.
    Xiong, X., Elmongui, H.G., Chai, X., Aref, W.G.: Place: a distributed spatio-temporal data stream management system for moving objects. In: 2007 International Conference on Mobile Data Management, pp. 44–51 (2007). https://doi.org/10.1109/MDM.2007.16
  42. 42.
    Hong, K., Lillethun, D.J., Ramachandran, U., Ottenwälder, B., Koldehofe, B.: Opportunistic spatio-temporal event processing for mobile situation awareness. In: The 7th ACM International Conference on Distributed Event-Based Systems, DEBS ’13, Arlington, TX, USA—June 29–July 03, 2013, pp. 195–206 (2013). https://doi.org/10.1145/2488222.2488266
  43. 43.
    Ottenwälder, B., Koldehofe, B., Rothermel, K., Hong, K., Lillethun, D.J., Ramachandran, U.: MCEP: A mobility-aware complex event processing system. ACM Trans. Internet Techn. 14(1), 6:1–6:24 (2014). https://doi.org/10.1145/2633688
  44. 44.
    Ottenwälder, B., Koldehofe, B., Rothermel, K., Hong, K., Ramachandran, U.: RECEP: selection-based reuse for distributed complex event processing. In: The 8th ACM International Conference on Distributed Event-Based Systems, DEBS ’14, Mumbai, India, May 26–29, 2014, pp. 59–70 (2014). https://doi.org/10.1145/2611286.2611297
  45. 45.
    Jain, R.: The art of computer systems performance evaluation. In: Limoncelli, T., Hogan, C., Chaiup, S. (eds.), The Practice of System and Network Administration, vol. 3, pp. 978–032, lSBN-I. Wiley (1991)Google Scholar
  46. 46.
    Kakkad, V., Santosa, A.E., Scholz, B.: Migrating operator placement for compositional stream graphs. In: Proceedings of the 15th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, pp. 125–134. ACM (2012)Google Scholar
  47. 47.
    Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: Planetlab: an overlay testbed for broad-coverage services. ACM SIGCOMM Comput. Commun. Rev. 33(3), 3–12 (2003)CrossRefGoogle Scholar
  48. 48.
    Kačer, J.: Discrete event simulations with j-sim. In: Proceedings of the Inaugural Conference on the Principles and Practice of programming, 2002 and Proceedings of the Second Workshop on Intermediate Representation Engineering for Virtual Machines, pp. 13–18. National University of Ireland (2002)Google Scholar
  49. 49.
    Varga, A., et al.: The omnet++ discrete event simulation system. In: Proceedings of the European Simulation Multiconference (ESM2001), vol. 9, p. 65. sn (2001)Google Scholar
  50. 50.
    Montresor, A., Jelasity, M.: Peersim: a scalable p2p simulator. In: 2009 IEEE Ninth International Conference on Peer-to-Peer Computing, pp. 99–100. IEEE (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Fabrice Starks
    • 1
  • Vera Goebel
    • 1
  • Stein Kristiansen
    • 1
  • Thomas Plagemann
    • 1
  1. 1.University of OsloOsloNorway

Personalised recommendations