Skip to main content

Implementing a Volunteer Notification System into a Scalable, Analytical Realtime Data Processing Environment

  • Chapter
  • First Online:
Automation, Communication and Cybernetics in Science and Engineering 2015/2016
  • 1493 Accesses

Abstract

The pace at which next-generation Internet of Things networks, consisting of wirelessly distributed sensors and devices, are being developed is speeding up. More and more devices produce data in automated manners and the demand of smartphones and wearable devices is continuously increasing. With respect to volunteer notification systems (VNS), the resulting vast amounts of data can be utilized for profiling and predicting the whereabouts of people that, combined with machine learning algorithms, complement artificial intelligence (AI)-based decision systems. Hence, VNS benefit from keeping pace with the current developments by using the corresponding data streams in order to improve decision making during the volunteer selection process. In emergency scenarios, the velocity, low latency and reaction times of the system are essential, which results in the need of online stream-processing and real-time computational solutions. This paper will focus on a basic concept for implementing a VNS approach into a scalable, fault-tolerant environment that uses state-of-the-art analytical tools to process information streams in real-time as well as on demand, and applies machine learning algorithms for an AI-based volunteer selection. This work concentrates on leveraging open source Big Data technologies with the aim to deliver a robust, secure and highly available enterprise-class Big Data platform. Within the given context, this work will furthermore give an insight on state-of-the-art proprietary solutions for Big Data processing that are currently available.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. T. Danova. The Internet of Everything. http://uk.businessinsider.com/the-internet-of-everything-2014-slide-deck-sai-2014-2. October 8, 2014

  2. M. Stonebraker, U. Çetintemel, S. Zdonik, The 8 Requirements of Real-time Stream Processing. ACM SIGMOD Record 34 (4), 2005, pp. 42–47

    Article  Google Scholar 

  3. J. Elsner, M.T. Schneiders, M. Haberstroh, D. Schilberg, S. Jeschke, An Introduction to a Transnational Volunteer Notification System Providing Cardiopulmonary Resuscitation for Victims Suffering a Sudden Cardiac Arrest. In: eTelemed2013. 2013, pp. 59–64

    Google Scholar 

  4. J. Elsner, P. Meisen, S. Thelen, D. Schilberg, S. Jeschke, EMuRgency – A Basic Concept for an AI Driven Volunteer Notification System for Integrating Laypersons into Emergency Medical Services. International Journal On Advances in Life Sciences 5 (3 and 4), 2013, pp. 223–236

    Google Scholar 

  5. J. Elsner, M.T. Schneiders, D. Schilberg, S. Jeschke, Determination of the Relevant First Aiders within a Volunteer Notification System. In: MedTel 2013, Luxembourg. 2013, pp. 245–249

    Google Scholar 

  6. C.C. Aggarwal, ed., Data Streams: Models and Algorithms, Advances in Database Systems, vol. 31. Kluwer, 2007

    Google Scholar 

  7. J. Taylor, Real-Time Responses with Big Data. 2014

    Google Scholar 

  8. Apache Storm. http://storm.apache.org/. October 12, 2014

  9. Apache Spark. https://spark.apache.org/. October 14, 2014

  10. Apache Samza. http://samza.apache.org/. October 14, 2014

  11. Amazon Kinesis. http://aws.amazon.com/de/kinesis/. October 14, 2014

  12. Google BigQuery. https://cloud.google.com/bigquery/. October 14, 2014

  13. Apache Kafka. http://kafka.apache.org/. October 15, 2014

  14. Amazon Webservices such as SNS or SQS. http://aws.amazon.com/de/sns/. October 15, 2014

  15. Apache Hadoop. http://hadoop.apache.org/. October 14, 2014

  16. J. Polo, Big Data Processing with MapReduce. In: Big Data Computing, 2013, pp. 295–313

    Google Scholar 

  17. M. Stonebraker, D. Abadi, D.J. DeWitt, S. Madden, E. Paulson, A. Pavlo, A. Rasin, MapReduce and Parallel DBMSs: Friends or Foes? Communications of the ACM 53 (1), 2010, pp. 64–71

    Article  Google Scholar 

  18. R. Lämmel, Google’s MapReduce programming model – Revisited. Science of Computer Programming 70 (1), 2008, pp. 1–30

    Article  MathSciNet  MATH  Google Scholar 

  19. V.K. Vavilapalli, A.C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth, B. Saha, C. Curino, O. O’Malley, S. Radia, B. Reed, E. Baldeschwieler, Apache Hadoop YARN: Yet Another Resource Negotiator. In: Proceedings of the 4th Annual Symposium on Cloud Computing. ACM, New York, NY, USA, 2013, SOCC ’13, pp. 1–16

    Google Scholar 

  20. M. Zaharia, M. Chowdhury, M.J. Franklin, S. Shenker, I. Stoica, Spark: Cluster Computing with Working Sets. In: Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing. USENIX Association, Berkeley, CA, USA, 2010, HotCloud’10, pp. 10–10

    Google Scholar 

  21. Apache Cassandra. http://cassandra.apache.org/. October 15, 2014

  22. Amazon Redshift. http://aws.amazon.com/de/redshift/. October 15, 2014

  23. Amazon DynamoDB. http://aws.amazon.com/de/dynamodb/. October 15, 2014

  24. Datastax Corporation, Comparing the Hadoop File System (HDFS) with the Cassandra File System (CFS). Tech. rep., 2013. http://www.datastax.com/resources/whitepapers/hdfs-vs-cfs. April 16, 2015

  25. I. Fette, A. Melnikov, The WebSocket Protocol. IETF, 2011

    Google Scholar 

  26. W. Reese, Nginx: the high-performance web server and reverse proxy. Linux Journal 173, 2008

    Google Scholar 

  27. R. Fielding, U. Irvine, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee, Hypertext transfer protocol – HTTP/1.1. RFC Editor, 1999

    Google Scholar 

  28. C. Le, X. Yang, Research of applying MVC pattern in distributed environment. Computer Engineering 32 (19), 2006, pp. 62–64

    Article  Google Scholar 

  29. Nginx+. http://nginx.com/products/technical-specs/. October 16, 2014

  30. RabbitMQ. http://www.rabbitmq.com/. October 16, 2014

  31. ActiveMQ. http://activemq.apache.org/. October 16, 2014

  32. HiveMQ. http://www.hivemq.com/. October 16, 2014

  33. CloudAMQP. https://www.cloudamqp.com/. October 17, 2014

  34. CloudMQTT. http://www.cloudmqtt.com/. October 17, 2014

  35. A. Piórkowski, A. Kempny, A. Hajduk, J. Strzelczyk, Load Balancing for Heterogeneous Web Servers. In: Computer Networks, Springer, 2010, pp. 189–198

    Google Scholar 

  36. Real time Analytics with Apache Kafka and Apache Spark. http://de.slideshare.net/rahuldausa/real-time-analytics-with-apache-kafka-and-apache-spark. October 17, 2014

  37. M. Zaharia, T. Das, H. Li, T. Hunter, S. Shenker, I. Stoica, Discretized Streams: Fault-tolerant Streaming Computation at Scale. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. New York, NY, USA, 2013, SOSP ’13, pp. 423–438

    Google Scholar 

  38. Samza / Kafka Security. http://bit.ly/1pl550b. October 17, 2014

  39. J. Kreps, N. Narkhede, J. Rao, Kafka: A distributed messaging system for log processing. NetDB, 2011

    Google Scholar 

  40. R. Ranjan, Streaming Big Data Processing in Datacenter Clouds. IEEE Cloud Computing 1 (1), 2014, pp. 78–83

    Article  MathSciNet  Google Scholar 

  41. J. Zollmann, NoSQL Databases. Proceedings of the NetDB, 2011. http://sewiki.iai.uni-bonn.de/_media/teaching/labs/xp/2012b/seminar/10-nosql.pdf

  42. J. Elsner, An AI Driven Volunteer Selection System. Ph.D. thesis, Aachen, 2015

    Google Scholar 

  43. MQTT Protocol 3.1.1. Spec. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html. February 10, 2014

  44. Google Cloud Datastore. https://cloud.google.com/datastore/. October 15, 2014

  45. E. Wilde, Putting Things to REST. School of Information, 2007. Series: Recent Work

    Google Scholar 

Download references

Acknowledgments

This paper is based on work done in the INTERREG IVa project EMuRgency (www.emurgency.eu). The project is partially financed through the European Regional Development Fund (ERDF) and co-financed by several regions and partners of the EMuRgency consortium.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jesko Elsner .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Elsner, J., Sivicki, T., Meisen, P., Meisen, T., Jeschke, S. (2016). Implementing a Volunteer Notification System into a Scalable, Analytical Realtime Data Processing Environment. In: Jeschke, S., Isenhardt, I., Hees, F., Henning, K. (eds) Automation, Communication and Cybernetics in Science and Engineering 2015/2016. Springer, Cham. https://doi.org/10.1007/978-3-319-42620-4_64

Download citation

Publish with us

Policies and ethics