Skip to main content
SpringerLink
Log in

Multi-modal traffic event detection using shapelets

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Traffic management continues to be one of the most critical challenges facing smart cities. Timely detection of incidents plays an important role in reducing fatality rates, avoiding congestion and improving traffic conditions. Currently, traditional traffic event detection approaches often rely on one source of data, such as road sensor readings or social media posts. However, there is a need for new approaches that can combine these channels and benefit from the diversity of the collected data for better event detection performance. This paper presents a new framework for near real-time event detection based on the fusion of sensor readings and social media data. The shapelets technique, used for sensor readings, generates sub-sequences of the time series representing distinctive patterns. Each pattern is called a shapelet and is selected based on the maximal differentiation achieved between the different classes of a time series set. In traffic events, shapelets can represent patterns of incidents/congestion as well as normal traffic situations that the framework utilizes to detect the occurrence of events. Similarly, social media posts can be featured as shapelets to enable the combination of both media channels creating a multi-modal solution. In the proposed framework, two pipelines are defined : sensor readings detection pipeline and social media detection pipeline. In addition, two multi-modal fusion techniques based on Shapelet Transform are suggested and compared, namely early fusion and late fusion. They help in combining the two pipelines either at the data level or at the decision level. Validation using the M25 London Circular road data shows that early fusion of both sources has better detection rate and better performance over late fusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Williams BM, Guin A (2007) Traffic management center use of incident detection algorithms: Findings of a nationwide survey. IEEE Trans Intell Trans Syst 8(2):351–358

    Article  Google Scholar 

  2.  Sakaki T, Okazaki M, Matsuo Y (2010) Earthquake shakes Twitter users: real-time event detection by social sensors. In: Proceedings of the 19th international conference on World wide web, ACM, 851–860

  3. Sakaki T, Matsuo Y, Yanagihara T, Chandrasiri NP, Nawa K (2012) Real-time event extraction for driving information from social sensors. In: IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2012, IEEE, 221–226

  4. Atefeh F, Khreich W (2015) A survey of techniques for event detection in twitter. Computat Intell 31(1):132–164

    Article  MathSciNet  Google Scholar 

  5. Wang D, Al-Rubaie A, Davies J, Clarke SS (2014) Real time road traffic monitoring alert based on incremental learning from tweets. In: (2014) IEEE Symposium on Evolving and Autonomous Learning Systems (EALS). IEEE 50–57

  6. Anantharam P, Barnaghi P, Thirunarayan K, Sheth A (2015) Extracting city traffic events from social streams. ACM Trans Intell Syst Technol (TIST) 6(4):43–48

    Google Scholar 

  7. Gutiérrez C, Figuerias P, Oliveira P, Costa R, Jardim-Goncalves R (2015) Twitter mining for traffic events detection. In: Science and Information Conference (SAI), 2015, IEEE, 371–378

  8. Schulz A, Ristoski P, Paulheim H (2013) I see a car crash: Real-time detection of small scale incidents in microblogs. In: Extended Semantic Web Conference, Springer, 22–33

  9. Cui J, Fu R, Dong C, Zhang Z (2014) Extraction of traffic information from social media interactions: Methods and experiments, in: 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), IEEE, 1549–1554

  10. LiuM, Fu K, Lu CT, Chen G, Wang H (2014) A search and summary application for traffic events detection based on twitter data. In: Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, 549–552

  11. Mai E, Hranac R (2013) Twitter interactions as a data source for transportation incidents. Tech Rep 9:12–35

    Google Scholar 

  12. Gu Y, Qian ZS, Chen F (2016) From Twitter to detector: Real-time traffic incident detection using social media data. Trans Res Part C: Emerg Technol 67:321–342

    Article  Google Scholar 

  13. D’Andrea E, Ducange P, Lazzerini B, Marcelloni F (2015) Real-time detection of traffic from twitter stream analysis. IEEE Trans Intell Trans Syst 16(4):2269–2283

    Article  Google Scholar 

  14. Deniz O, Celikoglu HB (2011) Overview to some existing incident detection algorithms: a comparative evaluation. Proced Soc Behav Sci 20(01):1–13

    Google Scholar 

  15. Parkany E, Xie C (2005) A complete review of incident detection algorithms & their deployment: what works and what doesn’t. Tech Rep 6(1):23–89

    Google Scholar 

  16. Adetiloye TO (2018) Predicting Short-Term Traffic Congestion on Urban Motorway Networks, Ph.D. thesis, Concordia University,

  17. Zhang N, Shi Y, Huang W (2012) Traffic Event Automatic Detection Based on OGS-DTW Algorithm. J Highway Trans Res Develop (Engl Ed) 6(1):54–60

    Article  Google Scholar 

  18. Motamed M et al. (2016) Developing a real-time freeway incident detection model using machine learning techniques, Ph.D. thesis

  19. AlDhanhani A, Damiani E, Mizouni R, Wang D (2019) Framework for traffic event detection using Shapelet Transform. Eng Appl Artif Intell 82:226–235

    Article  Google Scholar 

  20. Foo PH, Ng GW (2013) High-level Information Fusion: An Overview. J Adv Inf Fusion 8(1):33–72

    Google Scholar 

  21. Wang Y, Kankanhalli MS (2015) Tweeting cameras for event detection. In: Proceedings of the 24th International Conference on World Wide Web, ACM, 1231–1241

  22. Musaev A, Wang D, Cho CA, Pu C (2014) Landslide detection service based on composition of physical and social information services. In: Web Services (ICWS), 2014 IEEE International Conference on, IEEE, 97–104

  23. Soni U (2019) Integration of traffic data from social media and physical sensors for near real time road traffic analysis, Master’s thesis, University of Twente

  24. Moumtzidou A, Giannakeris P, Andreadis S, Mavropoulos A, Meditskos G, Gialampoukidis I, Avgerinakis K, Vrochidis S, Kompatsiaris I (2018) A Multimodal Approach in Estimating Road Passability Through a Flooded Area Using Social Media and Satellite Images. In: MediaEval

  25. Gialampoukidis I, Andreadis S, Vrochidis S, Kompatsiaris I (2021) Fusion Multimodal Data, of Social Media and Satellite Images for Emergency Response and Decision-Making. In: (2021) IEEE International Geoscience and Remote Sensing Symposium IGARSS. IEEE 228–231

  26. Mantsis DF, Bakratsas M, Andreadis S, Karsisto P, Moumtzidou A, Gialampoukidis I, Karppinen A, Vrochidis S, Kompatsiaris I (2020) Multimodal fusion of sentinel 1 images and social media data for snow depth estimation. IEEE Geosci Remote Sens Lett 63:97–111

    Google Scholar 

  27. Choi JG, Kong CW, Kim G, Lim S (2021) Car crash detection using ensemble deep learning and multimodal data from dashboard cameras. Expert Syst Appl 183:115400

    Article  Google Scholar 

  28. Cerqueira S, Arsenio E, Henriques R (2021) On how to incorporate public sources of situational context in descriptive and predictive models of traffic data. Europ Trans Res Rev 13(1):1–22

    Article  Google Scholar 

  29. Lemonde C, Arsenio E, Henriques R (2021) Integrative analysis of multimodal traffic data: addressing open challenges using big data analytics in the city of Lisbon. Europ Transt Res Rev 13(1):1–22

    Google Scholar 

  30. Tan H, Bansal M, Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprintarXiv:1908.07490

  31. Lu J, Batra D, Parikh D, Lee S (2019) Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Adv Neul Informat Process Syst 64:13–23

    Google Scholar 

  32. Li LH, Yatskar M, Yin D, Hsieh CJ, Chang KW, Visualbert: A simple and performant baseline for vision and language, arXiv preprint arXiv:1908.03557

  33. Hills J, Lines J, Baranauskas E, Mapp J, Bagnall A (2014) Classification of time series by shapelet transformation. Data Min Knowled Disc 28(4):851–881

    Article  MathSciNet  MATH  Google Scholar 

  34. AlDhanhani A, Damiani E, Mizouni R, Wang D (2018) Analysis of Shapelet Transform Usage in Traffic Event Detection. In: 2018 IEEE International Conference on Cognitive Computing (ICCC), IEEE, 41–48

  35. Bostrom A, Bagnall A, Lines J, Evaluating Improvements to the Shapelet Transform, www-bcf. usc. edu

  36. Bostrom A (2018) Shapelet Transforms for Univariate and Multivariate Time Series Classification, Ph.D. thesis, University of East Anglia

  37. Speer R, Chin J, Havasi C (2017) ConceptNet 5.5: an Open Multilingual Graph of General Knowledge, http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14972

  38. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: A nucleus for a web of open data. The semantic web. Springer, London, pp 722–735

    Chapter  Google Scholar 

  39. P. University About WordNet. (2019) https://wordnet.princeton.edu,

  40. Suliman AT, Al Kaabi K, Wang D, Al-Rubaie A, Al Dhanhani A, Ruta D, Davies J, Clarke SS (2016) Event identification and assertion from social media using auto-extendable knowledge base, in: International Joint Conference on Neural Networks (IJCNN), 2016, IEEE. pp 4443–4450

Download references

Author information

Authors and Affiliations

  1. Emirates ICT Innovation Center EBTIC, Khalifa University, Abu Dhabi, UAE

    Ahmed AlDhanhani, Ernesto Damiani, Di Wang & Ahmad Al-Rubaie

  2. Center on Cyber-Physical Systems, Khalifa University, Abu Dhabi, UAE

    Ernesto Damiani & Rabeb Mizouni

  3. Electrical Engineering and Computer Science Department, Khalifa University, Abu Dhabi, UAE

    Rabeb Mizouni

Authors
  1. Ahmed AlDhanhani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ernesto Damiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rabeb Mizouni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Di Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ahmad Al-Rubaie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rabeb Mizouni.

Ethics declarations

Conflict of Interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Data Access

The datasets generated and analysed during the current study are available in the multi-modal traffic event detection repository, https://github.com/eabayed/multimodal_traffic_event_detection.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

AlDhanhani, A., Damiani, E., Mizouni, R. et al. Multi-modal traffic event detection using shapelets. Neural Comput & Applic 35, 1395–1408 (2023). https://doi.org/10.1007/s00521-022-07837-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07837-7

Keywords

Search

Navigation