Skip to main content
SpringerLink
Log in

Model Free Identification of Traffic Conditions Using Unmanned Aerial Vehicles and Deep Learning

  • Original Paper
  • Published:
Journal of Big Data Analytics in Transportation Aims and scope Submit manuscript

Abstract

The purpose of this paper is to provide a methodological framework to identify traffic conditions based on non-calibrated video recordings captured from unmanned aerial vehicles (UAV) using deep learning. To this end, we propose two complementary to each other approaches: (i) identify in real time, with minimal computational cost, traffic conditions, (ii) localize, classify vehicles and approximate traffic variables (volume, speed, density) on a road segment from video captured by UAVs. Both problems are formulated as classification problems and tackled using Convolutional Neural Networks (CNN). The use of pre-trained CNNs is also investigated. Both approaches are, then, analysed based on their accuracy and feasibility in implementation. Findings indicate that all models developed achieve a detection accuracy of 89% and higher. The CNN with the best performance can classify traffic conditions between constrained and unconstrained traffic with 91% accuracy higher than what a pretrained model achieved and with significantly faster training times. Furthermore, findings indicated that pretrained neural network for traffic localization was able to predict the position and type of vehicles with a precision of 0.91. Based on the fundamental traffic diagram, it was shown that the two approaches provide compatible results and a feasible representation of traffic on the study area. Finally, possible applications in the field of transportation and traffic monitoring are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Ammour N, Alhichri H, Bazi Y, Benjdira B, Alajlan N, Zuair M (2017) Deep learning approach for car detection in UAV imagery. Remote Sens 9(4):312

    Article  Google Scholar 

  • Barmpounakis E, Geroliminis N (2020) On the new era of urban traffic monitoring with massive drone data: the pNEUMA large-scale field experiment. Transp Res Part C Emerg Technol 111:50–71

    Article  Google Scholar 

  • Barmpounakis E, Sauvin GM, Geroliminis N (2020) Lane detection and lane-changing identification with high-resolution data from a swarm of drones. Transp Res Rec J Transp Res Board 2674(7):1–15

    Article  Google Scholar 

  • Barmpounakis EN, Vlahogianni EI, Golias JC (2016) Unmanned Aerial Aircraft Systems for transportation engineering: Current practice and future challenges. Int J Transp Sci Technol 5(3):111–122

    Article  Google Scholar 

  • Barmpounakis EN, Vlahogianni EI, Golias JC, Babinec A (2017) How accurate are small drones for measuring microscopic traffic parameters? Transp Lett 11(6):1–9

    Google Scholar 

  • Barmpounakis E, Vlahogianni E, Golias J (2018) Identifying predictable patterns in the unconventional overtaking decisions of PTW for Cooperative ITS. IEEE Trans Intell Veh 3(1):102–111

    Article  Google Scholar 

  • Benjdira B, Khursheed T, Koubaa A, Ammar A, Ouni K (2019) Car detection using unmanned aerial vehicles: comparison between faster R-CNN and YOLOv3. In: 2019 1st international conference on unmanned vehicle systems-Oman (UVS), IEEE, February, pp 1–6

  • Boyer RS, Moore JS (1991). MJRTY—a fast majority vote algorithm. In Automated reasoning. Springer, Dordrecht, pp 105–117

  • Braut V, Culjak M, Vukotic V, Segvic S, Sevrovic M, Gold H (2012) Estimating OD Matrices at intersections in airborne video—a pilot study. MIPRO, 2012 Proceedings of the 35th International Convention, Opatija, Croatia, 21–25 May 2012, 977–982

  • Canziani A, Paszke A, Culurciello E (2016) An analysis of deep neural network models for practical applications. arXiv preprint.

  • Chakraborty P, Adu-Gyamfi YO, Poddar S, Ahsani V, Sharma A, Sarkar S (2018) Traffic congestion detection from camera images using deep convolution neural networks. Transp Res Rec 2672(45):222–231

    Article  Google Scholar 

  • Chen X, Xiang S, Liu CL, Pan CH (2014) Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geosci Remote Sens Lett 11(10):1797–1801

    Article  Google Scholar 

  • Coifman B, McCord M, Mishalani RG, Redmill K (2004) Surface transportation surveillance from unmanned aerial vehicles. Proceedings of the 83rd Annual Meeting of the Transportation Research Board. https://www2.ece.ohio-state.edu/~coifman/documents/UAV_paper.pdf

  • Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In Computer vision and pattern recognition, CVPR 2009. IEEE Conference on IEEE, pp 248–255

  • Gao H, Kong SL, Zhou S, Lv F, Chen Q (2014) Automatic extraction of multi-vehicle trajectory based on traffic videotaping from quadcopter model. Appl Mech Mater 552:232–239

    Article  Google Scholar 

  • Gkolias K, Vlahogianni EI (2018) Convolutional neural networks for on-street parking space detection in urban networks. IEEE Trans Intell Transp Syst 20(12):4318–4327

  • Google-Tensorflow (2019) github.com/tensorflow. Retrieved 2019, from TensorFlow-Slim image classification model library. https://github.com/tensorflow/models/tree/master/research/slim

  • Granlund G, Nordberg K, Wiklund J, Doherty P, Skarman E, Sandewall E (2000) WITAS: an intelligent autonomous aircraft using active vision. In: UAV 2000 international technical conference and exhibition. Euro UVS, Paris, France, June 2000

  • Gu X, Abdel-Aty M, Xiang Q, Cai Q, Yuan J (2019) Utilizing UAV video data for in-depth analysis of drivers’ crash risk at interchange merging areas. Accid Anal Prev 123:159–169

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Murphy K (2017) Speed/accuracy trade-offs for modern convolutional object detectors. Google Research

  • Huval B, Wang T, Tandon S, Kiske J, Song W, Pazhayampallil J et al (2015) An empirical evaluation of deep learning on highway driving. arxiv:1504.01716

  • Kaufmann S, Kerner BS, Hubert Rehborn MK, Klenov SL (2018) Aerial observations of moving synchronized flow patterns in over-saturated city traffic. Transp Res Part C Emerg Technol 86:393–406

    Article  Google Scholar 

  • Ke R, Li Z, Kim S, Ash J, Cui Z, Wang Y (2017) Real-time bidirectional traffic flow parameter estimation from aerial videos. IEEE Trans Intell Transp Syst 18(4):890–901

    Article  Google Scholar 

  • Ke R, Li Z, Tang J, Pan Z, Wang Y (2018) Real-time traffic flow parameter estimation from uav video based on ensemble classifier and optical flow. IEEE Trans Intell Transp Syst 99:1–11

    Google Scholar 

  • Khan MA, Ectorsa W, Bellemansa T, Ruichekb Y, Yasara A-H, Janssensa D, Wets G (2018a) Unmanned aerial vehicle-based traffic analysis: a case study to analyze traffic streams at urban roundabouts. Procedia Comput Sci 130:636–648

    Article  Google Scholar 

  • Khan M, Ectors W, Bellemans T, Janssens D, Wets G (2018b) Unmanned aerial vehicle-based traffic analysis: a case study for shockwave identification and flow parameters estimation at signalized intersections. Remote Sens 10(3):458

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

  • Kyrkou C, Plastiras G, Theocharides T, Venieris SI, Bouganis CS (2018) DroNet: efficient convolutional neural network detector for real-time UAV applications. In: 2018 design, automation & test in Europe conference & exhibition (DATE). IEEE, pp 967–972

  • Laña I, Sanchez-Medina JJ, Vlahogianni EI, Del Ser J (2021) From data to actions in intelligent transportation systems: a prescription of functional requirements for model actionability. Sensors 21(4):1121. https://doi.org/10.3390/s21041121

    Article  Google Scholar 

  • LeCun Y, Bengio Y (1995) Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 3361(10):1995

  • Lee, J, Zijia Z, Kitae K, Branislav D, Bo D, Slobodan G (2015) Examining the applicability of small quadcopter drone for traffic surveillance and roadway incident monitoring. Transportation Research Board 94th Annual Meeting Compendium of Papers, pp 15–4184.

  • Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer visio. Springer, Cham, pp 21–37

  • Lv Y, Duan Y, Kang W, Li Z, Wang FY (2014) Traffic flow prediction with big data: a deep learning approach. IEEE Trans Intell Transp Syst 16(2):865–873

    Google Scholar 

  • Mairaj A, Baba AI, Javaid AY (2019) Application specific drone simulators: recent advances and challenges. Simul Model Pract Theory. https://doi.org/10.1016/j.simpat.2019.01.004

    Article  Google Scholar 

  • Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  • Phon-Amnuaisuk S, Murata KT, Pavarangkoon P, Yamamoto K, Mizuhara T (2018) Exploring the applications of faster R-CNN and single-shot multi-box detection in a smart nursery domain. arxiv:1808.08675

  • Polson NG, Sokolov VO (2017) Deep learning for short-term traffic flow prediction. Transp Res Part C Emerg Technol 79:1–17

    Article  Google Scholar 

  • Puri A (2005) A survey of Unmanned Aerial Vehicles (UAV) for traffic surveillance. University of South Florida, Department of Computer Science and Engineering, pp 1–29

    Google Scholar 

  • Puri A, Valavanis K, Kontitsis M (2007) Statistical profile generation for traffic monitoring using realtime UAV based video data. 2007 Mediterr Conf Control Autom. https://doi.org/10.1109/MED.2007.4433658

    Article  Google Scholar 

  • Reinartz P, Lachaise M, Schmeer E, Krauss T, Runge H (2006) Traffic monitoring with serial images from airborne cameras. ISPRS J Photogramm Remote Sens 61(3):149–158

    Article  Google Scholar 

  • Rezaei M, Isehaghi M (2018) An efficient method for license plate localization using multiple statistical features in a multilayer perceptron neural network. In: 2018 9th conference on artificial intelligence and robotics and 2nd Asia-Pacific international symposium. IEEE, pp 7–13

  • Salvo G, Luigi C, Alessandro S (2014) Urban traffic analysis through an UAV. Procedia Soc Behav Sci 111:1083–1091

    Article  Google Scholar 

  • Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520

  • Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks. In: The 2011 international joint conference on neural networks. IEEE, pp 2809–2813

  • Shastry A, Schowengerdt R (2002) Airborne video registration for visualization and parameter estimation of traffic flows. In: Proceedings of pecora, vol 15, pp 391–405

  • Sivaraman S, Trivedi MM (2013) Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Trans Intell Transp Syst 14(4):1773–1795

    Article  Google Scholar 

  • Wang L, Chen F, Yin H (2016) Detecting and tracking vehicles in traffic by unmanned aerial vehicles. Autom Constr 72:294–308

    Article  Google Scholar 

  • Wang X, Cheng P, Liu X, Uzochukwu B (2018) Fast and accurate, convolutional neural network based approach for object detection from UAV. In: IECON 2018-44th annual conference of the IEEE industrial electronics society. IEEE, pp 3171–3175

  • Yu F, Xian W, Chen Y, Liu F, Liao M, Madhavan V, Darrell T (2018) BDD100K: a diverse driving video database, University of California, Berkeley

  • Zhang W, Jordan G, Livshits V (2016) Generating a vehicle trajectory database from time-lapse aerial photography. Transp Res Rec 2594:148–158

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Civil Engineering, National Technical University of Athens, Athens, Greece

    Eleni I. Vlahogianni

  2. TECNALIA and University of the Basque Country (UPV/EHU), Leioa, Spain

    Javier Del Ser & Ibai Laña

  3. School of Rural and Surveying Engineering, National Technical University of Athens, Athens, Greece

    Konstantinos Kepaptsoglou

Authors
  1. Eleni I. Vlahogianni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javier Del Ser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konstantinos Kepaptsoglou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ibai Laña
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eleni I. Vlahogianni.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vlahogianni, E.I., Del Ser, J., Kepaptsoglou, K. et al. Model Free Identification of Traffic Conditions Using Unmanned Aerial Vehicles and Deep Learning. J. Big Data Anal. Transp. 3, 1–13 (2021). https://doi.org/10.1007/s42421-021-00038-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42421-021-00038-z

Keywords

Search

Navigation