UAV Traffic Patrolling via Road Detection and Tracking in Anonymous Aerial Video Frames

  • Mücahit Karaduman
  • Ahmet Çınar
  • Haluk Eren


Unmanned Aerial Vehicles (UAV) have gained great importance for patrolling, exploration, and surveillance. In this study, we have estimated a route UAV to follow, using aerial road images. In the experimental setup, for estimation, test, and validation stages, anonymous aerial road videos have been exploited, meaning a special image database was not produced for this simulation approach. In the proposed study, road portion is initially detected. Two methods are utilized to help road detection, which are k-Nearest Neighbor and Hough transformation. To form a decision loop, both results are matched. If they match each other, they are fused using spatial and spectral schemes for the comparison purpose. Once road area is detected, the road type classification is realized by Fuzzy approach. The resultant image is utilized to estimate route, over which the UAV have to fly towards that direction. In the simulation stage, an anonymous video stream previously captured by UAV is experimented to assess the performance of the underlying system for different roads. According to the implementation results, the proposed algorithm has succeeded in finding all the trial roads in the given aerial images, and the proportion of all the estimated road-portion to actual road pixels for all the images is averagely calculated as %95.40. Eventually, it is shown that UAV has followed the correct route, which is estimated by proposed approach, over the specified road using assigned video frames, and also performances of spatial and spectral fusion results are compared.


UAV reconnaissance Nextgen traffic patrolling Aerial road tracking Fuzzy classifier Spatial-spectral fusion Route estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


Supplementary material

(MP4 3.94 MB)


  1. 1.
    Federal Aviation Administration (FAA), (2017)
  2. 2.
    Ehang, Homepage: (2017)
  3. 3.
    Finlayson, G.D., Drew, M.S., Lu, C.: Intrinsic images by entropy minimization. In: Computer Vision-ECCV 2004, pp 582–595. Springer, Berlin (2004)Google Scholar
  4. 4.
    He, Y., Wang, H., Zhang, B.: Color based road detection in urban traffic scenes. Intell Trans Syst IEEE 1, 730– 735 (2003)Google Scholar
  5. 5.
    Wang, Y., Teoh, E., Shen, D.: Lane detection and tracking using b-snake. Image Vis Comput 22(4), 269–280 (2004)CrossRefGoogle Scholar
  6. 6.
    Broggi, A.: Robust real-time lane and road detection in critical shadow conditions. In: Proceedings IEEE International Symposium on Computer Vision, pp 353–358 (1995)Google Scholar
  7. 7.
    Kong, H., Audibert, J.Y., Ponce, J.: Vanishing point detection for road detection. In: Proceedings IEEE Conference Computer Vision Pattern Recognition, pp 96–103 (2009 )Google Scholar
  8. 8.
    Lin, Y., Saripalli, S.: Road detection from aerial imagery. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp 3588–3593 (2012)Google Scholar
  9. 9.
    Fernandez, C., Izquierdo, R., Fernandez Llorca, D., Sotelo, M.A.: A comparative analysis of decision trees based classifiers for road detection in urban environments. In: 2015 IEEE 18th International Conference on Intelligent Transportation Systems (ITSC), pp 719–724 (2015)Google Scholar
  10. 10.
    Alvarez, J.M., Lopez, A.M.: Road detection based on illuminant invariance, vol. 12 (2011)Google Scholar
  11. 11.
    Cheng-Li, J., Ke-Feng, J., Yong-Mei, J., Gang-Yao, K.: Road Extraction from High-Resolution SAR Imagery Using Hough Transform. In: IEEE International Geoscience and Remote Sensing Symposium, IGARSS’05. vol. 1, pp. 4-pp, IEEE (2005)Google Scholar
  12. 12.
    Kong, H., Audibert, J.Y., Ponce, J.: General road detection from a single image. IEEE Trans. Image Process. 19(8), 2211–2220 (2010)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J., Andriluka, M., Rajpurkar, P., Migimatsu, T., Yue, R. C., Mujica, F., Coates, A., Ng, A.Y.: An empirical evaluation of deep learning on highway driving, arXiv:1504.01716 (2015)
  14. 14.
    Fritsch, J., Kuhnl, T., Kummert, F.: Monocular road terrain detection by combining visual and spatial information. IEEE Trans. Intell. Transp. Syst. 15(4), 1586–1596 (2014)CrossRefGoogle Scholar
  15. 15.
    Karaduman, O., Eren, H., Kurum, H., Celenk, M.: Road-Geometry-Based Risk estimation model for horizontal curves. IEEE Trans. Intell. Transp. Syst. 99, 1–11 (2016)Google Scholar
  16. 16.
    Gould, S., Fulton, S.R., Koller, D.: Decomposing a Scene into Geometric and Semantically Consistent Regions. In: IEEE 12th International Conference on Computer Vision, IEEE, pp 1–8 (2009)Google Scholar
  17. 17.
    Tighe, J., Lazebnik, S.: Superparsing: scalable nonparametric image parsing with super pixels. In: Computer Vision–ECCV, pp 352–365. Springer, Berlin (2010)Google Scholar
  18. 18.
    Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: Proceedings European Conference Computer Vision, pp. 1–15 (2008)Google Scholar
  19. 19.
    Ladicky, L., Sturgess, P., Alahari, K., Russell, C., Torr, P.H.S.: What, Where and How Many? Combining Object Detectors and CRFs. In: Proceedings European Conference Computer Vision, pp 424–437 (2010)Google Scholar
  20. 20.
    Zhang, C., Wang, L., Yang, R.: Semantic Segmentation of Urban Scenes using Dense Depth Maps. In: Proceedings European Conference Computer Vision, pp 708–721 (2010)Google Scholar
  21. 21.
    Sturgess, P., Alahari, K., Ladicky, L., Torr, P.H.S.: Combining appearance and structure from motion features for road scene understanding. In: British Machine Vision Conference, pp 1–11 (2009)Google Scholar
  22. 22.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  23. 23.
    Li, H., Manjunath, B.S., Mitra, S.K.: Multisensor Image Fusion using the Wavelet Transform. Graphical Models Image Process. 57(3), 235–245 (1995)CrossRefGoogle Scholar
  24. 24.
    Zhou, H., Kong, H., Wei, L., Creighton, D., Nahavandi, S.: On detecting road regions in a single UAV image. IEEE Trans. Intell. Transp. Syst. 18(7), 1713–1722 (2017)CrossRefGoogle Scholar
  25. 25.
    Sommer, L.W., Schuchert, T., Beyerer, J.: A comprehensive study on object proposals methods for vehicle detection in aerial images. In: 9th IAPR Workshop on Pattern Recogniton in Remote Sensing (PRRS), pp 1–6 (2016)Google Scholar
  26. 26.
    Xu, Y., Yu, G., Wu, X., Wang, Y., Ma, Y.: An enhanced Viola-Jones vehicle detection method from unmanned aerial vehicles imagery. IEEE Trans. Intell. Transp. Syst. 7, 1845–1856 (2017)CrossRefGoogle Scholar
  27. 27.
    Elliethy, A., Sharma, G.: Automatic registration of wide area motion imagery to vector road maps by exploiting vehicle detections. IEEE Trans. Image Process. 25(11), 5304–5315 (2016)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Trinder, J.C., Wang, Y.: Automatic road extraction from aerial images. Digi. Signal Process. 8(4), 215–224 (1998)CrossRefGoogle Scholar
  29. 29.
    Barkley, B.E., Paley, D.A.: Cooperative bayesian target detection on a real road network using aerial vehicles. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp 53–61 (2016)Google Scholar
  30. 30.
    Liu, K., Mattyus, G.: Fast multiclass vehicle detection on aerial images. IEEE Geosci. Remote Sens. Lett. 12(9), 1938–1942 (2015)CrossRefGoogle Scholar
  31. 31.
    Qu, Y., Jiang, L., Guo, X.: Moving vehicle detection with convolutional networks in UAV videos. In: 2nd International Conference on Control, Automation and Robotics (ICCAR), pp 225–229 (2016)Google Scholar
  32. 32.
    Zhou, H., Kong, H., Wei, L., Creighton, D., Nahavandi, S.: On Detecting road regions in a single UAV image. IEEE transactions on intelligent transportation systems (2016)Google Scholar
  33. 33.
    Peng, X.Z., Lin, H.Y., Dai, J.M.: Path planning and obstacle avoidance for vision guided quadrotor UAV navigation. In: 12th IEEE International Conference on Control and Automation (ICCA), pp 984–989 (2016)Google Scholar
  34. 34.
    Karila, K., Matikainen, L., Puttonen, E., Hyyppä, J.: Feasibility of multispectral airborne laser scanning data for road mapping. IEEE Geosci. Remote Sens. Lett. 14(3), 294–298 (2017)CrossRefGoogle Scholar
  35. 35.
    Rasmussen, S., Kalyanam, K., Kingston, D.: Field experiment of a fully autonomous multiple UAV/UGS intruder detection and monitoring system. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp 1293–1302 (2016)Google Scholar
  36. 36.
    Geiger, A., Lenz, P., Urtasun, R.: Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  37. 37.
    Staplin, L., Gish, K.W., Decina, L.E., Lococo, K.H., Harkey, D.L., Tarawneh, M.S., Garvey, P.: Synthesis of Human Factors Research On Older Drivers And Highway Safety, Human Factors And Highway Safety Research Synthesis, V(II) (1997)Google Scholar
  38. 38.
    Kanistras, K., Martins, G., Rutherford, M.J., Valavanis, K.P.: Survey of Unmanned Aerial Vehicles (UAVs) for Traffic Monitoring. In: Handbook of Unmanned Aerial Vehicles, pp 2643–2666. Springer, Netherlands (2015)Google Scholar
  39. 39.
    Michailidis, M.G., Kanistras, K., Agha, M., Rutherford, M.J., Valavanis, K.P.: Robust nonlinear control of the longitudinal flight dynamics of a circulation control fixed wing UAV. In: 56th IEEE Conference on Decision and Control, pp 3920–3927 (2017)Google Scholar
  40. 40.
    Li, L., Zhang, Y.: Route planning based on genetic algorithm. J. Math. Res. 10(2), 122–128 (2018)CrossRefGoogle Scholar
  41. 41.
    Yang, J., Yin, D., Shen, L., Cheng, Q., Xie, X.: Cooperative deconflicting heading maneuvers applied to unmanned aerial vehicles in Non-Segregated airspace. J. Intell. Robot. Syst. 92(1), 187–201 (2018)CrossRefGoogle Scholar
  42. 42.
    Zhou, H., Kong, H., Wei, L., Creighton, D., Nahavandi, S.: Efficient road detection and tracking for unmanned aerial vehicle. IEEE Trans. Intell. Transp. Syst. 16(1), 297–309 (2015)CrossRefGoogle Scholar
  43. 43.
    Lin, Y., Saripalli, S.: Road detection and tracking from aerial desert imagery. J. Intell. Robot. Syst. 65 (1-4), 345–359 (2012)CrossRefGoogle Scholar
  44. 44.
    Flores, G.F., Lozano Leal, R., Sanahuja, G.: Lyapunov-Based Switching Control for a Road Estimation and Tracking Applied on a Convertible MAV. In: Proceedings AIAA Guidance, Navigation, and Control (GNC) Conference, pp 1–13 (2013)Google Scholar
  45. 45.

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Computer Scienceİnönü UniversityMalatyaTurkey
  2. 2.Engineering Faculty, Computer EngineeringFırat UniversityElazığTurkey
  3. 3.The School of Aviation, Air Traffic ManagementFırat UniversityElazığTurkey

Personalised recommendations