Skip to main content
SpringerLink
Log in

UAV Traffic Patrolling via Road Detection and Tracking in Anonymous Aerial Video Frames

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Unmanned Aerial Vehicles (UAV) have gained great importance for patrolling, exploration, and surveillance. In this study, we have estimated a route UAV to follow, using aerial road images. In the experimental setup, for estimation, test, and validation stages, anonymous aerial road videos have been exploited, meaning a special image database was not produced for this simulation approach. In the proposed study, road portion is initially detected. Two methods are utilized to help road detection, which are k-Nearest Neighbor and Hough transformation. To form a decision loop, both results are matched. If they match each other, they are fused using spatial and spectral schemes for the comparison purpose. Once road area is detected, the road type classification is realized by Fuzzy approach. The resultant image is utilized to estimate route, over which the UAV have to fly towards that direction. In the simulation stage, an anonymous video stream previously captured by UAV is experimented to assess the performance of the underlying system for different roads. According to the implementation results, the proposed algorithm has succeeded in finding all the trial roads in the given aerial images, and the proportion of all the estimated road-portion to actual road pixels for all the images is averagely calculated as %95.40. Eventually, it is shown that UAV has followed the correct route, which is estimated by proposed approach, over the specified road using assigned video frames, and also performances of spatial and spectral fusion results are compared.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Federal Aviation Administration (FAA), http://www.faa.gov (2017)

  2. Ehang, Homepage: http://www.ehang.com/ehang184 (2017)

  3. Finlayson, G.D., Drew, M.S., Lu, C.: Intrinsic images by entropy minimization. In: Computer Vision-ECCV 2004, pp 582–595. Springer, Berlin (2004)

  4. He, Y., Wang, H., Zhang, B.: Color based road detection in urban traffic scenes. Intell Trans Syst IEEE 1, 730– 735 (2003)

    Google Scholar 

  5. Wang, Y., Teoh, E., Shen, D.: Lane detection and tracking using b-snake. Image Vis Comput 22(4), 269–280 (2004)

    Article  Google Scholar 

  6. Broggi, A.: Robust real-time lane and road detection in critical shadow conditions. In: Proceedings IEEE International Symposium on Computer Vision, pp 353–358 (1995)

  7. Kong, H., Audibert, J.Y., Ponce, J.: Vanishing point detection for road detection. In: Proceedings IEEE Conference Computer Vision Pattern Recognition, pp 96–103 (2009 )

  8. Lin, Y., Saripalli, S.: Road detection from aerial imagery. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp 3588–3593 (2012)

  9. Fernandez, C., Izquierdo, R., Fernandez Llorca, D., Sotelo, M.A.: A comparative analysis of decision trees based classifiers for road detection in urban environments. In: 2015 IEEE 18th International Conference on Intelligent Transportation Systems (ITSC), pp 719–724 (2015)

  10. Alvarez, J.M., Lopez, A.M.: Road detection based on illuminant invariance, vol. 12 (2011)

  11. Cheng-Li, J., Ke-Feng, J., Yong-Mei, J., Gang-Yao, K.: Road Extraction from High-Resolution SAR Imagery Using Hough Transform. In: IEEE International Geoscience and Remote Sensing Symposium, IGARSS’05. vol. 1, pp. 4-pp, IEEE (2005)

  12. Kong, H., Audibert, J.Y., Ponce, J.: General road detection from a single image. IEEE Trans. Image Process. 19(8), 2211–2220 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J., Andriluka, M., Rajpurkar, P., Migimatsu, T., Yue, R. C., Mujica, F., Coates, A., Ng, A.Y.: An empirical evaluation of deep learning on highway driving, arXiv:1504.01716 (2015)

  14. Fritsch, J., Kuhnl, T., Kummert, F.: Monocular road terrain detection by combining visual and spatial information. IEEE Trans. Intell. Transp. Syst. 15(4), 1586–1596 (2014)

    Article  Google Scholar 

  15. Karaduman, O., Eren, H., Kurum, H., Celenk, M.: Road-Geometry-Based Risk estimation model for horizontal curves. IEEE Trans. Intell. Transp. Syst. 99, 1–11 (2016)

    Google Scholar 

  16. Gould, S., Fulton, S.R., Koller, D.: Decomposing a Scene into Geometric and Semantically Consistent Regions. In: IEEE 12th International Conference on Computer Vision, IEEE, pp 1–8 (2009)

  17. Tighe, J., Lazebnik, S.: Superparsing: scalable nonparametric image parsing with super pixels. In: Computer Vision–ECCV, pp 352–365. Springer, Berlin (2010)

  18. Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: Proceedings European Conference Computer Vision, pp. 1–15 (2008)

  19. Ladicky, L., Sturgess, P., Alahari, K., Russell, C., Torr, P.H.S.: What, Where and How Many? Combining Object Detectors and CRFs. In: Proceedings European Conference Computer Vision, pp 424–437 (2010)

  20. Zhang, C., Wang, L., Yang, R.: Semantic Segmentation of Urban Scenes using Dense Depth Maps. In: Proceedings European Conference Computer Vision, pp 708–721 (2010)

  21. Sturgess, P., Alahari, K., Ladicky, L., Torr, P.H.S.: Combining appearance and structure from motion features for road scene understanding. In: British Machine Vision Conference, pp 1–11 (2009)

  22. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  23. Li, H., Manjunath, B.S., Mitra, S.K.: Multisensor Image Fusion using the Wavelet Transform. Graphical Models Image Process. 57(3), 235–245 (1995)

    Article  Google Scholar 

  24. Zhou, H., Kong, H., Wei, L., Creighton, D., Nahavandi, S.: On detecting road regions in a single UAV image. IEEE Trans. Intell. Transp. Syst. 18(7), 1713–1722 (2017)

    Article  Google Scholar 

  25. Sommer, L.W., Schuchert, T., Beyerer, J.: A comprehensive study on object proposals methods for vehicle detection in aerial images. In: 9th IAPR Workshop on Pattern Recogniton in Remote Sensing (PRRS), pp 1–6 (2016)

  26. Xu, Y., Yu, G., Wu, X., Wang, Y., Ma, Y.: An enhanced Viola-Jones vehicle detection method from unmanned aerial vehicles imagery. IEEE Trans. Intell. Transp. Syst. 7, 1845–1856 (2017)

    Article  Google Scholar 

  27. Elliethy, A., Sharma, G.: Automatic registration of wide area motion imagery to vector road maps by exploiting vehicle detections. IEEE Trans. Image Process. 25(11), 5304–5315 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  28. Trinder, J.C., Wang, Y.: Automatic road extraction from aerial images. Digi. Signal Process. 8(4), 215–224 (1998)

    Article  Google Scholar 

  29. Barkley, B.E., Paley, D.A.: Cooperative bayesian target detection on a real road network using aerial vehicles. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp 53–61 (2016)

  30. Liu, K., Mattyus, G.: Fast multiclass vehicle detection on aerial images. IEEE Geosci. Remote Sens. Lett. 12(9), 1938–1942 (2015)

    Article  Google Scholar 

  31. Qu, Y., Jiang, L., Guo, X.: Moving vehicle detection with convolutional networks in UAV videos. In: 2nd International Conference on Control, Automation and Robotics (ICCAR), pp 225–229 (2016)

  32. Zhou, H., Kong, H., Wei, L., Creighton, D., Nahavandi, S.: On Detecting road regions in a single UAV image. IEEE transactions on intelligent transportation systems (2016)

  33. Peng, X.Z., Lin, H.Y., Dai, J.M.: Path planning and obstacle avoidance for vision guided quadrotor UAV navigation. In: 12th IEEE International Conference on Control and Automation (ICCA), pp 984–989 (2016)

  34. Karila, K., Matikainen, L., Puttonen, E., Hyyppä, J.: Feasibility of multispectral airborne laser scanning data for road mapping. IEEE Geosci. Remote Sens. Lett. 14(3), 294–298 (2017)

    Article  Google Scholar 

  35. Rasmussen, S., Kalyanam, K., Kingston, D.: Field experiment of a fully autonomous multiple UAV/UGS intruder detection and monitoring system. In: International Conference on Unmanned Aircraft Systems (ICUAS), pp 1293–1302 (2016)

  36. Geiger, A., Lenz, P., Urtasun, R.: Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

  37. Staplin, L., Gish, K.W., Decina, L.E., Lococo, K.H., Harkey, D.L., Tarawneh, M.S., Garvey, P.: Synthesis of Human Factors Research On Older Drivers And Highway Safety, Human Factors And Highway Safety Research Synthesis, V(II) (1997)

  38. Kanistras, K., Martins, G., Rutherford, M.J., Valavanis, K.P.: Survey of Unmanned Aerial Vehicles (UAVs) for Traffic Monitoring. In: Handbook of Unmanned Aerial Vehicles, pp 2643–2666. Springer, Netherlands (2015)

  39. Michailidis, M.G., Kanistras, K., Agha, M., Rutherford, M.J., Valavanis, K.P.: Robust nonlinear control of the longitudinal flight dynamics of a circulation control fixed wing UAV. In: 56th IEEE Conference on Decision and Control, pp 3920–3927 (2017)

  40. Li, L., Zhang, Y.: Route planning based on genetic algorithm. J. Math. Res. 10(2), 122–128 (2018)

    Article  Google Scholar 

  41. Yang, J., Yin, D., Shen, L., Cheng, Q., Xie, X.: Cooperative deconflicting heading maneuvers applied to unmanned aerial vehicles in Non-Segregated airspace. J. Intell. Robot. Syst. 92(1), 187–201 (2018)

    Article  Google Scholar 

  42. Zhou, H., Kong, H., Wei, L., Creighton, D., Nahavandi, S.: Efficient road detection and tracking for unmanned aerial vehicle. IEEE Trans. Intell. Transp. Syst. 16(1), 297–309 (2015)

    Article  Google Scholar 

  43. Lin, Y., Saripalli, S.: Road detection and tracking from aerial desert imagery. J. Intell. Robot. Syst. 65 (1-4), 345–359 (2012)

    Article  Google Scholar 

  44. Flores, G.F., Lozano Leal, R., Sanahuja, G.: Lyapunov-Based Switching Control for a Road Estimation and Tracking Applied on a Convertible MAV. In: Proceedings AIAA Guidance, Navigation, and Control (GNC) Conference, pp 1–13 (2013)

  45. http://www.bloomberg.com/news/articles/2015-04-25/amazon-seeks-chance-to-show-u-s-drones-can-safely-deliver-cargo, 2015

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, İnönü University, Malatya, Turkey

    Mücahit Karaduman

  2. Engineering Faculty, Computer Engineering, Fırat University, Elazığ, Turkey

    Ahmet Çınar

  3. The School of Aviation, Air Traffic Management, Fırat University, Elazığ, Turkey

    Haluk Eren

Authors
  1. Mücahit Karaduman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmet Çınar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haluk Eren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haluk Eren.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 3.94 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karaduman, M., Çınar, A. & Eren, H. UAV Traffic Patrolling via Road Detection and Tracking in Anonymous Aerial Video Frames. J Intell Robot Syst 95, 675–690 (2019). https://doi.org/10.1007/s10846-018-0954-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0954-x

Keywords

Search

Navigation