Skip to main content
SpringerLink
Log in

ODRP: a new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Geographical information systems (GIS) are the systems where spatial data are stored and analyzed. The most important raw material in GIS is spatial data. Thus, it is essential to collect and update these data. On the other hand, exchangeable image file (EXIF) format is a special file format that contains camera direction, date-time information and GPS location provided by a digital camera that captures the images. Transferring the objects in EXIF data sets with absolute coordinates on the earth significantly contributes to GIS. In this study, a new hybrid approach, ODPR, which utilizes object detection (O), distance estimation (D), rotation (R) and projection (P) methods, is proposed to detect street sign objects in EXIF with their locations. The performance of the proposed approach has been examined on the natural EXIF data sets obtained from the Kayseri Metropolitan Municipality. In the proposed approach, a deep learning method detects a street sign object in the EXIF. Then, the object’s distance is calculated at the point where the photograph is taken. Finally, the spatial location of the detected object on the earth is calculated using distance, direction and GPS data with rotation and projection methods. In the proposed ODRP approach, the performances of convolutional neural network (CNN)-based Faster R-CNN, YOLO V5, YOLO V6 and transformer-based DETR models as deep learning models for object detection are examined. The F1 score metric is widely used to examine the performance of methods in deep learning models. The performances of the proposed approaches are reviewed according to the F1 score values, and ODRP Faster R-CNN, YOLO V5, YOLO V6 and DETR approaches achieved F1 scores of 0.909, 0.956, 0.948 and 0.922, respectively. In addition, to overcome the variability of light and background mixing problems, an improved supervised learning method (ISL) is proposed. Thanks to ISL, ODRP Faster R-CNN, YOLO V5, YOLO V6, and DETR approaches have reached 0.965, 0.985, 0.969 and 0.942 f1 scores, respectively. The proposed ODRP Faster R-CNN, YOLO V5, YOLO V6 and DETR approaches found the location of the street sign object to be 11434.76, 12818.39, 12454.63 and 9843.57 ms closer to its position on earth than the classical method, which considers the location of the EXIF, respectively. Regarding time cost, the ODRP Faster R-CNN, YOLO V5, YOLO V6 and DETR analyze EXIF data at an average of 0.99, 0.42, 0.41 and 0.53 s, respectively. The run time of the ODRP YOLO V5 and V6 approaches is almost equal to each other, and it works approximately 2.5 times faster than the ODRP Faster R-CNN method. Consequently, ODRP YOLO V5 outperforms ODRP Faster R-CNN, YOLO V6 and DETR for detecting the spatial location of street sign objects in EXIF and the F1 score.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

Data sharing is not applicable to this article as the data used in this study belong to Kayseri Metropolitan Municipality. However, the relevant institution requested that the data set not be made open data. Therefore, data sharing does not apply to this article as the data owner does not allow it.

References

  1. ArcView, G.: The geographic information system for everyone. Environ. Syst. Res. Inst. 3 (1996)

  2. Chang, K.-T.: Geographic information system. In: International Encyclopedia of Geography: People, the Earth, Environment and Technology: People, the Earth, Environment and Technology, pp. 1–9 (2016)

  3. Dong, P.: Generating and updating multiplicatively weighted voronoi diagrams for point, line and polygon features in gis. Comput. Geosci. 34(4), 411–421 (2008)

    Article  Google Scholar 

  4. Tasyurek, M., Celik, M.: 4d-gwr: geographically, altitudinal, and temporally weighted regression. Neural Comput. Appl. 1–15 (2022)

  5. Folger, P.: Geospatial Information and Geographic Information Systems (GIS): Current Issues and Future Challenges. DIANE Publishing, New York (2010)

    Google Scholar 

  6. Gangwar, D., Pathania, A.: Authentication of digital image using exif metadata and decoding properties. Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol. IJSR CSEIT 3(8), 335–341 (2018)

    Google Scholar 

  7. Bayoudh, K., Knani, R., Hamdaoui, F., Mtibaa, A.: A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. Vis. Comput. 38(8), 2939–2970 (2022)

    Article  Google Scholar 

  8. Kalsotra, R., Arora, S.: Background subtraction for moving object detection: explorations of recent developments and challenges. Vis. Comput. 1–28 (2021)

  9. TAŞYÜREK, M., ÖZTÜRK, C.: Ddl: A new deep learning based approach for multiple house numbers detection and clustering. J Fac. Eng. Arch. Gazi Univ. 37(2) (2022)

  10. Agrawal, A., Mittal, N.: Using cnn for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. Vis. Comput. 36(2), 405–412 (2020)

    Article  Google Scholar 

  11. Li, Y., Wang, Z., Yin, L., Zhu, Z., Qi, G., Liu, Y.: X-net: a dual encoding–decoding method in medical image segmentation. Vis. Comput. 1–11 (2021)

  12. Arslan, R.S., Tasyurek, M.: Amd-cnn: android malware detection via feature graph and convolutional neural networks. Concurr. Comput. Pract. Exp. 34(23), 7180 (2022)

    Article  Google Scholar 

  13. Ciaburro, G., Venkateswaran, B.: Neural networks with R: Smart Models Using CNN, RNN, Deep Learning, and Artificial Intelligence Principles, pp. 183–211. Packt Publishing Ltd, Birmingham (2017)

  14. Chen, Y., Li, W., Sakaridis, C., Dai, D., Van Gool, L.: Domain adaptive faster r-cnn for object detection in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3339–3348 (2018)

  15. Jiang, P., Ergu, D., Liu, F., Cai, Y., Ma, B.: A review of yolo algorithm developments. Proc. Comput. Sci. 199, 1066–1073 (2022)

    Article  Google Scholar 

  16. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213–229 (2020). Springer

  17. Zhang, J., Xie, W., Wang, C., Tu, R., Tu, Z.: Graph-aware transformer for skeleton-based action recognition. Vis. Comput. 1–12 (2022)

  18. Kiefer, S.: ExifLib.Net: A Fast Exif Data Extractor for .NET 4.5+. https://github.com/esskar/ExifLib.Net/tree/master/Sources/ExifLib Accessed 2022-08-03

  19. Rath, S.R.: Custom Object Detection Using PyTorch Faster RCNN. https://debuggercafe.com/custom-object-detection-using-pytorch-faster-rcnn Accessed 2022-05-21

  20. Jocher, G., Nishimura, K., Mineeva, T., Vilariño, R.: Yolov5.https://github.com/ultralytics/yolov5 Accessed 2022-05-21

  21. Chilicyy: YOLOv6: a Single-stage Object Detection Framework Dedicated to Industrial Application. https://github.com/Chilicyy/YOLOv6 Accessed 2022-07-29

  22. Cooperative, G., Collins, F.: The unique qualities of a geographic information system: a commentary. Photogramm. Eng. Remote. Sens. 54(11), 1547–9 (1988)

    Google Scholar 

  23. Alzubaidi, L., Zhang, J., Humaidi, A.J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M.A., Al-Amidie, M., Farhan, L.: Review of deep learning: concepts, cnn architectures, challenges, applications, future directions. J. Big Data 8(1), 1–74 (2021)

    Article  Google Scholar 

  24. Abdul-Rahman, A., Pilouk, M.: Spatial Data Modelling for 3D GIS. Springer, Amsterdam (2007)

    Google Scholar 

  25. Shekhar, S., Vatsavai, R.R., Celik, M.: Spatial and spatiotemporal data mining: recent advances. Next Gen. Data Min. 573–608 (2008)

  26. Fotheringham, S., Rogerson, P.: Spatial Analysis and GIS. Crc Press, Hong Kong (2013)

    Book  Google Scholar 

  27. Joseph, A., Geetha, P.: Facial emotion detection using modified eyemap-mouthmap algorithm on an enhanced image and classification with tensorflow. Vis. Comput. 36(3), 529–539 (2020)

    Article  Google Scholar 

  28. Iqbal, Z., Khan, M.A., Sharif, M., Shah, J.H., ur Rehman, M.H., Javed, K.: An automated detection and classification of citrus plant diseases using image processing techniques: a review. Comput. Electron. Agric. 153, 12–32 (2018)

    Article  Google Scholar 

  29. Taşyürek, M., Çelik, M.: Fastgtwr: Hızlı coğrafi ve zamansal ağırlıklı regresyon yaklaşımı. Gazi Univ Muhendislik Mimar Fak Derg 36(2), 715–726 (2021)

    Google Scholar 

  30. Zhang, Q., Ge, Y., Zhang, C., Bi, H.: Tprnet: camouflaged object detection via transformer-induced progressive refinement network. Vis. Comput. 1–15 (2022)

  31. Alburshaid, E., Mangoud, M.: Palm trees detection using the integration between gis and deep learning. In: 2021 International Symposium on Networks, Computers and Communications (ISNCC), pp. 1–6 (2021). IEEE

  32. Chun, P.-J., Yamane, T., Tsuzuki, Y.: Automatic detection of cracks in asphalt pavement using deep learning to overcome weaknesses in images and gis visualization. Appl. Sci. 11(3), 892 (2021)

    Article  Google Scholar 

  33. Kearney, S.P., Coops, N.C., Sethi, S., Stenhouse, G.B.: Maintaining accurate, current, rural road network data: An extraction and updating routine using rapideye, participatory gis and deep learning. Int. J. Appl. Earth Obs. Geoinf. 87, 102031 (2020)

    Google Scholar 

  34. Malaainine, M.E.I., Lechgar, H., Rhinane, H.: Yolov2 deep learning model and gis based algorithms for vehicle tracking. J. Geogr. Inf. Syst. 13(4), 395–409 (2021)

    Google Scholar 

  35. LeCun, Y., Haffner, P., Bottou, L., Bengio, Y.: Object recognition with gradient-based learning. In: Shape. Contour and Grouping in Computer Vision, pp. 319–345. Springer, Red Bank (1999)

  36. Paul, S., Singh, L., et al.: A review on advances in deep learning. In: 2015 IEEE Workshop on Computational Intelligence: Theories, Applications and Future Directions (WCI), pp. 1–6 (2015). IEEE

  37. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25 (2012)

  38. Alom, M.Z., Taha, T.M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M.S., Hasan, M., Van Essen, B.C., Awwal, A.A., Asari, V.K.: A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3), 292 (2019)

    Article  Google Scholar 

  39. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  40. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

  41. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  42. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  43. Tang, W., He, F., Liu, Y.: Ydtr: infrared and visible image fusion via y-shape dynamic transformer. IEEE Trans. Multimed. (2022)

  44. Tang, W., He, F., Liu, Y., Duan, Y.: Matr: multimodal medical image fusion via multiscale adaptive transformer. IEEE Trans. Image Process. 31, 5134–5149 (2022)

    Article  Google Scholar 

  45. Research, F.: DETR: End-to-End Object Detection with Transformers. https://github.com/facebookresearch/detr Accessed 2022-01-11

  46. Si, T., He, F., Zhang, Z., Duan, Y.: Hybrid contrastive learning for unsupervised person re-identification. IEEE Trans. Multimed. (2022)

  47. Shao, F., Chen, L., Shao, J., Ji, W., Xiao, S., Ye, L., Zhuang, Y., Xiao, J.: Deep learning for weakly-supervised object detection and localization: a survey. Neurocomputing (2022)

  48. Adrakatti, A., Wodeyar, R., Mulla, K.: Search by image: a novel approach to content based image retrieval system. Int. J. Libr. Sci. 14(3), 41–47 (2016)

    Google Scholar 

  49. Hadlow, N., Brown, S., Wardrop, R., Conradie, J., Henley, D.: Where in the world? Latitude, longitude and season contribute to the complex co-ordinates determining cortisol levels. Clin. Endocrinol. 89(3), 299–307 (2018)

    Article  Google Scholar 

  50. Yang, Q., Snyder, J., Tobler, W.: Map Projection Transformation: Principles and Applications. CRC Press, New York (1999)

    Google Scholar 

  51. Canters, F.: Small-scale Map Projection Design. CRC Press, London (2002)

    Book  Google Scholar 

  52. Nicolai, R., Simensen, G.: The new epsg geodetic parameter registry. In: 70th EAGE Conference and Exhibition Incorporating SPE EUROPEC 2008, European Association of Geoscientists & Engineers, p. 40 (2008)

  53. Santiago, A.: The Book of Openlayers 3. Theory and Practice, Leanpub, Victoria, BC (2015)

  54. Jain, S., Barclay, T.: Adding the EPSG: 4326 Geographic Longitude-Latitude Projection to TerraServer (2003)

  55. Maier, G.: Openstreetmap, the wikipedia map. Region 1(1), 3–10 (2014)

    Article  Google Scholar 

  56. Malkauthekar, M.: Analysis of euclidean distance and manhattan distance measure in face recognition. In: Third International Conference on Computational Intelligence and Information Technology (CIIT 2013), pp. 503–507 (2013). IET

  57. Merigó, J.M., Casanovas, M.: A new minkowski distance based on induced aggregation operators. Int. J. Comput. Intell. Syst. 4(2), 123–133 (2011)

    Google Scholar 

  58. Prasetya, R.P., Utaminingrum, F.: Triangle similarity approach for detecting eyeball movement. In: 2017 5th International Symposium on Computational and Business Intelligence (ISCBI), pp. 37–40 (2017). IEEE

  59. Zope, V., Joshi, N., Iyengar, S., Mahadevan, K., Singh, M.: Efficient social distancing detection using object detection and triangle similarity. In: International Conference on Advances in Computing and Data Sciences, pp. 81–89 (2021). Springer

  60. Dal, A.: Yolov4-Detector-and-Distance-Estimator. https://github.com/Asadullah-Dal17/Yolov4-Detector-and-Distance-Estimator Accessed 2022-08-12

  61. Tripathi, G., Singh, K., Vishwakarma, D.K.: Convolutional neural networks for crowd behaviour analysis: a survey. Vis. Comput. 35(5), 753–776 (2019)

    Article  Google Scholar 

  62. Jayalakshmi, G., Kumar, V.S.: Performance analysis of convolutional neural network (cnn) based cancerous skin lesion detection system. In: 2019 International Conference on Computational Intelligence in Data Science (ICCIDS), pp. 1–6 (2019). IEEE

  63. Hussain, M., Bird, J.J., Faria, D.R.: A study on cnn transfer learning for image classification. In: UK Workshop on Computational Intelligence, pp. 191–202 (2018). Springer

  64. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

  65. Redmon, J., Farhadi, A.: Yolov3: An Incremental Improvement. arXiv preprint arXiv:1804.02767 (2018)

  66. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: Optimal Speed and Accuracy of Object Detection. arXiv preprint arXiv:2004.10934 (2020)

  67. Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., et al.: A Survey on Visual Transformer. arXiv preprint arXiv:2012.125562(4) (2020)

  68. Cai, G., Zhu, Y., Wu, Y., Jiang, X., Ye, J., Yang, D.: A multimodal transformer to fuse images and metadata for skin disease classification. Vis. Comput. 1–13 (2022)

  69. Batuk, F., Öztürk, D., Ozan, E.: Türkiye ulusal konumsal veri altyapısı için temel veriler. Jeodezi Jeoinf Derg 96, 3–12 (2007)

    Google Scholar 

  70. Tarık, T.: Adres kayıt sistemi ile kent bilgi sistemlerinin bütünleştirilmesi. Jeodezi Jeoinf Derg 99, 13–22 (2008)

    Google Scholar 

  71. Chen, W., Huang, H., Peng, S., Zhou, C., Zhang, C.: Yolo-face: a real-time face detector. Vis. Comput. 37(4), 805–813 (2021)

    Article  Google Scholar 

  72. Liu, C., Ying, J., Yang, H., Hu, X., Liu, J.: Improved human action recognition approach based on two-stream convolutional neural network model. Vis. Comput. 37(6), 1327–1341 (2021)

    Article  Google Scholar 

  73. Quan, Q., He, F., Li, H.: A multi-phase blending method with incremental intensity for training detection networks. Vis. Comput. 37(2), 245–259 (2021)

    Article  Google Scholar 

  74. Tasyurek, M.: EXIF Direction Reader. https://github.com/murattasyurek Accessed 2022-11-07

  75. Powell, M.J., Sabin, M.A.: Piecewise quadratic approximations on triangles. ACM Trans. Math. Softw. TOMS 3(4), 316–325 (1977)

    Article  MathSciNet  Google Scholar 

  76. PostGIS: ST_Transform. https://postgis.net/docs/ST_Transform.html Accessed 2022-08-03

  77. Salt, A., noise to OpenCV Image, P.: Add Salt and Pepper Noise to OpenCV Image. https://gist.github.com/gutierrezps/f4ddad3bbd2ad5a9b96e3c06378e28b4, urldate = 2022-01-11

  78. Versloot, C.: How to Create a Train/test Split for Your Machine Learning Model? https://github.com/christianversloot/machine-learning-articles Accessed 2022-06-03

  79. Ding, Y.: LabelImg. https://github.com/heartexlabs/labelImg Accessed 2022-08-10

  80. Fort, S., Hu, H., Lakshminarayanan, B.: Deep ensembles: A Loss Landscape Perspective. arXiv preprint arXiv:1912.02757 (2019)

  81. Agafonkin, V.: Leaflet. https://leafletjs.com/ Accessed 2022-08-05

  82. Korotcov, A., Tkachenko, V., Russo, D.P., Ekins, S.: Comparison of deep learning with multiple machine learning methods and metrics using diverse drug discovery data sets. Mol. Pharm. 14(12), 4462–4475 (2017)

    Article  Google Scholar 

  83. Kamilaris, A., Prenafeta-Boldú, F.X.: Deep learning in agriculture: a survey. Comput. Electron. Agric. 147, 70–90 (2018)

    Article  Google Scholar 

  84. Wang, G., Ng, T.E., Shaikh, A.: Programming your network at run-time for big data applications. In: Proceedings of the First Workshop on Hot Topics in Software Defined Networks, pp. 103–108 (2012)

  85. Nguyen, K., Wang, K., Bu, Y., Fang, L., Hu, J., Xu, G.: Facade: a compiler and runtime for (almost) object-bounded big data applications. ACM SIGARCH Comput. Arch. News 43(1), 675–690 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

We thank Kayseri Metropolitan Municipality for sharing the images taken with the camera in EXIF format in 2017.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Kayseri University, Kayseri, Turkey

    Murat Taşyürek

Authors
  1. Murat Taşyürek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Murat Taşyürek.

Ethics declarations

Conflict of interest

The author certifies that there is no conflict of interest with any individual/organization for the present work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Taşyürek, M. ODRP: a new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system. Vis Comput 40, 983–1003 (2024). https://doi.org/10.1007/s00371-023-02827-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02827-9

Keywords

Search

Navigation