Skip to main content

Advertisement

SpringerLink
Log in

PotSpot: Participatory sensing based monitoring system for pothole detection using deep learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Proper maintenance of roads is an extremely complex task and also an important issue all over the world. One of the most critical road monitoring and maintenance activities is the detection of road anomalies such as potholes. Identification of potholes is necessary to avoid road accidents, prevent damage of vehicles, enhance travelling comforts, etc. Although maintenance of roads is considered to be a serious issue by the authorities over the years, lack of proper detection and mapping of road potholes makes the issue more severe. To overcome this problem, an end-to-end system called PotSpot is built for real-time detection, monitoring, and spatial mapping of potholes across the city A Convolutional Neural Network (CNN) model is proposed and evaluated on real-world dataset for pothole detection. Additionally, real-time pothole-marked maps are generated with the help of Google Maps API (Application Programming Interface). To provide an end-to-end service through this system, both the pothole detection and pothole mapping are integrated through an android application. The proposed model is also compared with six baselines namely Artificial Neural Network (ANN), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and three pre-trained CNN models InceptionV3, VGG19 and VGG16 in terms of performance metrics to verify its effectiveness. The proposed model achieves better accuracy (≈ 97.6 %) as compared to the above-mentioned baseline methods. It is also observed that the Area Under the Curve (AUC) value for the proposed pothole detection model (AUC= 0.97) is higher than the baseline methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://www.tensorflow.org/lite/guide/android

  2. https://keras.io/api/optimizers/adam/

References

  1. Ai D, Jiang G, Kei LS, Li C (2018) Automatic pixel-level pavement crack detection using information of multi-scale neighborhoods. IEEE Access 6:24,452–24,463

    Article  Google Scholar 

  2. Albawi S, Mohammed TA, Al-Zawi S (2017) Understanding of a convolutional neural network. In: 2017 International conference on engineering and technology (ICET). IEEE, pp 1–6

  3. Ali A, Zhu Y, Zakarya M (2021) A data aggregation based approach to exploit dynamic spatio-temporal correlations for citywide crowd flows prediction in fog computing. Multimed Tools Appl:1–33

  4. All about Android Platforms. [Online] Available: https://developer.android.com/about

  5. Anaissi A, Khoa NLD, Rakotoarivelo T, Alamdari MM, Wang Y (2019) Smart pothole detection system using vehicle-mounted sensors and machine learning. J Civ Struct Health Monitor 9(1):91–102

    Article  Google Scholar 

  6. Anand S, Gupta S, Darbari V, Kohli S (2018) Crack-pot: Autonomous road crack and pothole detection. In: 2018 Digital image computing: Techniques and applications (DICTA). IEEE, pp 1–6

  7. Arya D, Maeda H, Ghosh SK, Toshniwal D, Mraz A, Kashiyama T, Sekimoto Y (2020) Transfer learning-based road damage detection for multiple countries. arXiv:2008.13101

  8. Bhatia Y, Rai R, Gupta V, Aggarwal N, Akula A et al (2019) Convolutional neural networks based potholes detection using thermal imaging Journal of King Saud University-Computer and Information Sciences

  9. Bhatt U, Mani S, Xi E, Kolter JZ (2017) Intelligent pothole detection and road condition assessment. arXiv:1710.02595

  10. Burke JA, Estrin D, Hansen M, Parker A, Ramanathan N, Reddy S, Srivastava MB (2006) Participatory sensing

  11. Butt RA, Faheem M, Arfeen A, Ashraf MW, Jawed M (2019) Machine learning based dynamic load balancing dwba scheme for twdm pon. Opt Fiber Technol 52(101):964

    Google Scholar 

  12. Cabral FS, Pinto M, Mouzinho FA, Fukai H, Tamura S (2018) An automatic survey system for paved and unpaved road classification and road anomaly detection using smartphone sensor. In: 2018 IEEE International conference on service operations and logistics, and informatics (SOLI). IEEE, pp 65–70

  13. Chen K, Lu M, Fan X, Wei M, Wu J (2011) Road condition monitoring using on-board three-axis accelerometer and gps sensor. In: 2011 6Th international ICST conference on communications and networking in China (CHINACOM). IEEE, pp 1032–1037

  14. Chen H, Yao M, Gu Q (2020) Pothole detection using location-aware convolutional neural networks. Int J Mach Learn Cybern 11(4):899–911

    Article  Google Scholar 

  15. Chitale PA, Kekre KY, Shenai HR, Karani R, Gala JP (2020) Pothole detection and dimension estimation system using deep learning (yolo) and image processing. In: 2020 35Th international conference on image and vision computing New Zealand (IVCNZ). IEEE, pp 1–6

  16. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on computer vision and pattern recognition. IEEE, pp 248–255

  17. Detho A, Samo SR, Mukwana KC, Samo KA, Siyal AA (2018) Evaluation of road traffic accidents (rtas) on hyderabad karachi m-9 motorway section. Eng Technol Appl Sci Res 8(3):2875–2878

    Article  Google Scholar 

  18. Dhiman A, Klette R (2019) Pothole detection using computer vision and learning. IEEE Transactions on Intelligent Transportation Systems

  19. Firebase Realtime Database. [online] Available: https://firebase.google.com/docs/database

  20. Ghosh A, Kumari K, Kumar S, Saha M, Nandi S, Saha S (2019) Noiseprobe: Assessing the dynamics of urban noise pollution through participatory sensing. In: 2019 11Th international conference on communication systems & networks (COMSNETS). IEEE, pp 451–453

  21. Google Maps Platform Documentation. [online] Available: https://developers.google.com/maps/documentation

  22. Gopalakrishnan K, Khaitan SK, Choudhary A, Agrawal A (2017) Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr Build Mater 157:322–330

    Article  Google Scholar 

  23. Hameed H, Mazhar S, Hassan N (2018) Real-time road anomaly detection, using an on-board data logger. In: 2018 IEEE 87Th vehicular technology conference (VTC spring). IEEE, pp 1–5

  24. Hara K, Saito D, Shouno H (2015) Analysis of function of rectified linear unit used in deep learning. In: 2015 International joint conference on neural networks (IJCNN). IEEE, pp 1–8

  25. Hassan N, Siddiqui I, Mazhar S, Hameed H (2019) Road anomaly classification for low-cost road maintenance and route quality maps. In: 2019 IEEE International conference on pervasive computing and communications workshops (percom workshops). IEEE, pp 645–650

  26. Jokela M, Kutila M, Le L (2009) Road condition monitoring system based on a stereo camera. In: 2009 IEEE 5Th international conference on intelligent computer communication and processing. IEEE, pp 423–428

  27. Kanhere SS (2013) Participatory sensing: Crowdsourcing data from mobile smartphones in urban spaces. In: International conference on distributed computing and internet technology. Springer, pp 19–26

  28. Kar D, Middya AI, Roy S (2019) An approach to detect travel patterns using smartphone sensing. In: 2019 IEEE International conference on advanced networks and telecommunications systems (ANTS). IEEE, pp 1–6

  29. Keras: the Python deep learning API. [Online] Available: https://keras.io/about/

  30. Kwasigroch A, Mikołajczyk A, Grochowski M (2017) Deep neural networks approach to skin lesions classification—a comparative analysis. In: 2017 22Nd international conference on methods and models in automation and robotics (MMAR). IEEE, pp 1069–1074

  31. Li K, Misener JA, Hedrick K (2007) On-board road condition monitoring system using slip-based tyre-road friction estimation and wheel speed signal analysis. Proc Inst Mech Eng Part K: J Multi-body Dyn 221(1):129–146

    Google Scholar 

  32. Lin J, Liu Y (2010) Potholes detection based on svm in the pavement distress image. In: 2010 Ninth international symposium on distributed computing and applications to business, engineering and science. IEEE, pp 544–547

  33. Liu CH, Zhao J, Zhang H, Guo S, Leung KK, Crowcroft J (2016) Energy-efficient event detection by participatory sensing under budget constraints. IEEE Syst J 11(4):2490–2501

    Article  Google Scholar 

  34. Middya AI, Roy S, Dutta J, Das R (2020) Jusense: a unified framework for participatory-based urban sensing system. Mob Netw Appl:1–26

  35. Motwani P, Sharma R (2020) Comparative study of pothole dimension using machine learning, manhattan and euclidean algorithm. Int J Innov Sci Res Technol 5(2):165–170

    Google Scholar 

  36. Nunes DE, Mota VF (2019) A participatory sensing framework to classify road surface quality. J Internet Serv Appl 10(1):13

    Article  Google Scholar 

  37. Pan Y, Zhang X, Sun M, Zhao Q (2017) Object-based and supervised detection of potholes and cracks from the pavement images acquired by uav. Int Arch Photogramm, Remote Sens Spatial Inf Sci:42

  38. Pereira V, Tamura S, Hayamizu S, Fukai H (2018) A deep learning-based approach for road pothole detection in timor leste. In: 2018 IEEE International conference on service operations and logistics, and informatics (SOLI). IEEE, pp 279–284

  39. Ranjbar S, Nejad FM, Zakeri H (2021) An image-based system for pavement crack evaluation using transfer learning and wavelet transform. Int J Pavement Res Technol 14(4):437–449

    Article  Google Scholar 

  40. Raza B, Aslam A, Sher A, Malik AK, Faheem M (2020) Autonomic performance prediction framework for data warehouse queries using lazy learning approach. Appl Soft Comput:106216

  41. Road traffic injuries (2020) [online] Available: https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries

  42. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  43. Sarma S, Kandhway K, Kotnis B, Kuri J (2016) Urban monitoring using participatory sensing: an optimal budget allocation approach. In: 2016 8Th international conference on communication systems and networks (COMSNETS). IEEE, pp 1–6

  44. Sehgal A, Kehtarnavaz N (2019) Guidelines and benchmarks for deployment of deep learning models on smartphones as real-time apps. Mach Learn Knowl Extract 1(1):450–465

    Article  Google Scholar 

  45. Silvister S, Komandur D, Kokate S, Khochare A, More U, Musale V, Joshi A (2019) Deep learning approach to detect potholes in real-time using smartphone. In: 2019 IEEE Pune section international conference (punecon). IEEE, pp 1–4

  46. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  47. Sun M, Song Z, Jiang X, Pan J, Pang Y (2017) Learning pooling for convolutional neural network. Neurocomputing 224:96–104

    Article  Google Scholar 

  48. Suong LK, Kwon J (2018) Detection of potholes using a deep convolutional neural network. J UCS 24(9):1244–1257

    Google Scholar 

  49. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  50. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  51. Takahashi J, Kobana Y, Isoyama N, Tobe Y, Lopez G (2018) Ykob: Participatory sensing-based road condition monitoring using smartphones worn by cyclist. Electron Commun Jpn 101(4):3–14

    Article  Google Scholar 

  52. The official home of the Python Programming Language. [Online] Available: https://www.python.org/

  53. Verster T, Fourie E (2018) The good, the bad and the ugly of south african fatal road accidents. S Afr J Sci 114(7-8):63–69

    Google Scholar 

  54. Wang H, Fan R, Sun Y, Liu M (2020) Applying surface normal information in drivable area and road anomaly detection for ground mobile robots. arXiv:2008.11383

  55. Ye W, Jiang W, Tong Z, Yuan D, Xiao J (2021) Convolutional neural network for pothole detection in asphalt pavement. Road Mater Pavement Des 22(1):42–58

    Article  Google Scholar 

  56. Yik YK, Alias NE, Yusof Y, Isaak S (2021) A real-time pothole detection based on deep learning approach. In: Journal of physics: Conference series, vol 1828. IOP publishing, pp 012001

Download references

Acknowledgements

This research work is supported by the project entitled- “Participatory and Realtime Pollution Monitoring System For Smart City”, funded by Higher Education, Science & Technology and Biotechnology, Department of Science & Technology, Government of West Bengal, India.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India

    Susmita Patra, Asif Iqbal Middya & Sarbani Roy

Authors
  1. Susmita Patra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Asif Iqbal Middya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sarbani Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarbani Roy.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patra, S., Middya, A.I. & Roy, S. PotSpot: Participatory sensing based monitoring system for pothole detection using deep learning. Multimed Tools Appl 80, 25171–25195 (2021). https://doi.org/10.1007/s11042-021-10874-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10874-4

Keywords

Search

Navigation