Skip to main content
SpringerLink
Log in

Artificial intelligence inspired IoT-fog based framework for generating early alerts while train passengers traveling in dangerous states using surveillance videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Train Surfing is an extremely dangerous practice that involves riding on the roof of a moving train. Every year a lot of people especially youths lose their life due to this illegal phenomenon. To bring this phenomenon under control the government must book the train surfers before they could even reach the top of the train. To fulfill this, we need artificial intelligence-based real-time monitoring of the trains. In this paper, we present an artificial intelligence-inspired IoT-Fog-based framework for the detection of susceptible ways of people traveling in trains based on surveillance videos. In this study, a framework consisting of feature extraction, feature expression, and assessment criteria for identifying train surfing is proposed. The proposed framework is not constrained by camera angle and includes guidelines for determining unsafe status. The proposed framework can quickly and accurately identify vulnerable passengers during travel and send out early warnings to concerned authorities. The comparative analysis between the proposed framework and other state-of-the-art algorithms shows that it performs better than most of them with a precision score of 95%. The framework would help authorities apprehend the actual culprits and ensure safer rail transport.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Algorithm 2
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

All the datasets generated and analyzed during the current study are available from the corresponding author upon reasonable request.

Notes

  1. https://github.com/developer0hye/CSPDarknet53

  2. https://link.springer.com/chapter/10.1007/978-3-319-10578-9_23

  3. https://ieeexplore.ieee.org/iel7/8576498/8578098/08579011.pdf

  4. https://machinelearningmastery.com/k-fold-cross-validation/

  5. https://github.com/AlexeyAB/darknet/#user-content-when-should-i-stop-training

References

  1. Afif M, Ayachi R, Pissaloux E, Said Y, Atri M (2020) Indoor objects detection and recognition for an ICT mobility assistance of visually impaired people. Multimed Tools Appl 79(41):31645–31662

    Google Scholar 

  2. Agarwal A, Deshmukh M, Singh M (2020) Object detection framework to generate secret shares. Multimed Tools Appl 79(33):24685–24706

    Google Scholar 

  3. Ahmed I, Jeon G, Chehri A, Hassan MM (2021) Adapting Gaussian YOLOv3 with transfer learning for overhead view human detection in smart cities and societies. Sustain Cities Soc 70:102908

    Google Scholar 

  4. Al-Taleb N, Saqib NA (2020) Attacks detection and prevention systems for IoT networks: A survey. In: International conference on computing and information technology (ICCIT-1441) (pp 1–5). https://doi.org/10.1109/iccit-144147971.2020.9213770

  5. Bianco S, Celona L, Napoletano P, Schettini R (2018) On the use of deep learning for blind image quality assessment. SIViP 12(2):355–362

    Google Scholar 

  6. Bozkurt F (2022) A deep and handcrafted features-based framework for diagnosis of COVID-19 from chest x-ray images. Concurrency Comput Pract Experience 34(5):e6725

    Google Scholar 

  7. Broad R, Cavanagh J, Bello W (2002) Development: the market is not enough. In International political economy (pp 402–414). Routledge

  8. Chen C, Gong W, Hu Y, Chen Y, Ding Y (2017) Learning oriented region-based convolutional neural networks for building detection in satellite remote sensing images. The Int Archives Photogramm Remote Sens Spatial Inf Sci 42:461

    Google Scholar 

  9. Chen W, Huang H, Peng S, Zhou C, Zhang C (2021) YOLO-face: a real-time face detector. Vis Comput 37(4):805–813

    Google Scholar 

  10. da Costa MN (2017) Video-based risk assessment for cyclists. Thesis, Universidade de Lisboa

  11. Dastjerdi AV, Gupta H, Calheiros RN, Ghosh SK, Buyya R (2016) Fog computing: Principles, architectures, and applications. Internet of things (pp 61–75). https://doi.org/10.48550/ARXIV.1601.02752

  12. Deb PK, Misra S, Mukherjee A (2021) Latency-aware horizontal computation offloading for parallel processing in fog-enabled IoT. IEEE Syst J 16(2):2537–2544

    Google Scholar 

  13. Deepa R, Tamilselvan E, Abrar ES, Sampath S (2019) Comparison of YOLO, ssd, faster rcnn for real time tennis ball tracking for action decision networks. In: International conference on advances in computing and communication engineering (ICACCE) (pp 1–4). https://doi.org/10.1109/icacce46606.2019.9079965

  14. Du J (2018) Understanding of object detection based on CNN family and YOLO. J Physics: Conference Series 1004:012029. https://doi.org/10.1088/1742-6596/1004/1/012029

  15. Fang Y, Guo X, Chen K, Zhou Z, Ye Q (2021) Accurate and automated detection of surface knots on sawn timbers using YOLO-V5 model. BioResources 16(3):5390–5406. https://doi.org/10.15376/biores.16.3.5390-5406

  16. Fedunina NY (2016) The principles of psychological prevention of transport accidents (train hitching phenomenon as example). Psychol-Educ Stud 8(1):96–104

    Google Scholar 

  17. Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2009) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Google Scholar 

  18. Fernando N, Loke SW, Rahayu W (2013) Mobile cloud computing: a survey. Futur Gener Comput Syst 29(1):84–106

    Google Scholar 

  19. Gai K, Wu Y, Zhu L, Zhang Z, Qiu M (2019) Differential privacy-based blockchain for industrial internet-of-things. IEEE Trans Industrial Inf 16(6):4156–4165

    Google Scholar 

  20. Ge Z, Liu S, Wang F, Li Z, Sun J (2021) YOLOx: Exceeding YOLO series in 2021. arXiv preprint arXiv:2107.08430. https://doi.org/10.48550/arXiv.2107.08430

  21. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision (pp 1440–1448). https://doi.org/10.1109/ICCV.2015.169

  22. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 580–587). https://doi.org/10.1109/CVPR.2014.81

  23. Gokhale P, Bhat O, Bhat S (2018) Introduction to IOT. Int Adv Res J Sci Eng Technol 5(1):41–44

    Google Scholar 

  24. Gorbenko I, Levanova E, Pushkareva T (2019) Personal determinants of the risky behavior of minors (on the example of “Train Surfing”). In XVI European Congress of Psychology (pp 1063–1063)

  25. Grosser L (2019) Modeling hierarchical OPS labels in multilabel recurrent neural network based document classification. Thesis, Humboldt University of Berlin

  26. Guan Y, Shao J, Wei G, Xie M (2018) Data security and privacy in fog computing. IEEE Netw 32(5):106–111

    Google Scholar 

  27. Guo J, Yuan C, Zhao Z, Feng P, Luo Y, Wang T (2020) Object detector with enriched global context information. Multimed Tools Appl 79(39):29551–29571

    Google Scholar 

  28. Gupta H, Vahid Dastjerdi A, Ghosh SK, Buyya R (2017) iFogSim: a toolkit for modeling and simulation of resource management techniques in the internet of things, edge and fog computing environments. Softw Pract Experience 47(9):1275–1296

    Google Scholar 

  29. Gupta S, Kumar M, Garg A (2019) Improved object recognition results using SIFT and ORB feature detector. Multimed Tools Appl 78(23):34157–34171. https://doi.org/10.1007/s11042-019-08232-6

  30. Gupta P, Pareek B, Singal G, Rao DV (2021) Edge device based military vehicle detection and classification from UAV. Multimed Tools Appl 1–22

  31. Han XF, Jin JS, Wang MJ, Jiang W, Gao L, Xiao LP (2017) Video fire detection based on Gaussian mixture model and multi-color features. SIViP 11(8):1419–1425

    Google Scholar 

  32. Haq EU, Jianjun H, Li K, Haq HU (2020) Human detection and tracking with deep convolutional neural networks under the constrained of noise and occluded scenes. Multimed Tools Appl 79(41):30685–30708

    Google Scholar 

  33. He C, Shah AD, Tang Z, Sivashunmugam DFN, Bhogaraju K, Shimpi M, Avestimehr S (2021) FedCV: A federated learning framework for diverse computer vision tasks. arXiv preprint arXiv:2111.11066. https://doi.org/10.48550/arXiv.2111.11066

  34. Hesselink A (2008) Train surfing: a new phenomenon in South Africa? Acta Criminol Afr J Criminol Victimology 2008(sed-1):117–130

    Google Scholar 

  35. Horzyk A, Ergün E (2020) YOLOv3 precision improvement by the weighted centers of confidence selection. In: 2020 international joint conference on neural networks (IJCNN) (pp 1–8). https://doi.org/10.1109/ijcnn48605.2020.9206848

  36. Huang R, Pedoeem J, Chen C (2018) YOLO-LITE: A real-time object detection algorithm optimized for non-GPU computers. In: 2018 IEEE international conference on big data (big data) (pp 2503–2510). https://doi.org/10.1109/BigData.2018.8621865

  37. Jha S, Seo C, Yang E, Joshi GP (2021) Real time object detection and trackingsystem for video surveillance system. Multimed Tools Appl 80:3981–3996

    Google Scholar 

  38. Jiao L, Zhang F, Liu F, Yang S, Li L, Feng Z, Qu R (2019) A survey of deep learning-based object detection. IEEE Access 7:128837–128868

    Google Scholar 

  39. Jin Z, Qu P, Sun C, Luo M, Gui Y, Zhang J, Liu H (2021) DWCA-YOLOv5: An improve single shot detector for safety helmet detection. J Sens, 2021:1–12. https://doi.org/10.1155/2021/4746516

  40. Jindal N, Singh K (2014) Image and video processing using discrete fractional transforms. SIViP 8(8):1543–1553

    Google Scholar 

  41. Kaarmukilan SP, Poddar SKAT (2020) FPGA based deep learning models for object detection and recognition comparison of object detection comparison of object detection models using FPGA. In: 2020 fourth international conference on computing methodologies and communication (ICCMC) (pp 471–474). https://doi.org/10.1109/iccmc48092.2020.iccmc-00088

  42. Kahlon GS, Singh H, Saini M, Kaur S (2023) An intelligent framework to detect and generate alert while cattle lying on road in dangerous states using surveillance videos. Multimed Tools Appl 1–19

  43. Kaur H, Sood SK (2019) Energy-efficient IoT-fog-cloud architectural paradigm for real-time wildfire prediction and forecasting. IEEE Syst J 14(2):2003–2011

    Google Scholar 

  44. Kempen A (2019) Train surfing-chasing death on moving trains. Servamus Community-Based Safety Secur Mag 112(1):22–25

    Google Scholar 

  45. Kontoghiorghe CN (2021) Train accidents: Orthopaedic injury and management at Groote Schuur hospital, Cape Town, South Africa. Thesis, University of Cape Town

  46. Shreyamsha Kumar BK (2015) Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 9(5):1193–1204. https://doi.org/10.1007/s11760-013-0556-9

  47. Kumar A, Kalia A, Verma K, Sharma A, Kaushal M (2021) Scaling up face masks detection with YOLO on a novel dataset. Optik 239:166744

    Google Scholar 

  48. Lampert CH, Blaschko MB, Hofmann T (2008) Beyond sliding windows: object localization by efficient subwindow search. In 2008 IEEE conference on computer vision and pattern recognition (pp 1–8). IEEE

  49. Lee J, Hwang KI (2022) YOLO with adaptive frame control for real-time object detection applications. Multimed Tools Appl 81:36375–36396. https://doi.org/10.1007/s11042-021-11480-0

  50. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision. Springer, Cham, pp 21–37

    Google Scholar 

  51. Lu J, Yan WQ, Nguyen M (2018) Human behaviour recognition using deep learning. In 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS) (pp 1–6). IEEE

  52. Lumenta DB, Vierhapper MF, Kamolz LP, Keck M, Frey M (2011) Train surfing and other high voltage trauma: Differences in injury-related mechanisms and operative outcomes after fasciotomy, amputation and soft-tissue coverage. Burns 37(8):1427–1434. https://doi.org/10.1016/j.burns.2011.07.016

  53. Macek K (2008) Pareto principle in datamining: an above-average fencing algorithm. Acta Polytechnica 48(6)

  54. Maiti P, Shukla J, Sahoo B, Turuk AK (2018) Mathematical modeling of QoS-Aware fog computing architecture for iot services. Advances in Intelligent Systems and Computing, pp 13–21. https://doi.org/10.1007/978-981-13-1501-5_2

  55. Malone K (2005) Train surfing: It’s like bungee jumping without a rope. Sexual Sport Culture Risk 6:154

    Google Scholar 

  56. McLaughlin N, Del Rincon JM, Miller P (2016) Recurrent convolutional network for video-based person re-identification. In proceedings of the IEEE conference on computer vision and pattern recognition (pp 1325-1334)

  57. Murthy P, Bhattacharyya S (2001) Shared buffer implementations of signal processing systems using lifetime analysis techniques. IEEE Trans Computer-Aid Design of Integrated Circuits and Systems 20(2):177–198. https://doi.org/10.1109/43.908427

  58. Narejo S, Pandey B, Esenarro Vargas D, Rodriguez C, Anjum MR (2021) Weapon detection using YOLO V3 for smart surveillance system. Math Probl Eng 2021:1–9

    Google Scholar 

  59. Negash B, Rahmani AM, Liljeberg P, Jantsch A (2018) Fog computing fundamentals in the internet-of-things. In: Fog computing in the internet of things. Springer, Cham, pp 3–13

    Google Scholar 

  60. Oliveira LC, Fox C, Birrell S, Cain R (2019) Analysing passengers’ behaviours when boarding trains to improve rail infrastructure and technology. Robot Comput Integr Manuf 57:282–291

    Google Scholar 

  61. Padilla R, Netto SL, da Silva EA (2020) A survey on performance metrics for object-detection algorithms. In 2020 international conference on systems, signals and image processing (IWSSIP) (pp 237–242). IEEE

  62. Pavlo A, Paulson E, Rasin A, Abadi DJ, DeWitt DJ, Madden S, Stonebraker M (2009) A comparison of approaches to large-scale data analysis. In: Proceedings of the 2009 ACM SIGMOD international conference on management of data (pp 165–178). https://doi.org/10.1145/1559845.1559865

  63. Pouyanfar S, Sadiq S, Yan Y, Tian H, Tao Y, Reyes MP, Shyu ML, Chen SC, Iyengar SS (2018) A survey on deep learning. ACM Comput Surv 51(5):1–36. https://doi.org/10.1145/3234150

  64. Prabhu VU, Birhane A (2020) Large image datasets: a pyrrhic win for computer vision?. arXiv preprint arXiv:2006.16923

  65. Rajagopal A, Joshi GP, Ramachandran A, Subhalakshmi RT, Khari M, Jha S, Shankar K, You J (2020) A deep learning model based on multi-objective particle swarm optimization for scene classification in unmanned aerial vehicles. IEEE Access 8:135383–135393. https://doi.org/10.1109/access.2020.3011502

  66. Rashmi M, Ashwin TS, Guddeti RMR (2021) Surveillance video analysis for student action recognition and localization inside computer laboratories of a smart campus. Multimed Tools Appl 80(2):2907–2929

    Google Scholar 

  67. Rathore S, Kwon BW, Park JH (2019) BlockSecIoTNet: Blockchain-based decentralized security architecture for IoT network. J Netw Comput Appl 143:167–177

    Google Scholar 

  68. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In proceedings of the IEEE conference on computer vision and pattern recognition (pp 779-788)

  69. Ristić-Durrant D, Franke M, Michels K (2021) A review of vision-based on-board obstacle detection and distance estimation in railways. Sensors 21(10):3452

    Google Scholar 

  70. Rosenberg C, Hebert M, Schneiderman H (2005) Semi-supervised self-training of object detection models

  71. Sanal Kumar KP, Bhavani R (2020) Human activity recognition in egocentric video using HOG, GiST and color features. Multimed Tools Appl 79(5–6):3543–3559. https://doi.org/10.1007/s11042-018-6034-1

  72. Santa J, Toledo-Moreo R, Zamora-Izquierdo MA, Ubeda B, Gomez-Skarmeta AF (2010) An analysis of communication and navigation issues in collision avoidance support systems. Transport Res Part C: Emerg Technol 18(3):351–366

    Google Scholar 

  73. Sengar SS, Mukhopadhyay S (2017) Moving object detection based on frame difference and W4. SIViP 11(7):1357–1364

    Google Scholar 

  74. Sharma V, Mir RN (2020) A comprehensive and systematic look up into deep learning based object detection techniques: a review. Comput Sci Rev 38:100301

    MathSciNet  Google Scholar 

  75. Singh VP, Srivastava R (2018) Improved image retrieval using fast colour-texture features with varying weighted similarity measure and random forests. Multimed Tools Appl 77:14435–14460

    Google Scholar 

  76. Singh S, Ahuja U, Kumar M, Kumar K, Sachdeva M (2021) Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment. Multimed Tools Appl 80:19753–19768

    Google Scholar 

  77. Strauch H, Wirth I, Geserick G (1998) Fatal accidents due to train surfing in Berlin. Forensic Sci Int 94(1–2):119–127

    Google Scholar 

  78. Trees K (2017) Response and response-ability to the death of others who are vulnerable. Text: J Writ Writ Courses 45:1–10

    Google Scholar 

  79. Tsakanikas V, Dagiuklas T (2018) Video surveillance systems-current status and future trends. Comput Electr Eng 70:736–753

    Google Scholar 

  80. van der Klashorst E, Cyrus K (2012) Train surfing: apposite recreation provision as alternative to adolsecnt risk-taking and sensation-seeking behaviour. J Sci Med Sport 15:S318

    Google Scholar 

  81. Vigil MA, Barhanpurkar MM, Anand NR, Soni Y, Anand A (2019) EYE SPY face detection and identification using YOLO. In 2019 international conference on smart systems and inventive technology (ICSSIT) (pp 105–110). IEEE

  82. Wang M, Cai H, Zhou J, Gong M (2021) Interlayer and intralayer scale aggregation for scale-invariant crowd counting. Neurocomputing 441:128–137. https://doi.org/10.1016/j.neucom.2021.01.112

  83. Wang H, Xu Y, He Y, Cai Y, Chen L, Li Y, … Li Z (2022) YOLOv5-fog: a multiobjective visual detection algorithm for fog driving scenes based on improved YOLOv5. IEEE Trans Instrum Meas 71:1–12

    Google Scholar 

  84. Xiao Y, Tian Z, Yu J, Zhang Y, Liu S, Du S, Lan X (2020) A review of object detection based on deep learning. Multimed Tools Appl 79(33):23729–23791

    Google Scholar 

  85. Xie X, Cheng G, Wang J, Yao X, Han J (2021) Oriented R-CNN for object detection. In proceedings of the IEEE/CVF international conference on computer vision (pp 3520-3529)

  86. Yao J, Qi J, Zhang J, Shao H, Yang J, Li X (2021) A real-time detection algorithm for kiwifruit defects based on YOLOv5. Electron 10(14):1711. https://doi.org/10.3390/electronics10141711

  87. Zhang P, Sun B, Ma R, Li A (2019) A novel visualization malware detection method based on Spp-net. In 2019 IEEE 5th international conference on computer and communications (ICCC) (pp 510–514). IEEE

  88. Zhang Y, Song C, Zhang D (2022) Small-scale aircraft detection in remote sensing images based on faster-RCNN. Multimed Tools Appl 81(13):18091–18103. https://doi.org/10.1007/s11042-022-12609-5

  89. Zhao J, Li C, Xu Z, Jiao L, Zhao Z, Wang Z (2021) Detection of passenger flow on and off buses based on video images and YOLO algorithm. Multimed Tools Appl 81:4669–4692. https://doi.org/10.1007/s11042-021-10747-w

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Technology, Guru Nanak Dev University, Amritsar, India

    Munish Saini, Eshan Sengupta & Harnoor Singh

Authors
  1. Munish Saini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eshan Sengupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Harnoor Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have contributed equally.

Corresponding author

Correspondence to Munish Saini.

Ethics declarations

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saini, M., Sengupta, E. & Singh, H. Artificial intelligence inspired IoT-fog based framework for generating early alerts while train passengers traveling in dangerous states using surveillance videos. Multimed Tools Appl 83, 13613–13635 (2024). https://doi.org/10.1007/s11042-023-16107-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16107-0

Keywords

Search

Navigation