Skip to main content
SpringerLink
Log in

A Critical Study on Suspicious Object Detection with Images and Videos Using Machine Learning Techniques

  • Review Article
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Security has emerged as a significant concern in today's world, particularly in public places such as railway stations, airports, shopping malls, and crowded areas. Unattended object detection plays a crucial role in bolstering video surveillance systems. This paper presents an innovative approach to detect suspicious objects in images and real-time videos through the Internet of Things (IoT). Various object detection techniques are employed and compared for their performance. The study aims to enhance security measures by providing real-time monitoring and analysis. Various deep learning models, such as Faster-RCNN, Mask-RCNN, and Yolo, are compared for their effectiveness in detecting suspicious objects in images and real-time videos through the Internet of Things (IoT). The study highlights challenges like illumination changes, occlusion, noise, poor resolution, and real-time processing complexities. While machine learning methods show promising results, they still struggle with simultaneous recognition of multiple activities, leading to lower accuracy. The research also addresses current limitations, paving the way for improving the proposed surveillance solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Meng Z, Zhang M, Wang H (2020) CNN with Pose segmentation for suspicious object detection in MMW security images, Sensors 2020, Vol. 20, Page 4974, 20(17): 4974, https://doi.org/10.3390/S20174974.

  2. Yang Xi, Tan Wu, Zhang L, Yang D, Wang N, Song B, Gao X. CNN with spatio-temporal information for fast suspicious object detection and recognition in THz security images. Signal Process. 2019;160:202–14.

    Article  Google Scholar 

  3. Yang X, Wei Z, Wang N, Song B, Gao X. A novel deformable body partition model for MMW suspicious object detection and dynamic tracking. Signal Process. 2020;174: 107627. https://doi.org/10.1016/J.SIGPRO.2020.107627.

    Article  Google Scholar 

  4. Cai W, Li J, Xie Z, Zhao T, Lu K Street object detection based on faster R-CNN, Chinese Control Conf. CCC, vol. 2018-July, pp. 9500–9503, Oct. 2018, https://doi.org/10.23919/CHICC.2018.8482613.

  5. Shivthare KV, Bhujbal PD, Darekar AP, YNN, Suspicious Activity Detection Network for Video Surveillance Using Machine Learning, 6: 2456–0774, 2021, doi: https://doi.org/10.51319/2456-0774.2021.4.0017.

  6. Jain H, Vikram A, Mohana AK, Jain A Weapon Detection using artificial intelligence and deep learning for security applications, Proc. Int. Conf. Electron. Sustain. Commun. Syst. ICESC 2020, pp. 193–198, Jul. 2020, https://doi.org/10.1109/ICESC48915.2020.9155832.

  7. Cristyan Rufino Gil Morales, “Video Analysis to Detect Suspicious Activity Based on Deep Learning - DZone AI”, https://dzone.com/articles/video-analysis-to-detect-suspicious-activity-based Accessed: 2022–04–26.

  8. Singh V, Singh S, Gupta P. Real-time anomaly recognition through CCTV using neural networks. Procedia Comput Sci. 2020;173:254–63. https://doi.org/10.1016/J.PROCS.2020.06.030.

    Article  Google Scholar 

  9. Saba T, Rehman A, Latif R, Fati SM, Raza M, Sharif M. Suspicious activity recognition using proposed deep L4-branched-actionnet with entropy coded ant colony system optimization. IEEE Access. 2021;9:89181–97. https://doi.org/10.1109/ACCESS.2021.3091081.

    Article  Google Scholar 

  10. Ramzan M, Abid A, Ullah Khan H, Mahmood Awan S, Ismail A, Ahmed M, Ilyas M, Mahmood A A review on state-of-the-art violence detection techniques. IEEE Access 7 (2019): 107560–107575.

  11. Tian G, Liu J, Zhao H, Yang W Small object detection via dual inspection mechanism for UAV visual images. Appl Intell (2022): 1–14.

  12. Rani, Shilpa, Deepika Ghai, and Sandeep Kumar. "Object detection and recognition using contour based edge detection and fast R-CNN. Multimedia Tools and Applications (2022): 1–25.

  13. Almahasneh M, Paiement A, Xie X, Aboudarham J. MLMT-CNN for object detection and segmentation in multi-layer and multi-spectral images. Mach Vis Appl. 2022;33:1–15.

    Article  Google Scholar 

  14. Ge Ji, Wang C, Zhang Bo, Changgui Xu, Wen X. Azimuth-sensitive object detection of high-resolution SAR images in complex scenes by using a spatial orientation attention enhancement network. Remote Sensing. 2022;14(9):2198.

    Article  Google Scholar 

  15. Posilović L, Medak D, Milković F, Subašić M, Budimir M, Lončarić S. Deep learning-based anomaly detection from ultrasonic images. Ultrasonics. 2022;124: 106737.

    Article  Google Scholar 

  16. Hirooka K, Md Al Mehedi Hasan, Shin J, Yakin Srizon A Ensembled transfer learning based multichannel attention networks for human activity recognition in still images. IEEE Access 10 (2022): 47051–47062.

  17. Yuan M, Zhang Q, Li Y, Yan Y, Zhu Y. A suspicious multi-object detection and recognition method for millimeter wave sar security inspection images based on multi-path extraction network. Remote Sensing. 2021;13(24):4978.

    Article  Google Scholar 

  18. Tian G, Liu J, Yang W. A dual neural network for object detection in UAV images. Neurocomputing. 2021;443:292–301.

    Article  Google Scholar 

  19. Song Z, Sui H, Hua Li. A hierarchical object detection method in large-scale optical remote sensing satellite imagery using saliency detection and CNN. Int J Remote Sens. 2021;42(8):2827–47.

    Article  Google Scholar 

  20. Bravo DT, Araujo Lima G, Alves WAL, Pessoa Colombo V, Djogbenou L, Vicente Denser Pamboukian S, Capellani Quaresma C, Alves de Araujo S Automatic detection of potential mosquito breeding sites from aerial images acquired by unmanned aerial vehicles. Computers, Environment and Urban Systems 90 (2021): 101692.

  21. Meng Z, Zhang M, Wang H. CNN with pose segmentation for suspicious object detection in MMW security images. Sensors. 2020;20(17):4974.

    Article  Google Scholar 

  22. Javed AR, Jalil Z Byte-level object identification for forensic investigation of digital images. 2020 International Conference on Cyber Warfare and Security (ICCWS). IEEE, 2020.

  23. Chen C, Zhong J, Tan Yi. Multiple-oriented and small object detection with convolutional neural networks for aerial image. Remote Sens. 2019;11(18):2176.

    Article  Google Scholar 

  24. Pang Y, Cao J, Wang J, Han J. JCS-Net: Joint classification and super-resolution network for small-scale pedestrian detection in surveillance images. IEEE Trans Inf Forensics Secur. 2019;14(12):3322–31.

    Article  Google Scholar 

  25. Ali MM, Qaseem MS Strategies and tools for effective suspicious event detection from video: a survey perspective (COVID-19). Contactless Healthcare Facilitation and Commodity Delivery Management During COVID 19 Pandemic. Springer, Singapore, 2022. 79–94.

  26. Pavithra S, Muruganantham B DL Based Automatic Robbery Detection using Video Surveillance in Residential Areas. In: 2022 International Conference on Advances in Computing, Communication and Applied Informatics (ACCAI). IEEE, 2022.

  27. Manikandan VP, Rahamathunnisa U. A neural network aided attuned scheme for gun detection in video surveillance images. Image Vis Comput. 2022;120: 104406.

    Article  Google Scholar 

  28. Jaleel A, Khurshid SK, Mustafa R, Mehmood Aamir K, Tahir M, Ziar A Towards Proactive Surveillance through CCTV Cameras under Edge-Computing and Deep Learning. Mathematical Problems in Engineering 2022 (2022).

  29. Guo, Junpeng, Shengqing Xia, and Chunyi Peng. "VPPlus: Exploring the Potentials of Video Processing for Live Video Analytics at the Edge." 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS). IEEE, 2022.

  30. Nandhini S, Salomiyamary S, Suganthi S Anomaly Detection In Robbery Using A Video Surveillance Survey.

  31. Arthi B, PoornaPushkala K, Arya A, Rajasekhar D Wearable Sensors and Real-Time System for Detecting violence using Artificial Intelligence. In: 2022 International Conference on Advanced Computing Technologies and Applications (ICACTA), pp. 1–5. IEEE, 2022.

  32. Alairaji RM, Aljazaery IA, Haider TH ALRikabi. Abnormal behavior detection of students in the examination hall from surveillance videos. Advanced Computational Paradigms and Hybrid Intelligent Computing. Springer, Singapore, 2022. 113–125.

  33. Wan S, Ding S, Chen C. Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles. Pattern Recogn. 2022;121: 108146.

    Article  Google Scholar 

  34. Dou Z, Ma X, Xie X, Liu H, Guo C. A hybrid method of detecting flame from video stream. IET Image Proc. 2022;16(11):2937–46.

    Article  Google Scholar 

  35. Ren S, Li J, Tianyi Tu, Peng Y, Jiang J. Towards efficient video detection object super-resolution with deep fusion network for public safety. Secur Commun Netw. 2021;2021:1–14.

    Google Scholar 

  36. Khan AR, Saba T, Zeeshan Khan M, Fati SM, Ghani Khan MU (2022) Classification of human's activities from gesture recognition in live videos using deep learning. Concurrency and Computation: Practice and Experience 34(10): e6825.

  37. Salem FG, Ibrahim RH, Ahmed AA, Douma A Detection of suspicious activities of human from surveillance videos. In:2021 IEEE 1st international maghreb meeting of the conference on sciences and techniques of automatic control and computer engineering MI-STA, pp. 794–801. IEEE, 2021.

  38. Kulkarni AS, Naresh E, Swetha M, Kusuma SM Automated System for Detection of Suspicious Activity in Examination Hall. In: 2021 IEEE international conference on electronics, computing and communication technologies (CONECCT), pp. 01–05. IEEE, 2021.

  39. Gorave A, Misra S, Padir O, Patil A, Ladole K "Suspicious activity detection using live video analysis. In: Proceeding of international conference on computational science and applications: ICCSA 2019, pp. 203–214. Singapore: Springer Singapore, 2020.

  40. Kamthe UM, Patil CG Suspicious activity recognition in video surveillance system. In: 2018 Fourth international conference on computing communication control and automation (ICCUBEA). IEEE, 2018.

  41. Dhulekar PA, Gandhe ST, Sawale N, Shinde V, Khute S Surveillance system for detection of suspicious human activities at war field. In: 2018 international conference on advances in communication and computing technology (ICACCT), pp. 357–360. IEEE, 2018.

  42. Singh S, Pandey S, Gupta N, Sharma R, Agarwal M IoT based Criminal Activity Detection (ICAD).

  43. Muvva M, Muvva M, Sharma M, Mukherjee R, Yadav A Border Management Using Internet of Things. In: 2022 8th International conference on advanced computing and communication systems (ICACCS), vol. 1, pp. 6–10. IEEE, 2022.

  44. Rajavel R, Kumar Ravichandran S, Harimoorthy K, Nagappan P, Ramasubramanian Gobichettipalayam K IoT-based smart healthcare video surveillance system using edge computing. J Ambient Intell Human Comput (2022): 1–13.

  45. Arun P, Prajith N, Melvin C, Sreejith SN, Sandesh S. Design and development of an IoT based intelligent multi parameter screening system. Mater Today. 2022;58:7–12.

    Google Scholar 

  46. Gavaskar K, Ragupathy US, Elango S, Ramyadevi M, Preethi S. A novel design and implementation of IoT based real-time ATM surveillance and security system. Adv Comput Intell. 2022;2(1):1.

    Article  Google Scholar 

  47. Kumar A, Abhishek K, Rukunuddin Ghalib M, Shankar A, Cheng X (2022) Intrusion detection and prevention system for an IoT environment. Digital Commun Netw 8(4): 540–551.

  48. Lysenko S, Bobrovnikova K, Kharchenko V, Savenko O. IoT multi-vector cyberattack detection based on machine learning algorithms: traffic features analysis, experiments, and efficiency. Algorithms. 2022;15(7):239.

    Article  Google Scholar 

  49. Prabu AV, Sateesh Kumar G, Rajasoundaran S, Malla PP, Routray S, Mukherjee A (2022) Internet of things‐based deeply proficient monitoring and protection system for crop field. Expert Syst 39(5): e12876.

  50. Gupta BB, Chaudhary P, Chang X, Nedjah N (2022) Smart defense against distributed Denial of service attack in IoT networks using supervised learning classifiers. Comput Electr Eng 98: 107726.

  51. Ullah FUM, Muhammad K, Ul Haq I, Khan N, Asghar Heidari A, Wook Baik S, de Albuquerque VHC AI-assisted edge vision for violence detection in IoT-based industrial surveillance networks. IEEE Transactions on Industrial Informatics 18, no. 8 (2021): 5359–5370.

  52. Gautam A, Singh S. Deep learning based object detection combined with internet of things for remote surveillance. Wireless Pers Commun. 2021;118(4):2121–40.

    Article  Google Scholar 

  53. Vallathan G, John A, Thirumalai C, Mohan SK, Srivastava G, Chun-Wei Lin J (2021) Suspicious activity detection using deep learning in secure assisted living IoT environments. J Supercomput 77: 3242–3260.

  54. Vijayaprabakaran K, Kodidela P, Gurram P. IoT Based Smart Intruder Detection System For Smart Homes. International Journal of Scientific Research in Science and Technology, 8(4), (2021): 49–53.

  55. Yu K, Qi X, Sato T, Wen Z, Katsuyama Y, Tokuda K, Kameyama W, Sato T. Design and performance evaluation of an ai-based w-band suspicious object detection system for moving persons in the iot paradigm. IEEE Access. 2020;8:81378–93.

    Article  Google Scholar 

  56. B Harum N, Faeq Ali M, Zakaria NA, Anawar S (2018) Smart surveillance system using background subtraction technique in IoT application. Int J Adv Comput Sci Appl 9(12).

  57. Xie X, et al. Oriented R-CNN for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

  58. Deng J et al. Voxel r-cnn: Towards high performance voxel-based 3d object detection. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 2. 2021.

  59. Rajjak SSA, Kureshi AK (2021) Multiple-Object Detection and segmentation based on deep learning in high-resolution video using mask-RCNN. Int J Pattern Recognit Artificial Intell 35(13)x: 2150038.

  60. Fan Z et al. (2021) Detection and segmentation of underwater objects from forward-looking sonar based on a modified Mask RCNN. Signal, Image and Video Processing 15(6): 1135–1143.

  61. Yi D, Jinya Su, Chen W-H. Probabilistic faster R-CNN with stochastic region proposing: Towards object detection and recognition in remote sensing imagery. Neurocomputing. 2021;459:290–301.

    Article  Google Scholar 

  62. Körez A, Barışçı N. Object detection with low capacity GPU systems using improved faster R-CNN. Appl Sci. 2019;10(1):83.

    Article  Google Scholar 

  63. Fang W, Wang L, Ren P. Tinier-YOLO: A real-time object detection method for constrained environments. IEEE Access. 2019;8:1935–44.

    Article  Google Scholar 

  64. Junos, Mohamad Haniff, et al. "An optimized YOLO‐based object detection model for crop harvesting system." IET Image Processing 15.9 (2021): 2112–2125

  65. Gawande U, Hajari K, Golhar Y. Real-time deep learning approach for pedestrian detection and suspicious activity recognition. Procedia Computer Science. 2023;218:2438–47.

    Article  Google Scholar 

  66. Indhumathi J, Balasubramanian M, Balasaigayathri B. Real-time video based human suspicious activity recognition with transfer learning for deep learning. Int J Image Graphics Signal Process. 2023;13(1):47.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Rabindranath Tagore University, Bhopal, India

    Prati Dubey & Rakesh Kumar Mittan

Authors
  1. Prati Dubey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rakesh Kumar Mittan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prati Dubey.

Ethics declarations

Conflict of interest

I hereby declare that I have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advanced Computing and Data Sciences” guest edited by Mayank Singh, Vipin Tyagi and P. K. Gupta.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dubey, P., Mittan, R.K. A Critical Study on Suspicious Object Detection with Images and Videos Using Machine Learning Techniques. SN COMPUT. SCI. 5, 505 (2024). https://doi.org/10.1007/s42979-024-02869-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-02869-3

Keywords

Search

Navigation