Multimedia Tools and Applications

, Volume 78, Issue 5, pp 5791–5818 | Cite as

A scene image classification technique for a ubiquitous visual surveillance system

  • Maryam Asadzadeh Kaljahi
  • Shivakumara Palaiahnakote
  • Mohammad Hossein AnisiEmail author
  • Mohd Yamani Idna IdrisEmail author
  • Michael Blumenstein
  • Muhammad Khurram Khan


The concept of smart cities has quickly evolved to improve the quality of life and provide public safety. Smart cities mitigate harmful environmental impacts and offences and bring energy-efficiency, cost saving and mechanisms for better use of resources based on ubiquitous monitoring systems. However, existing visual ubiquitous monitoring systems have only been developed for a specific purpose. As a result, they cannot be used for different scenarios. To overcome this challenge, this paper presents a new ubiquitous visual surveillance mechanism based on classification of scene images. The proposed mechanism supports different applications including Soil, Flood, Air, Plant growth and Garbage monitoring. To classify the scene images of the monitoring systems, we introduce a new technique, which combines edge strength and sharpness to detect focused edge components for Canny and Sobel edges of the input images. For each focused edge component, a patch that merges nearest neighbor components in Canny and Sobel edge images is defined. For each patch, the contribution of the pixels in a cluster given by k-means clustering on edge strength and sharpness is estimated in terms of the percentage of pixels. The same percentage values are considered as a feature vector for classification with the help of a Support Vector Machine (SVM) classifier. Experimental results show that the proposed technique outperforms the state-of-the-art scene categorization methods. Our experimental results demonstrate that the SVM classifier performs better than rule and template-based methods.


Ubiquitous visual surveillance Edge strength Sharpness K-means clustering Focused edges Image classification SVM classifier 



This research work was supported by the Faculty of Computer Science and Information Technology, University of Malaya under a special allocation of Post Graduate Funding for the RP036B-15AET project. The authors also extend their appreciation to the Dean of Scientific Research at King Saud University for funding this work through Research Group Number (RGP-288). The authors convey special thanks to Sangheeta Roy, Faculty of Computer Science and Information Technology, University of Malaya for her help in implementing existing classification methods and conducting experiments for the comparative study.


  1. 1.
    Addabbo AD, Refice A, Pasquariello G, Lovergine FP, Capolongo D, Manfreda S (2016) A Bayesian network for flood detection combining SAR imagery and ancillary data. IEEE Trans Geosci Remote Sens 3612–3625Google Scholar
  2. 2.
    Afsharinejad A, Davy A, Jennings B, Brennan C (2016) Performance analysis of plant monitoring nanosensor networks at THz frequencies. IEEE Internet Things J 59–69CrossRefGoogle Scholar
  3. 3.
    Arabe SB, Gao X, Wang B, Yang F, Brost V (2014) Multi-kernel implicit curve evolution for selected texture region segmentation in VHR satellite images. IEEE Trans Geosci Remote Sens 5183–5192Google Scholar
  4. 4.
    Bai S (2017) Growing random forest on deep convolutional neural networks for scene categorization. Expert Syst Appl 279–287CrossRefGoogle Scholar
  5. 5.
    Bar, E. O., & Trivedi, M. M. (2017) Are all objects equal? Deep spatio-temporal importance prediction in driving videos”, Patt Recog 425–436Google Scholar
  6. 6.
    Bastawesy BE, Ali RR, Deocampo DM, Baroudi MSA (2012) Detection and assessment of the waterlogging in the dryland drainage basins using remote sensing and GIS techniques. IEEE J Select Topics Appl Earth Observ Remote Sens 1564–1571Google Scholar
  7. 7.
    Bosch A, Zisserman A, Munoz X (2008) Scene classification using a hybrid generative/discriminative approach. IEEE Trans Pattern Anal Mach Intell 30(4):712–727CrossRefGoogle Scholar
  8. 8.
    Davis TW, Liang X, Kuo CM, Liang Y (2012). Analysis of power characteristics for sap flow, soil moisture and soil water potential sensors in wireless sensor networking systems. IEEE Sens J 1933–1945CrossRefGoogle Scholar
  9. 9.
    Dornaika F, Moujahid A, Merabt YE, Ruichek Y (2016) Building detection from orthophotos using a machine learning approach: an empirical study on image segmentation and descriptors. Expert System with Applications 58:130–142CrossRefGoogle Scholar
  10. 10.
    Du V, Ling H (2016) Dynamic scene classification using redundant spatial scenelets. IEEE Trans Cybernet 2156–2165CrossRefGoogle Scholar
  11. 11.
    Dunlop H (2010) Scene classification and video via semantic segmentation. In Proc Int Workshop Comput Vis Patt Recog Workshop 72–79Google Scholar
  12. 12.
  13. 13.
    Guerrero JM, Guijarro M, Montalvo M, Romeo J, Emmi L, Ribeiro A, Pajares G (2013) Automatic expert system based on images for accuracy crop row detection in maize fields. Exp Syst Appl 656–664CrossRefGoogle Scholar
  14. 14.
    Hastie T, Tibshirani R (1998) Classification by pairwise coupling. Ann Stat 26(2):451–471MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hayat M, Khan SH, Bennamoun M, An S (2016) A spatial layout and scale invariant feature representation for Indoor scene classification. IEEE Trans Image Proc 4289–4841Google Scholar
  16. 16.
    Jang SW, Cha SH (2014) An approach to segmenting initial object movements in visual sensor networks. Int J Distrib Sensor NetwGoogle Scholar
  17. 17.
    Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei LF (2014) Large-scale video classification with convolutional neural networks. In Proc Int Conf Comput Vis Patten Recog 1725–1732Google Scholar
  18. 18.
    Li Z, Zheng J (2015) Edge preserving decomposition based single image haze removal. IEEE Trans Image Proc 5432–5441MathSciNetCrossRefGoogle Scholar
  19. 19.
    Liu J, Chen C, Zhu Y, Liu W, Metaxas DN (2016) Video classification via weakly supervised sequence modeling. Comput Vis Image Understand 79–87CrossRefGoogle Scholar
  20. 20.
    Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE (2017) A survey of deep neural network architecture and their applications. Neurocomputing 11–26CrossRefGoogle Scholar
  21. 21.
    Longhi, S., Marzioni, D., Alidori, E. & Buo, G. D. (2012) Solid waste management architecture using wireless senor networks technology. In Proc Int Conf New Technol Mobil Sec 1–5Google Scholar
  22. 22.
    Lu X, Manduchi R (2011) Fast image motion segmentation for surveillance applications. Image Vis Comput 104–116CrossRefGoogle Scholar
  23. 23.
    Messer H, Sendik O (2014) A new approach to precipitation monitoring: A critical survey of technologies and challenges. IEEE Sig Proc Mag 110–122Google Scholar
  24. 24.
    Nirmala DE, Vignesh RK, Vaidehi V (2013). Multimodal image fusion in visual sensor networks. In Proc. International Conference on Electronics, Computing and Communication, 1–6Google Scholar
  25. 25.
    Nogueira, K., Penatti, O. A. B., & Santos, J. A. D. (2017) Towards better exploiting convolutional neural networks for remote sensing scene classification. Patt Recogn 539–556Google Scholar
  26. 26.
    Olive. A., Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope (2001) Int J Comput Vis 42(3):145–175CrossRefGoogle Scholar
  27. 27.
    Qin L, Shivakumara P, Lu T, Pal V, Tan CL (2016) Video scene text frames categorization for text detection and recognition. In Proc Int Conf Patt Recog 3875–3880Google Scholar
  28. 28.
    Rao VSH, Kumar MN (2012) A new intelligence based approach for computer aided diagnosis of dengue fever. IEEE Trans Info Technol Biomed 112–118MathSciNetCrossRefGoogle Scholar
  29. 29.
    Shaban KB, Kadri A, Rezk E (2016) Urban air pollution monitoring system with forecasting models. IEEE Sens J 2598–2606Google Scholar
  30. 30.
    Sharma A, Sankar PK (2016) Adapting off-the-self CNNs for word spotting and recognition. In Proc Int Conf Document Anal Recog 986–990Google Scholar
  31. 31.
    Shou Z, Wang D, Chang SF (2016) Temporal action localization in untrimmed videos via multi-stage CNNs. In Proc Int Conf Comput Vis Patt Recog 1049–1058Google Scholar
  32. 32.
    Sun X, Liu Z, Hu Y, Zhang L, Zimmermann R (2018) Perceptual multi-channel visual feature fusion for scene categorization. Inf Sci 429:37–48MathSciNetCrossRefGoogle Scholar
  33. 33.
    Tian D, Sun H, Vetro A (2016) Keypoint trajectory coding on compact descriptor for video analysis. In Proc Int Conf Image Proc 171–175Google Scholar
  34. 34.
    Wu Z, Fu Y, Jiang YG, Sigal L (2016) Harnessing object and scene semantics for large-scale video understanding”, In Proc. Int Conf Comput Vis Patt Recog 3112–3121Google Scholar
  35. 35.
    Xie L, Tian Q, Zhang B (2016) Simple technique make sense: Feature pooling and normalization for image classification, IEEE Trans Circ Syst Video Proc 1251–1264CrossRefGoogle Scholar
  36. 36.
    Yang DW, Park HW (2016) A new shape feature for vehicle classification in thermal video sequences. IEEE Trans Circ Syst Video Proc 1363–1375CrossRefGoogle Scholar
  37. 37.
    Yi X, Eramian M (2016) LBP based segmentation of defocus blur. IEEE Trans Image Proc 1626–1638MathSciNetCrossRefGoogle Scholar
  38. 38.
    Yuan L, Chen F, Zhou L Hu D (2015) Improve scene classification by using feature and kernel combination. Neurocomputing 213–220CrossRefGoogle Scholar
  39. 39.
    Zhang X, Feng X, Wang W, Xue W (2013) Edge strength similarity for image quality assessment. IEEE Trans Image Proc 319–322CrossRefGoogle Scholar
  40. 40.
    Zhou Q, Zheng B, Zhu W, Latecki LJ (2016) Multi-scale context for scene labeling via flexible segmentation graph. Patt Recog 312–324CrossRefGoogle Scholar
  41. 41.
    Zhu J, Wu T, Zhu SC, Yang X, Zhang W (2016) A reconfigurable tangram model for scene representation and categorization. IEEE Trans Image Proc 150–166MathSciNetCrossRefGoogle Scholar
  42. 42.
    Zuo Z, Wang G, Shuai B, Zhao L, Yang Q (2015) Exemplar based deep discriminative and sharable feature learning for scene image classification. Patt Recog 3004–3015CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Computer Science and Information TechnologyUniversity of MalayaKuala LumpurMalaysia
  2. 2.School of Computer Science and Electronic EngineeringUniversity of EssexColchesterUK
  3. 3.School of softwareUniversity of Technology SydneySydneyAustralia
  4. 4.Center of Excellence in Information AssuranceKing Saud UniversityRiyadhSaudi Arabia

Personalised recommendations