Multimedia Analysis on User-Generated Content for Safety-Oriented Applications

  • Nikolaos Papadakis
  • Antonios Litke
  • Anastasios DoulamisEmail author
  • Eftychios Protopapadakis
  • Nikolaos Doulamis
Part of the Security Informatics and Law Enforcement book series (SILE)


An important factor that boosts the rapid penetration of smartphone devices is the increasing incorporation of sensors, which have stimulated a new type of content, the so-called user-generated content. The huge amount of media information accumulating every day presents an opportunity to incorporate image analysis methods and applications for a safer and more secure environment for citizens. This chapter proposed an anomaly detection mechanism for video streams, especially from social media. The methodology employs low-level feature extraction over non-overlapping frame patches and density-based clustering. The core idea consists of two steps: cluster the image patches and observe the difference in the number of clusters for successive images. A threshold-based approach triggers the detection mechanism by investigating the change in the number of clusters. The proposed unsupervised approach runs smoothly on ordinary desktop computers and operates in real time. This chapter outlines the approach and underlying methodology together with an evaluation based on YouTube videos depicting car explosions.


Smartphones Video streams Anomaly detection Social media analysis Methodology 


  1. Addington, L. A. (2009). Cops and cameras: Public school security as a policy response to columbine. The American Behavioral Scientist, 52, 1426–1446. Scholar
  2. Agrafiotis, P., Stathopoulou, E. K., Georgopoulos, A., & Doulamis, A. D. (2015). HDR imaging for enhancing people detection and tracking in indoor environments. In VISAPP (Vol. 2, pp. 623–630).Google Scholar
  3. Akrivas, G., Doulamis, N. D., Doulamis, A. D., & Kollias, D. (2000). Scene detection methods for MPEG-encoded video signals. Presented at the 2000 10th Mediterranean Electrotechnical Conference. Information Technology and Electrotechnology for the Mediterranean Countries. Proceedings. MeleCon 2000 (Cat. No.00CH37099) (Vol. 2, pp. 677–680)..
  4. Ankerst, M., Breunig, M. M., Kriegel, H. -P., & Sander, J. (1999). OPTICS: Ordering points to identify the clustering structure. Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, SIGMOD ’99 (pp. 49–60). New York: ACM.
  5. Borden, S. L., & Tew, C. (2007). The role of journalist and the performance of journalism: Ethical lessons from “fake” news (seriously). Journal of Mass Media Ethics, 22, 300–314. Scholar
  6. Brandenburg, K., & Stoll, G. (1994). ISO/MPEG-1 audio: A generic standard for coding of high-quality digital audio. Journal of the Audio Engineering Society, 42, 780–792.Google Scholar
  7. Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.-Y., & Moon, S. (2009). Analyzing the video popularity characteristics of large-scale user generated content systems. IEEE ACM Transactions on Network, 17, 1357–1370. Scholar
  8. Coleman, R. (2004). Images from a Neoliberal City: The state, surveillance and social control. Critical Criminology, 12, 21–42. Scholar
  9. Doulamis, A., & Doulamis, N. (2001). Fuzzy histograms for efficient visual content representation: Application to content-based image retrieval. IEEE International Conference on Multimedia and Expo. ICME 2001.(ICME) (p. 227).
  10. Doulamis, N., & Doulamis, A. (2012). Fast and adaptive deep fusion learning for detecting visual objects. In A. Fusiello, V. Murino, & R. Cucchiara (Eds.), Computer vision—ECCV 2012. Workshops and demonstrations, lecture notes in computer science (pp. 345–354). Berlin: Springer.Google Scholar
  11. Doulamis, A., & Katsaros, G. (2016). 3D modelling of cultural heritage objects from photos posted over the Twitter. Presented at the 2016 IEEE International Conference on Imaging Systems and Techniques (IST) (pp. 389–394).
  12. Doulamis, N. D., Doulamis, A. D., Avrithis, Y., & Kollias, S. D. (1999a). A stochastic framework for optimal key frame extraction from MPEG video databases. Presented at the 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No.99TH8451) (pp. 141–146).
  13. Doulamis, A. D., Doulamis, N. D., Ntalianis, K. S., & Kollias, S. D. (1999b). Unsupervised semantic object segmentation of stereoscopic video sequences. Presented at the Proceedings 1999 International Conference on Information Intelligence and Systems (Cat. No.PR00446) (pp. 527–533).
  14. Doulamis, N. D., Doulamis, A. D., Avrithis, Y. S., Ntalianis, K. S., & Kollias, S. D. (2000a). Efficient summarization of stereoscopic video sequences. IEEE Transactions on Circuits and Systems for Video Technology, 10, 501–517. Scholar
  15. Doulamis, A. D., Doulamis, N. D., & Kollias, S. D. (2000b). On-line retrainable neural networks: Improving the performance of neural networks in image analysis problems. IEEE Transactions on Neural Networks, 11, 137–155. Scholar
  16. Doulamis, A. D., Doulamis, N. D., & Kollias, S. D. (2000c). A fuzzy video content representation for video summarization and content-based retrieval. Signal Processing, 80, 1049–1067. Scholar
  17. Doulamis, A., Doulamis, N., Ntalianis, K., & Kollias, S. (2000d). Efficient unsupervised content-based segmentation in stereoscopic video sequences. International Journal on Artificial Intelligence Tools, 09, 277–303. Scholar
  18. Doulamis, A., Doulamis, N., & Maragos, P. (2001). Generalized multiscale connected operators with applications to granulometric image analysis. Presented at the Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205) (Vol. 3, pp. 684–687).
  19. Doulamis, A., Doulamis, N., Ntalianis, K., & Kollias, S. (2003a). An efficient fully unsupervised video object segmentation scheme using an adaptive neural-network classifier architecture. IEEE Transactions on Neural Networks, 14, 616–630. Scholar
  20. Doulamis, N. D., Doulamis, A. D., & Varvarigou, T. A. (2003b). Adaptive algorithms for interactive multimedia. IEEE Multimedia, 10, 38–47. Scholar
  21. Doulamis, N. D., Doulamis, A. D., Kokkinos, P., & Varvarigos, E. M. (2016). Event detection in twitter microblogging. IEEE Transactions on Cybernetics, 46, 2810–2824. Scholar
  22. Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd, 96(34), 226–231).Google Scholar
  23. Fujii, Y., Maru, K., Kobayashi, K., Yoshiura, N., Ohta, N., Ueda, H., et al. (2010). e-JIKEI Network using e-JIKEI Cameras: Community security using considerable number of cheap stand-alone cameras. Safety Science, 48, 921–925. Scholar
  24. Fyfe, N. R., & Bannister, J. (1996). City watching: Closed circuit television surveillance in public spaces. Area, 28, 37–46.Google Scholar
  25. Gelfand, N., Adams, A., Park, S. H., & Pulli, K. (2010). Multi-exposure imaging on mobile devices. Proceedings of the 18th ACM International Conference on Multimedia, MM ’10 (pp. 823–826). New York: ACM.
  26. Grabner, H., Leistner, C., & Bischof, H. (2008). Semi-supervised on-line boosting for robust tracking. In D. Forsyth, P. Torr, & A. Zisserman (Eds.), Computer vision—ECCV 2008, Lecture Notes in Computer Science (pp. 234–247). Berlin: Springer.Google Scholar
  27. Karamolegkos, P. N., Patrikakis, C. Z., Doulamis, N. D., Vlacheas, P. T., & Nikolakopoulos, I. G. (2009). An evaluation study of clustering algorithms in the scope of user communities assessment. Computers & Mathematcs with Applications, 58, 1498–1519. Scholar
  28. Kokkinos, P. C., Koumoutsos, K., Doulamis, N. D., Varvarigos, E. A., Petrantonakis, D., Kardara, M., Sardis, E., Vekris, A., & Gerontidis, S. (2013). PERIKLIS-electronic democracy in the 21st century using mobile and social network technologies. In EGOV/EPart ongoing research (pp. 242–249). Citeseer.Google Scholar
  29. Kontogianni, G., Stathopoulou, E. K., Georgopoulos, A., & Doulamis, A. (2015). HDR imaging for feature detection on detailed architectural scenes. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XL-5/W4, 325–330.CrossRefGoogle Scholar
  30. Koskela, H. (2000). ‘The gaze without eyes’: Video-surveillance and the changing nature of urban space. Progress in Human Geography, 24, 243–265. Scholar
  31. Kosmopoulos, D. I., Doulamis, N. D., & Voulodimos, A. S. (2012). Bayesian filter based behavior recognition in workflows allowing for user feedback. Computer Vision and Image Understanding, 116, 422–434.CrossRefGoogle Scholar
  32. Krumm, J., Davies, N., & Narayanaswami, C. (2008). User-generated content. IEEE Pervasive Computing, 7, 10–11. Scholar
  33. Lenders, V., Koukoumidis, E., Zhang, P., & Martonosi, M. (2008). Location-based trust for mobile user-generated content: Applications, challenges and implementations. Proceedings of the 9th workshop on mobile computing systems and applications, HotMobile ’08 (pp. 60–64). New York: ACM.
  34. Makantasis, K., Doulamis, A., Doulamis, N., & Ioannides, M. (2016). In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction. Multimedia Tools and Applications, 75, 3593–3629. Scholar
  35. Ntalianis, K. S., Doulamis, A. D., Doulamis, N. D., Mastorakis, N. E., & Drigas, A. S. (2015). Unsupervised segmentation of stereoscopic video objects: Constrained segmentation fusion versus greedy active contours. Journal of Signal Processing Systems, 81, 153–181. Scholar
  36. Pancha, P., & Zarki, M. E. (1993). Bandwidth-allocation schemes for variable-bit-rate MPEG sources in ATM networks. IEEE Transactions on Circuits and Systems for Video Technology, 3, 190–198. Scholar
  37. Popescu, A. -M., & Pennacchiotti, M. (2010). Detecting controversial events from twitter. Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM ’10 (pp. 1873–1876). New York: ACM.
  38. Rosenberg, C., Hebert, M., & Schneiderman, H. (2005). Semi-supervised self-training of object detection models. WACV/MOTION (pp. 29–36).Google Scholar
  39. Sardis, E., Voulodimos, A., Anagnostopoulos, V., Lalos, C., Doulamis, A., & Kosmopoulos, D. (2010). An industrial video surveillance system for quality assurance of a manufactory assembly. Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments, PETRA ’10 (pp. 66:1–66:2). New York: ACM.
  40. Saroiu, S., & Wolman, A. (2010). I am a sensor, and I approve this message. Proceedings of the 11th Workshop on Mobile Computing Systems & Applications, HotMobile ’10 (pp. 37–42). New York: ACM.
  41. Soursos, S., & Doulamis, N. (2012). Connected TV and beyond. Consumer communications and networking conference (CCNC), 2012 IEEE (pp. 582–586). IEEE.Google Scholar
  42. Stuetzle, W. (2003). Estimating the cluster tree of a density by analyzing the minimal spanning tree of a sample. Journal of Classification, 20, 025–047. Scholar
  43. Vertan, C., & Boujemaa, N. (2000). Using fuzzy histograms and distances for color image retrieval. In Challenge of image retrieval retrieval, 6, 1–6Google Scholar
  44. Voulodimos, A. S., Doulamis, N. D., Kosmopoulos, D. I., & Varvarigou, T. A. (2012). Improving multi-camera activity recognition by employing neural network based readjustment. Applied Artificial Intelligence, 26, 97–118. Scholar
  45. Weng, J., & Lee, B.-S. (2011). Event detection in twitter. ICWSM, 11, 401–408.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Nikolaos Papadakis
    • 1
    • 2
  • Antonios Litke
    • 3
  • Anastasios Doulamis
    • 2
    Email author
  • Eftychios Protopapadakis
    • 2
  • Nikolaos Doulamis
    • 2
  1. 1.Hellenic Army AcademyVariGreece
  2. 2.National Technical University of AthensAthensGreece
  3. 3.Infili Technologies PCZografouGreece

Personalised recommendations