Personal and Ubiquitous Computing

, Volume 19, Issue 7, pp 1099–1110 | Cite as

A video cloud platform combing online and offline cloud computing technologies

  • Weishan Zhang
  • Liang Xu
  • Pengcheng Duan
  • Wenjuan Gong
  • Qinghua Lu
  • Su Yang
Original Article

Abstract

With the rapid growth of video data from various sources, like security and transportation surveillance, there arise requirements for both online real-time analysis and offline batch processing of large-scale video data. Existing video processing systems fall short in addressing many challenges in large-scale video processing, for example performance, data storage, and fault tolerance. The emerging cloud computing and big data techniques shed lights to intelligent processing for large-scale video data. This paper proposes a general cloud-based architecture and platform that can provide a robust solution to intelligent analysis and storage for video data, which is named as BiF (Batch processing Integrated with Fast processing) architecture. We have implemented the BiF architecture using both Hadoop platform and Storm platform, which are typical offline batch processing cloud platform and online real-time processing cloud platform, respectively. The proposed architecture can handle continual surveillance video data effectively, where real-time analysis, batch processing, distributed storage and cloud services are seamlessly integrated to meet the requirements of video data processing and management. The evaluations show that the proposed approach is efficient in terms of performance, storage, and fault tolerance.

Keywords

Cloud computing Video processing Hadoop Storm 

References

  1. 1.
    Benfold B, Reid I (2011) Stable multi-target tracking in real-time surveillance video. In: Computer vision and pattern recognition (CVPR), 2011 IEEE conference on IEEE, pp 3457–3464Google Scholar
  2. 2.
    Chowdhury A, Tripathy SS (2014) Detection of human presence in a surveillance video using fuzzy approach. In: Signal processing and integrated networks (SPIN), 2014 international conference on, IEEE pp 216–219Google Scholar
  3. 3.
    Zhou H et al (2010) Feature extraction and clustering for dynamic video summarisation. Neurocomputing 73(10):1718–1729CrossRefGoogle Scholar
  4. 4.
    Ali SF, Jaffar J, Malik AS (2011) Proposed framework of intelligent video automatic target recognition system (ivatrs). In: National postgraduate conference (NPC), IEEE 2011, pp. 1–5Google Scholar
  5. 5.
    Cavallaro A, Steiger O, Ebrahimi T (2005) Tracking video objects in cluttered background. Circuits Syst Video Technol IEEE Trans 15(4):575–584CrossRefGoogle Scholar
  6. 6.
    Miller M (2008) Cloud computing: web-based applications that change the way you work and collaborate online. Que publishing, New YorkGoogle Scholar
  7. 7.
    Neal D, Rahman SM (2012) Video surveillance in the cloud-computing? In: Electrical and computer engineering (ICECE), 2012 7th international conference on, IEEE pp 58–61Google Scholar
  8. 8.
    Ryu C, Lee D, Jang M, Kim C, Seo E (2013) Extensible video processing framework in apache hadoop. In: Cloud computing technology and science (CloudCom), 2013 IEEE 5th international conference on, IEEE 2, pp 305–310Google Scholar
  9. 9.
    Zhang W, Duan P, Lu Q (2014) A realtime framework for video object detection with storm. In: The 2014 international symposium on ubicom frontiers - innovative research, systems and technologies (UFirst 2014), IEEE pp 732–737Google Scholar
  10. 10.
    Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113CrossRefGoogle Scholar
  11. 11.
    Shirahama K (2011) Intelligent video processing using data mining techniques. ACM SIGMultimed Rec 3(2):7–9CrossRefGoogle Scholar
  12. 12.
    Chen Z, Ellis T (2014) A self-adaptive gaussian mixture model. Comput Vis Image Underst 122:35–46CrossRefGoogle Scholar
  13. 13.
    Marz N, Warren J (2013) Big data: principles and best practices of scalable realtime data systems. O’Reilly Media, SebastopolGoogle Scholar
  14. 14.
    Kim M, Cui Y, Han S, Lee H (2013) Towards efficient design and implementation of a hadoop-based distributed video transcoding system in cloud computing environment. Int J Multimed Ubiquitous Eng 8(2):213–224Google Scholar
  15. 15.
    Vora MN (2011) Hadoop-hbase for large-scale data. In: Computer science and network technology (ICCSNT), 2011 international conference on IEEE, vol 1, pp 601–605Google Scholar
  16. 16.
    Zhu X, Wu X, Elmagarmid AK, Feng Z, Wu L (2005) Video data mining: semantic indexing and event detection from the association perspective. Knowl Data Eng IEEE Trans 17(5):665–677CrossRefGoogle Scholar
  17. 17.
    Shirahama Kimiaki, Iwamoto Kazuhisa, Uehera Kuniaki (2004) Video data mining: rhythms in a movie. In Multimedia and Expo, 2004. ICME’04. 2004 IEEE International Conference on, volume 2, pages 1463–1466. IEEEGoogle Scholar
  18. 18.
    Oh JH, Lee JK, Kote S, Bandi B (2003) Multimedia data mining framework for raw video sequences. In: Mining multimedia and complex data, Springer: Berlin pp 18–35Google Scholar
  19. 19.
    Khan BUI, Olanrewaju RF, Altaf H, Shah A (2014) Critical insight for mapreduce optimization in hadoop. Int J Comput Sci Control Eng 2(1):1–7Google Scholar
  20. 20.
    Fielding RT, Taylor RN (2002) Principled design of the modern web architecture. ACM Trans Internet Technol 2(2):115–150CrossRefGoogle Scholar
  21. 21.
    Bass L, Clements P, Kazman R (2012) Software architecture in practice. Addison-Wesley, BostonGoogle Scholar
  22. 22.
    Zhang W (2014) Klaus Marius Hansen, and Mads Ingstrup. A hybrid approach to self-management in a pervasive service middleware. Knowl-Based Syst 67:143–161CrossRefGoogle Scholar
  23. 23.
    Zhang W, Wang W, Duan P, Liu X, Lu Q (2014) Online multiperson tracking and counting with cloud computing. In: 2014 International conference on identification, information and knowledge in the internet of things, IEEE pp 72–75Google Scholar
  24. 24.
    Ghemawat S, Gobioff H, Leung S-T (2003) The google file system. In: ACM SIGOPS operating systems review, 37, pp 29–43Google Scholar
  25. 25.
    White T (2012) Hadoop: the definitive guide. O’Reilly Media, Inc., SebastopolGoogle Scholar
  26. 26.
    Zhang K, Chen X (2014) Large-scale deep belief nets with mapreduce. Access, IEEEGoogle Scholar

Copyright information

© Springer-Verlag London 2015

Authors and Affiliations

  • Weishan Zhang
    • 1
  • Liang Xu
    • 1
  • Pengcheng Duan
    • 1
  • Wenjuan Gong
    • 1
  • Qinghua Lu
    • 1
  • Su Yang
    • 2
  1. 1.Department of Software EngineeringChina University of PetroleumQingdaoChina
  2. 2.Department of Computer Science and EngineeringFudan UniversityShanghaiChina

Personalised recommendations