Fast key-frame image retrieval of intelligent city security video based on deep feature coding in high concurrent network environment

  • Chuhong Li
  • Bo ZhouEmail author
Original Research


Aiming at the problems faced by the connection management module in single packet processing in intelligent city security video retrieval, we firstly propose the traffic locality quantization index based on the traffic characteristics of the backbone link to quantitatively analyze the traffic locality characteristics in the backbone link. And then, a key-frame abstraction and retrieval of videos based on deep learning is proposed to improve the efficiency and accuracy of video retrieval, where an adaptive key-frame selection algorithm is designed and the existing convolutional neural network framework is used to extract the features of key-frames, and unsupervised, semi-supervised and supervised retraining models are designed to improve the effectiveness of the feature extraction of the convolutional neural network and the accuracy of the video retrieval. Experimental results based on the public video datasets show that our proposed key-frame image retrieval model realizes a good precision for key-frame representation, and achieves high accuracy and efficiency for video retrieval.


Intelligent city Security video Deep feature Deep learning Key-frame selection Retraining model 



  1. Aote SS, Potnurwar A (2018) An automatic video annotation framework based on two level keyframe extraction mechanism [J]. Multimed Tools Appl 24(12):78–92Google Scholar
  2. Atmojo UD et al (2015) System-level approach to the design of ambient intelligence systems based on wireless sensor and actuator networks. J Ambient Intell Humaniz Comput 6(2):153–169CrossRefGoogle Scholar
  3. Burtnyk N, Wein M (1971) Computer-generated key-frame animation[J]. J SMPTE 80(3):149–153CrossRefGoogle Scholar
  4. Burtnyk N, Wein M (1976) Interactive skeleton techniques for enhancing motion dynamics in key-frame animation[J]. Commun ACM 19(10):564–569CrossRefGoogle Scholar
  5. Cappallo S, Svetlichnaya S, Garrigues P et al (2018) The new modality: emoji challenges in prediction, anticipation, and retrieval[J]. IEEE Trans Multimed 24(7):191–202Google Scholar
  6. Chen W, Qing Z, Yeting Z et al (2017) A NoSQL-SQL hybrid organization and management approach for real-time geospatial data: a case study of public security video surveillance[J]. ISPRS Int J Geo-Inform 6(1):21–43CrossRefGoogle Scholar
  7. Chenchen J, Zhen D, Mingtao P et al (2018) Heterogeneous hashing network for face retrieval across image and video domains[J]. IEEE Trans Multimed 24(23):1–12Google Scholar
  8. Deng Y, Yu Y (2019) Self-feedback image retrieval algorithm based on annular color moments[J]. EURASIP J Image Video Process 20(1):36–45Google Scholar
  9. Dong J, Li X, Snoek CGM (2018) Predicting visual features from text for image and video caption retrieval[J]. IEEE Trans Multimed 24(4):91–107Google Scholar
  10. Feng Y, Zhou P, Xu J et al (2019) Video big data retrieval over media cloud: a context-aware online learning approach[J]. IEEE Trans Multimed 24(7):12–31Google Scholar
  11. Gong Y, Lazebnik S, Gordo A et al (2013) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval[J]. IEEE Trans Pattern Anal Mach Intell 35(12):2916–2929CrossRefGoogle Scholar
  12. Haijun Z, Yuzhu J, Wang H et al (2018) Sitcom-star-based clothing retrieval for video advertising: a deep learning framework[J]. Neural Comput Appl 44(23):208–231Google Scholar
  13. Huang S, Mao C, Tao J et al (2018) A novel chinese sign language recognition method based on keyframe-centered clips[J]. IEEE Signal Process Lett 25(3):442–446CrossRefGoogle Scholar
  14. Jakub L, Werner B, Klaus S et al (2018) On influential trends in interactive video retrieval [J]. IEEE Trans Multimed 24(21):13–21Google Scholar
  15. Li X (2017) Tag relevance fusion for social image retrieval[J]. Multimed Syst 23(1):29–40CrossRefGoogle Scholar
  16. Lin J, Duan LY, Wang S et al (2017) HNIP: compact deep invariant representations for video matching, localization, and retrieval[J]. IEEE Trans Multimed 19(9):1968–1983CrossRefGoogle Scholar
  17. Liu L, Fan G (2005) Combined key-frame extraction and object-based video segmentation[J]. IEEE Trans Circuits Syst Video Technol 15(7):869–884CrossRefGoogle Scholar
  18. Liu T, Zhang HJ, Qi F (2003) A novel video key-frame-extraction algorithm based on perceived motion energy model[J]. IEEE Trans Circuits Syst Video Technol 13(10):1006–1013CrossRefGoogle Scholar
  19. Liu W, Wen Y, Yu Z et al (2016) Large-margin softmax loss for convolutional neural networks[J]. Comput Sci 2:507–516Google Scholar
  20. Mademlis I, Tefas A, Pitas I (2018) A salient dictionary learning framework for activity video summarization via key-frame extraction[J]. Inf Sci 43(2):319–331CrossRefGoogle Scholar
  21. Naruse N, Tenmyo O, Tomita K et al (2014) 1-HKUST: object detection in ILS VRC 2014[J]. Comput Sci 42(6):837–845Google Scholar
  22. Naveed E, Wook BS, Hammad M et al (2018) Multi-scale contrast and relative motion-based key-frame extraction[J]. EURASIP J Image Video Process 2018(1):40–48CrossRefGoogle Scholar
  23. Okabe M, Dobashi Y, Anjyo K (2018) Animating pictures of water scenes using video retrieval[J]. Vis Comput 34(3):347–358CrossRefGoogle Scholar
  24. Ponni ASS, Ramakrishnan S (2018) Fibonacci Based key-frame Selection and Scrambling for Video Watermarking in DWT–SVD Domain[J]. Wirel Personal Commun 69(25):28–35Google Scholar
  25. Radhakrishnan R (2001) Video summarization using descriptors of motion activity: a motion activity based approach to key-frame extraction from video shots[J]. J Electron Imaging 10(4):90–99CrossRefGoogle Scholar
  26. Rameshnath S, Bora PK (2019) Perceptual video hashing based on temporal wavelet transform and random projections with application to indexing and retrieval of near-identical videos. Multimed Tools Appl 78:18055–18075CrossRefGoogle Scholar
  27. Roy PP, Bhunia AK, Pal U (2017) Date-field retrieval in scene image and video frames using text enhancement and shape coding[J]. Neurocomputing. 274:S0925231217306689Google Scholar
  28. Song J, Yang Y, Huang Z et al (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval[J]. IEEE Trans Multimed 15(8):1997–2008CrossRefGoogle Scholar
  29. Song J, Gao L, Liu L et al (2017) Quantization-based hashing: a general framework for scalable image and video retrieval[J]. Pattern Recognit 75:S0031320317301322Google Scholar
  30. Song J, Gao L, Liu L et al (2018) Quantization-based hashing: a general framework for scalable image and video retrieval[J]. Pattern Recogn 75:175–187CrossRefGoogle Scholar
  31. Stippick J (2004) Advanced tray support system using orthogonal grillage[J]. J Comput Aided Des Comput Graph 24(22):632–642Google Scholar
  32. Wang Q, Si L, Zhang D (2014) Learning to hash with partial tags: exploring correlation between tags and hashing bits for large scale image retrieval[C]//European Conference on Computer Vision. Springer, NY, pp 378–392Google Scholar
  33. Wu S, Song H, Cheng G et al (2018) Civil engineering supervision video retrieval method optimization based on spectral clustering and R-tree[J]. Neural Comput Appl 21(2):12–23Google Scholar
  34. Ying Z, Daiyin Z (2018) Height retrieval in postprocessing-based VideoSAR image sequence using shadow information[J]. IEEE Sens J 44(13):8–21Google Scholar
  35. Yoo G, Kim H, Hong S et al (2017) Implementation of convergence P2P information retrieval system from captured video frames[J]. Peer-to-Peer Netw Appl 11:1–11Google Scholar
  36. Zheng F, Tang H, Liu YH (2018) Odometry-vision-based ground vehicle motion estimation with SE(2)-constrained SE(3) poses[J]. IEEE Trans Cybern 127(99):1–12Google Scholar
  37. Zhu X, Huang Z, Shen HT et al (2013) Linear cross-modal hashing for efficient multimedia search[J]. Comput Sci 2013:143–152Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2020

Authors and Affiliations

  1. 1.College of Architecture and EnvironmentSichuan UniversityChengduChina

Personalised recommendations