Multimedia Tools and Applications

, Volume 76, Issue 5, pp 7283–7300 | Cite as

Compressed domain visual information retrieval based on I-frames in HEVC

Article

Abstract

Compressed domain retrieval is important in the retrieval of HEVC coded videos, because full decompression of HEVC coded videos is very time consuming. In this paper, a texture based retrieval method is proposed for either video retrieval of HEVC coded videos or image retrieval of coded images as HEVC I-frames. The various prediction unit sizes and high number of prediction modes which are employed in I-frame coding of HEVC, provides higher information about the texture of the coded picture compared to the I-frame coding in H.264/AVC. The proposed texture based image retrieval method for HEVC coded visual information is based on histograms of prediction modes and prediction unit sizes. This method achieves 0.34 ANMRR in the image retrieval experiments and 0.33 ANMRR in the conducting video retrieval experiments, which are better compared to the other compressed domain retrieval methods based on H.264/AVC. On the other hand, the experimental evaluations indicate good robustness of the proposed method against variations in quantization parameter and image intensity. Moreover, the timing analysis indicates that feature extraction time by the proposed method is about 35 % of the decoding time of a coded video.

Keywords

HEVC Compressed domain Texture based retrieval Prediction modes Prediction unit size 

References

  1. 1.
    Akrami F, Zargari F (2014) An efficient compressed domain video indexing method. Multimed Tools Appl 72(1):705–721CrossRefGoogle Scholar
  2. 2.
    Babu RV, Tom M, Wadekar P (2016) A Survey on compressed domain video analysis techniques. Multimed Tools Appl 75(2):1043–1078Google Scholar
  3. 3.
    Dissanayake MB, Abeyrathna DLB (2015) Performance comparison of HEVC and H.264/AVC standards in broadcasting environments. J Inf Process Syst 11(3):483–494Google Scholar
  4. 4.
    Divakaran A, Vetro A, Asai K, Nishikawa H (2000) Video browsing system based on compressed domain feature extraction. IEEE Trans Consum Electron 46(3):637–644CrossRefGoogle Scholar
  5. 5.
    Gao L, Song J, Zou F, Zhang D, Shao J (2015) Scalable multimedia retrieval by deep learning hashing with relative similarity learning. Proceedings of the 23rd ACM international conference on Multimedia, New York, USAGoogle Scholar
  6. 6.
    Garcia R, Kalva H, Raton B (2013) Human mobile-device interaction on HEVC and H.264 subjective evaluation for video use in mobile environment. IEEE International Conference on Consumer Electronics (ICCE)Google Scholar
  7. 7.
    H.264/AVC Reference Software, available on: http://iphome.hhi.de/suehring/tml/download/> Last seen on June 2015
  8. 8.
    H.265/HEVC Reference Software, available on:< https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/> Last seen on June 2015
  9. 9.
    INRIA HOLIDAY Dataset, available on: < https://lear.inrialpes.fr/~jegou/data.php> Last seen on Jan 2015
  10. 10.
    Jiang J, Weng Y, Li P (2006) Dominant colour extraction in DCT domain. Image Vis Comput 24(2):1269–1277Google Scholar
  11. 11.
    Joint video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG, (2009) H.264/14496-10 AVC Reference Software Manual. 31st Meeting, London, UK, 28 June 3 July, 2009Google Scholar
  12. 12.
    JTC 1/SC 29; ISO Standards, (2005) MPEG-7: Multimedia Content Description Interface, April 2005Google Scholar
  13. 13.
    Kim IK, Min J, Lee T, Han WJ, Park JH (2012) Block partitioning structure in the HEVC standard. IEEE Trans Circ Syst Video Technol Vol. 22, No. 12Google Scholar
  14. 14.
    Lu ZM, Li SZ, Burkhardt H (2006) A content-based image retrieval scheme In JPEG compressed domain. Int J Innov Comput, Inf Control 2, NO 4Google Scholar
  15. 15.
    Manjunath BS, Salembier P, Sikora T (2002) Introduction to MPEG-7 multimedia content description interface., pp 177–280Google Scholar
  16. 16.
    Mehrabi M, Zargari F, Ghanbari M (2012) Compressed domain content based retrieval using H.264 DC-pictures. Multimed Tools Appl 60(2):443–453CrossRefGoogle Scholar
  17. 17.
    Oxford Buildings Dataset, available on: < http://www.robots.ox.ac.uk/~vgg/data/oxbuildings/>, Last seen on Jan 2015
  18. 18.
    Rao KR, Kim DN, Hewang JJ (2013) Video coding standards, Springer, pp. 125–148Google Scholar
  19. 19.
    Rodriguez MD, Ahmed J, Shah M (2008) Action MACH: a spatio-temporal maximum average correlation height filter for action recognition. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, pp 1–8Google Scholar
  20. 20.
    Simone FD, Goldmann L, Lee JS, Ebrahimi T (2011) Towards high efficiency video coding: subjective evaluation of potential coding technologies. J Vis Commun Image Represent 22:734–748CrossRefGoogle Scholar
  21. 21.
    Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Patterns Anal Mach Intell 22:1349–1380CrossRefGoogle Scholar
  22. 22.
    Song J, Gao L, Yan Y, Zhang D, Sebe N (2015) Supervised hashing with pseudo labels for scalable multimedia retrieval. Proceedings of the 23rd ACM international conference on Multimedia, New York, USAGoogle Scholar
  23. 23.
    Song J, Yang Y, Huang Z, Shen HT, Hong R (2011) Multiple feature hashing for real-time large scale near-duplicate video retrieval. MM ’11 Proceedings of the 19th ACM international conference on Multimedia, New York, USAGoogle Scholar
  24. 24.
    Soomro K, Zamir AR (2014) Action recognition in realistic sports videos. Computer Vision in Sports. Springer International PublishingGoogle Scholar
  25. 25.
    Sullivan GJ, Ohm JR, Han WJ, Wiegand T (2012) Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits And Systems For Video Technology, Vol. 22, NO 12Google Scholar
  26. 26.
    Sze V, Budagavi M, Sullivan GJ (2014) High Efficiency Video Coding (HEVC), algorithms and architectures. Springer International Publishing, SwitzerlandGoogle Scholar
  27. 27.
    Wang RJ, Yang YT, Chang PC (2014) Content-based image retrieval using H.264 intra coding features. J Vis Commun Image Represent 25:963–969CrossRefGoogle Scholar
  28. 28.
    Wien M (2015) High efficiency video coding, coding tool and specification. Springer, Berlin HeidelbergGoogle Scholar
  29. 29.
    www.h265.net, Last seen on June 2015
  30. 30.
    Zargari F, Mehrabi M, Ghanbari M (2008) A robust compressed domain Feature vector for texture based image retrieval. Content-Based Multimedia Indexing, International Workshop on, pp. 489–495Google Scholar
  31. 31.
    Zargari F, Mehrabi M, Ghanbari M (2010) Compressed domain texture based visual information retrieval method for I-frame coded pictures. IEEE Trans Consum Electron 56(2):728–736CrossRefGoogle Scholar
  32. 32.
    Zargari F, Mehrabi M, Moin MS (2007) Compressed domain texture retrieval based on I-Frame coding in H.264. 2007 IEEE International Conference on Multimedia and Expo, Beijing, pp 831–834Google Scholar
  33. 33.
    Zargari F, Mosleh A, Ghanbari M (2008) A fast and efficient compressed domain JPEG2000 image retrieval method. IEEE Trans Consum Electron 54(4):1886–1893CrossRefGoogle Scholar
  34. 34.
    Zargari F, Rahmani F (2015) Visual information retrieval in HEVC compressed domain. 23rd Iranian Conference on Electrical Engineering, ICEEGoogle Scholar
  35. 35.
    Zhu M (2004) Recall, precision, and average precision, Dept. Stat. Actuarial Sci., Univ. Waterloo, CA, Tech. Rep. 9Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Information Technology FacultyIran Telecom Research CenterTehranIran

Personalised recommendations