Skip to main content
SpringerLink
Log in

Saliency-Based Fidelity Adaptation Preprocessing for Video Coding

  • Short Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, we present a video coding scheme which applies the technique of visual saliency computation to adjust image fidelity before compression. To extract visually salient features, we construct a spatio-temporal saliency map by analyzing the video using a combined bottom-up and top-down visual saliency model. We then use an extended bilateral filter, in which the local intensity and spatial scales are adjusted according to visual saliency, to adaptively alter the image fidelity. Our implementation is based on the H.264 video encoder JM12.0. Besides evaluating our scheme with the H.264 reference software, we also compare it to a more traditional foreground-background segmentation-based method and a foveation-based approach which employs Gaussian blurring. Our results show that the proposed algorithm can improve the compression ratio significantly while effectively preserving perceptual visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Tao B, Dickinson B W, Peterson H A. Adaptive model-driven bit allocation for MPEG video coding. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(1): 147–157.

    Article  Google Scholar 

  2. Tang C W, Chen C H, Yu Y H, Tsai C J. Visual sensitivity guided bit allocation for video coding. IEEE Transactions on Multimedia, 2006, 8(1): 11–18.

    Article  Google Scholar 

  3. Chen M J, Chi M C, Hsu C T, Chen J W. ROI video coding based on H.263+ with robust skin color detection technique. IEEE Transactions on Consumer Electron, 2003, 49(3): 724–730.

    Google Scholar 

  4. Chai D, Ngan K N. Foreground/background video coding scheme. In Proc. IEEE Int. Symp. Circuits Syst, Hong Kong, China, Jun. 9–12, 1997, pp.1448–1451.

  5. Lee S, Pattichis M S, Bovik A C. Foveated video compression with optimal rate control. IEEE Transactions on Image Process, 2001, 10(7): 977–992.

    Article  MATH  MathSciNet  Google Scholar 

  6. Wang D, Speranza F, Vincent A, Martin T, Blanchfield P. Towards optimal rate control: A study of the impact of spatial resolution, frame rate and quantization on subjective video quality and bit rate. In Proc. SPIE 2003, Lugano, Switzerland, Jul. 8–11, 2003, pp.198–209.

  7. Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Anal. and Machine Intell., 1998, 20(11): 1254–1259.

    Article  Google Scholar 

  8. Itti L. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing, 2004, 13(10): 1304–1318.

    Article  Google Scholar 

  9. Cavallaro A, Steiger O, Ebrahimi T. Semantic video analysis for adaptive content delivery and automatic description. IEEE Transactions on Circuits and Systems for Video Technology, 2005, 15(10): 1200–1209.

    Article  Google Scholar 

  10. Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In Proc. ICCV, Bombay, India, Jan. 4–7, 1998, pp.839–846.

  11. Eisemann E, Durand F. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics, 2004, 23(3): 673–678.

    Article  Google Scholar 

  12. Huang H, Zang Y, Rosin P L, Qi C. Edge-aware level set diffusion and bilateral filtering reconstruction for image magnification. Journal of Computer Science and Technology, 2009, 4(24): 734–744.

    Article  Google Scholar 

  13. Bennett E P, McMillan L. Video enhancement using per-pixel virtual exposures. ACM Transactions on Graphics, 2005, 24(3): 845–852.

    Article  Google Scholar 

  14. Winnemöller H, Olsen S C, Gooch B. Real-time video abstraction. ACM Transactions on Graphics, 2006, 25(3): 1221–1226.

    Article  Google Scholar 

  15. Xiao J J, Cheng H, Sawhney H, Rao C, Isnardi M. Bilateral filtering-based optical flow estimation with occlusion detection. In Proc. ECCV, Graz, Austria, May 7–13, 2006, pp.211–224.

  16. Paris S, Durand F. A fast approximation of the bilateral filter using a signal processing approach. In Proc. ECCV, Graz, Austria, May 7–13, 2006, pp.568–580.

  17. Pham T Q, Van Vliet L J. Separable bilateral filtering for fast video preprocessing. In Proc. IEEE ICME, Amsterdam, Netherlands, Jul. 6–9, 2005, pp.454–457.

  18. William J. The Principles of Psychology. Cambridge, MA: Harvard University Press, 1981.

    Google Scholar 

  19. Cerf M, Harel J, Einhäuser W, Koch C. Predicting human gaze using low-level saliency combined with face detection. In Proc. NIPS, Vancouver, Canada, Dec. 3–7, 2007, pp.241–248.

  20. Sebe N, Lew M S. Comparing salient point detectors. Pattern Recognition Letters, 2003, 24(1): 89–96.

    Article  MATH  Google Scholar 

  21. Robert J P, Iyer A, Itti L, Koch C. Components of bottom-up gaze allocation in natural scenes. Journal of Vision, 2005, 5(8): 692–692.

    Google Scholar 

  22. Tsapatsoulis N, Pattichis C, Rapantzikos K. Biologically inspired region of interest selection for low bit-rate video coding. In Proc. ICIP, San Antonio, USA, Sept. 16–19, 2007, pp.305–308.

  23. Chen W F, Liu C H, Lander K, Fu X L. Comparison of human face matching behavior and computational image similarity measure. Science in China Series F: Information Sciences, 2009, 52(2): 316–321.

    Article  MATH  Google Scholar 

  24. Lee K W. Guiding attention by cooperative cues. Journal of Computer Science and Technology, 2008, 5(23): 874–884.

    Article  Google Scholar 

  25. Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In Proc. CVPR, Hawaii, USA, Dec. 11–13, 2001, pp.511–518.

  26. Paris S. Edge-preserving smoothing and mean-shift segmentation of video streams. In Proc. ECCV, Marseille, France, Oct. 12–18, 2008, pp.460–473.

  27. Zhu S H, Liu Y C. Two-dimensional entropy model for video shot partitioning. Science in China Series F: Information Sciences, 2009, 52(2): 183–194.

    Article  MATH  Google Scholar 

  28. Gargi U, Kasturi R, Strayer S H. Performance characterization of video-shot-change detection methods. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(1): 1–13.

    Article  Google Scholar 

  29. H.264/AVC reference software [online]. http://iphome.hhi.de/suehring/html.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Shao-Ping Lu (Student Member, CCF, ACM) & Song-Hai Zhang (Member, CCF, ACM, IEEE)

  2. Beijing Engineering Research Center for Intelligent Processing of Visual Media and Content Security, Beijing, 100084, China

    Shao-Ping Lu (Student Member, CCF, ACM) & Song-Hai Zhang (Member, CCF, ACM, IEEE)

Authors
  1. Shao-Ping Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Song-Hai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shao-Ping Lu.

Additional information

This work was supported partially by the National High-Tech Research and Development 863 Program of China under Grant No. 2009AA01Z330, the National Natural Science Foundation of China under Grant Nos. 61033012 and 60970100.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 79.5 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, SP., Zhang, SH. Saliency-Based Fidelity Adaptation Preprocessing for Video Coding. J. Comput. Sci. Technol. 26, 195–202 (2011). https://doi.org/10.1007/s11390-011-9426-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-011-9426-5

Keywords

Search

Navigation