Advertisement

Non-causal Temporal Prior for Video Deblocking

  • Deqing Sun
  • Ce Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)

Abstract

Real-world video sequences coded at low bit rates suffer from compression artifacts, which are visually disruptive and can cause problems to computer vision algorithms. Unlike the denoising problem where the high frequency components of the signal are present in the noisy observation, most high frequency details are lost during compression and artificial discontinuities arise across the coding block boundaries. In addition to sparse spatial priors that can reduce the blocking artifacts for a single frame, temporal information is needed to recover the lost spatial details. However, establishing accurate temporal correspondences from the compressed videos is challenging because of the loss of high frequency details and the increase of false blocking artifacts. In this paper, we propose a non-causal temporal prior model to reduce video compression artifacts by propagating information from adjacent frames and iterating between image reconstruction and motion estimation. Experimental results on real-world sequences demonstrate that the deblocked videos by the proposed system have marginal statistics of high frequency components closer to those of the original ones, and are better input for standard edge and corner detectors than the coded ones.

Keywords

Motion Estimation High Frequency Component Image Detail Block Match Block Boundary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
  3. 3.
    MPEG4 Verification Model, VM 18.0, pp. 271–275 (2001)Google Scholar
  4. 4.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92(1), 1–31 (2011)CrossRefGoogle Scholar
  5. 5.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18, 509–517 (1975)zbMATHCrossRefGoogle Scholar
  6. 6.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  8. 8.
    Dabov, K., Foi, A., Egiazarian, K.: Video denoising by sparse 3d transform-domain collaborative filtering. In: EUSIPCO (2007)Google Scholar
  9. 9.
    Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: SIGGRAPH, pp. 327–340 (2001)Google Scholar
  10. 10.
    Horn, B., Schunck, B.: Determining optical flow. AI 16, 185–203 (1981)Google Scholar
  11. 11.
    Jin, X., Goto, S., Ngan, K.-N.: Optical flow based DC surface compensation for artifacts reduction. In: Picture Coding Symposium (2009)Google Scholar
  12. 12.
    Li, Z., Delp, E.: MAP-based post processing of video sequences using 3-d huber-markov random field model. In: ICME (2002)Google Scholar
  13. 13.
    Liang, L., Liu, C., Xu, Y.-Q., Guo, B., Shum, H.-Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)CrossRefGoogle Scholar
  14. 14.
    Liew, A., Yan, H.: Blocking artifacts suppression in block-coded images using overcomplete wavelet representation. IEEE TCSVT 14(4), 450–461 (2004)Google Scholar
  15. 15.
    Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. PhD thesis. MIT (2009)Google Scholar
  16. 16.
    Liu, C., Freeman, W.T.: A High-Quality Video Denoising Algorithm Based on Reliable Motion Estimation. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 706–719. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Liu, C., Sun, D.: A Bayesian approach to adaptive video super resolution. In: CVPR (2011)Google Scholar
  18. 18.
    Meier, T., Ngan, K., Crebbin, G.: Reduction of blocking artifacts in image and video coding. IEEE TCSVT 9(3), 490–500 (1999)Google Scholar
  19. 19.
    Minami, S., Zakhor, A.: An optimization approach for removing blocking effects in transform coding. IEEE TCSVT 5(2), 74–82 (1995)Google Scholar
  20. 20.
    Reddy, D., Veeraraghavan, A., Chellappa, R.: P2C2: Programmable pixel compressive camera for high speed imaging. In: CVPR (2011)Google Scholar
  21. 21.
    Richardson, I.E.: The H.264 Advanced Video Compression Standard. Addison-Wesley (2010)Google Scholar
  22. 22.
    Robertson, M.A., Stevenson, R.L.: Restoration of compressed video using temporal information. In: Proc. of SPIE (2001)Google Scholar
  23. 23.
    Roth, S., Black, M.: Fields of experts: a framework for learning image priors. In: CVPR (2005)Google Scholar
  24. 24.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1-4), 259–268 (1992)zbMATHCrossRefGoogle Scholar
  25. 25.
    Sikora, T.: MPEG digital video-coding standards. IEEE Signal Processing Magazine 14(5), 82–100 (1997)CrossRefGoogle Scholar
  26. 26.
    Sun, D., Cham, W.-K.: Postprocessing of low bit-rate block DCT coded images based on a fields of experts prior. IEEE TIP 16(11), 2743–2751 (2007)MathSciNetGoogle Scholar
  27. 27.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR (2010)Google Scholar
  28. 28.
    Szeliski, R.: Locally adapted hierarchical basis preconditioning. In: SIGGRAPH (2006)Google Scholar
  29. 29.
    Wallace, G.K.: The JPEG still picture compression standard. Commun. ACM 34, 30–44 (1991)CrossRefGoogle Scholar
  30. 30.
    Wiegand, T., Sullivan, G., Bjontegaard, G., Luthra, A.: Overview of the H.264/AVC video coding standard. IEEE TCSVT 13(7), 560–576 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Deqing Sun
    • 1
  • Ce Liu
    • 2
  1. 1.Harvard UniversityUSA
  2. 2.Microsoft Research New EnglandUSA

Personalised recommendations