Advertisement

A High-Quality Video Denoising Algorithm Based on Reliable Motion Estimation

  • Ce Liu
  • William T. Freeman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6313)

Abstract

Although the recent advances in the sparse representations of images have achieved outstanding denosing results, removing real, structured noise in digital videos remains a challenging problem. We show the utility of reliable motion estimation to establish temporal correspondence across frames in order to achieve high-quality video denoising. In this paper, we propose an adaptive video denosing framework that integrates robust optical flow into a non-local means (NLM) framework with noise level estimation. The spatial regularization in optical flow is the key to ensure temporal coherence in removing structured noise. Furthermore, we introduce approximate K-nearest neighbor matching to significantly reduce the complexity of classical NLM methods. Experimental results show that our system is comparable with the state of the art in removing AWGN, and significantly outperforms the state of the art in removing real, structured noise.

Keywords

Motion Estimation Additive White Gaussian Noise Priority Queue Temporal Coherence Structure Noise 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

Supplementary material

978-3-642-15558-1_51_MOESM1_ESM.wmv (3.6 mb)
Electronic Supplementary Material (3,696 KB)
978-3-642-15558-1_51_MOESM2_ESM.wmv (6.6 mb)
Electronic Supplementary Material (6,788 KB)
978-3-642-15558-1_51_MOESM3_ESM.wmv (3.4 mb)
Electronic Supplementary Material (3,490 KB)

References

  1. 1.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)CrossRefGoogle Scholar
  2. 2.
    Mairal, J., Elad, M., Sapiro, G.: Multiscale sparse image representation with learned dictionaries. In: IEEE International Conference on Image Processing, ICIP (2007)Google Scholar
  3. 3.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
  4. 4.
    Simoncelli, E.P., Adelson, E.H.: Noise removal via Bayesian wavelet coring. In: IEEE International Conference on Image Processing (ICIP), vol. I, pp. 379–382 (1996)Google Scholar
  5. 5.
    Portilla, J., Strela, V., Wainwright, M.J., Simoncelli, E.P.: Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Transactions on Image Processing (TIP) 12, 1338–1351 (2003)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)Google Scholar
  7. 7.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3d transform-domain collaborative filtering. IEEE Transactions on Image Processing (TIP) 16 (2007)Google Scholar
  8. 8.
    MPEG: Mpeg-2 video encoding, h.262 (2006), http://www.digitalpreservation.gov/formats/fdd/fdd000028.shtml
  9. 9.
    Lee, C.U., Pian, D.: Interframe video encoding and decoding system (1996) US Patent 5,576,767Google Scholar
  10. 10.
    Dabov, K., Foi, A., Egiazarian, K.: Video denoising by sparse 3d transform-domain collaborative filtering. In: European Signal Processing Conference, EUSIPCO (2007)Google Scholar
  11. 11.
    Buades, A., Coll, B., Morel, J.M.: Nonlocal image and movie denoising. International Journal of Computer Vision (IJCV) 76, 123–139 (2008)CrossRefGoogle Scholar
  12. 12.
    Roth, S., Black, M.J.: Fields of experts: A framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)Google Scholar
  13. 13.
    Weiss, Y., Freeman, W.: What makes a good model of natural images. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2007)Google Scholar
  14. 14.
    Elad, M., Aharon, M.: Image denoising via learned dictionaries and sparse representation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2006)Google Scholar
  15. 15.
    Tschumperlé, D.: Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE’s. International Journal of Computer Vision (IJCV) 68, 65–82 (2006)CrossRefGoogle Scholar
  16. 16.
    Liu, C., Szeliski, R., Kang, S.B., Zitnick, C.L., Freeman, W.T.: Automatic estimation and removal of noise from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 30, 299–314 (2008)CrossRefGoogle Scholar
  17. 17.
    Buades, A., Coll, B., Morel, J.M.: Denoising image sequences does not require motion estimation. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (2005)Google Scholar
  18. 18.
    Cho, T., Butman, M., Avidan, S., Freeman, W.: The patch transform and its applications to image editing. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
  19. 19.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patchmatch: A randomized correspondence algorithm for structural image editing. In: Proceedings of ACM SIGGRAPH (2009)Google Scholar
  20. 20.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 30, 1068–1080 (2008)CrossRefGoogle Scholar
  21. 21.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunk: combining local and global optical flow methods. International Journal of Computer Vision (IJCV) 61, 211–231 (2005)CrossRefGoogle Scholar
  22. 22.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  23. 23.
    Horn, B.K.P., Schunck, B.G.: Determing optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  24. 24.
    Liu, C., Freeman, W.T., Szeliski, R., Kang, S.B.: Noise estimation from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 901–908 (2006)Google Scholar
  25. 25.
    Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. PhD thesis, Massachusetts Insitute of Technology (2009)Google Scholar
  26. 26.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. ICCV (2007)Google Scholar
  27. 27.
    Liu, C., Freeman, W.T., Adelson, E.H., Weiss, Y.: Human-assisted motion annotation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ce Liu
    • 1
  • William T. Freeman
    • 1
    • 2
  1. 1.Microsoft Research NewEngland
  2. 2.Massachusetts Institute of Technology 

Personalised recommendations

Cite paper