International Journal of Computer Vision

, Volume 110, Issue 2, pp 185–201 | Cite as

Deblurring Shaken and Partially Saturated Images

Article

Abstract

We address the problem of deblurring images degraded by camera shake blur and saturated (over-exposed) pixels. Saturated pixels violate the common assumption that the image-formation process is linear, and often cause ringing in deblurred outputs. We provide an analysis of ringing in general, and show that in order to prevent ringing, it is insufficient to simply discard saturated pixels. We show that even when saturated pixels are removed, ringing is caused by attempting to estimate the values of latent pixels that are brighter than the sensor’s maximum output. Estimating these latent pixels is likely to cause large errors, and these errors propagate across the rest of the image in the form of ringing. We propose a new deblurring algorithm that locates these error-prone bright pixels in the latent sharp image, and by decoupling them from the remainder of the latent image, greatly reduces ringing. In addition, we propose an approximate forward model for saturated images, which allows us to estimate these error-prone pixels separately without causing artefacts. Results are shown for non-blind deblurring of real photographs containing saturated regions, demonstrating improved deblurred image quality compared to previous work.

Keywords

Non-blind deblurring Saturation Ringing Outliers 

References

  1. Afonso, M., Bioucas-Dias, J., & Figueiredo, M. (2010). Fast image recovery using variable splitting and constrained optimization. IEEE Transactions on Image Processing, 19(9), 2345–2356.MathSciNetCrossRefGoogle Scholar
  2. Almeida, M. S. C., & Figueiredo, M. A. T. (2013). Deconvolving images with unknown boundaries using the alternating direction method of multipliers. IEEE Transactions on Image Processing, 22(8), 3074–3086.MathSciNetCrossRefGoogle Scholar
  3. Bar, L., Sochen, N., & Kiryati, N. (2006). Image deblurring in the presence of impulsive noise. International Journal of Computer Vision, 70(3), 279–298.CrossRefGoogle Scholar
  4. Blake, A., & Zisserman, A. (1987). Visual Reconstruction. London: MIT Press.Google Scholar
  5. Bouman, C., & Sauer, K. (1993). A generalized Gaussian image model for edge-preserving MAP estimation. IEEE Transactions on Image Processing, 2(3), 296–310.CrossRefGoogle Scholar
  6. Cai, J.-F., Ji, H., Liu, C., Shen, Z (2009). Blind motion deblurring from a single image using sparse approximation. In: Proc. CVPRGoogle Scholar
  7. Chen, C., & Mangasarian, O. L. (1996). A class of smoothing functions for nonlinear and mixed complementarity problems. Computational Optimization and Applications, 5(2), 97–138.MathSciNetCrossRefMATHGoogle Scholar
  8. Cho, S., & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics, 28(5), 145–148.CrossRefGoogle Scholar
  9. Cho, S., Wang, J., & Lee, S. (2011). Handling outliers in non-blind image deconvolution. In:Proc ICCVGoogle Scholar
  10. Chou, P. B., & Brown, C. M. (1990). The theory and practice of Bayesian image labeling. International Journal of Computer Vision, 4(3), 185–210.CrossRefGoogle Scholar
  11. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794.CrossRefGoogle Scholar
  12. Gamelin, T. W. (2001). Complex Analysis. New York: Springer-Verlag.CrossRefMATHGoogle Scholar
  13. Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M., Curless, B. (2010). Single image deblurring using motion density functions. In: Proc. ECCVGoogle Scholar
  14. Harmeling, S., Hirsch, M., Schölkopf, B (2010a) Space-variant single-image blind deconvolution for removing camera shake. In:NIPSGoogle Scholar
  15. Harmeling, S., Sra, S., Hirsch, M., Schölkopf, B (2010b) Multiframe blind deconvolution, super-resolution, and saturation correction via incremental EM. In: Proc. ICIP.Google Scholar
  16. Ji, H., & Wang, K. (2012). Robust image deblurring with an inaccurate blur kernel. IEEE Transactions on Image Processing, 21(4), 1624–1634.MathSciNetCrossRefGoogle Scholar
  17. Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. ACM Transactions on Graphics, 29(4), 30–39.CrossRefGoogle Scholar
  18. Krishnan, D., Fergus, R. (2009). Fast image deconvolution using hyper-Laplacian priors. In NIPS.Google Scholar
  19. Krishnan, D., Tay, T., Fergus, R (2011). Blind deconvolution using a normalized sparsity measure. In: Proc. CVPR.Google Scholar
  20. Levin, A., Fergus, R., Durand, F., & Freeman, W. T. (2007). Image and depth from a conventional camera with a coded aperture. ACM Transactions on Graphics, 26(3), 70–79.CrossRefGoogle Scholar
  21. Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. MIT: Technical report.Google Scholar
  22. Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011). Efficient marginal likelihood optimization in blind deconvolution. CVPR: In Proc. Google Scholar
  23. Lucy, L. B. (1974). An iterative technique for the rectification of observed distributions. Astronomical Journal, 79(6), 745–754.CrossRefGoogle Scholar
  24. Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. The Journal of the Optical Society of America, 62(1), 55–59.CrossRefGoogle Scholar
  25. Schultz, R. R., & Stevenson, R. L. (1994). A bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing, 3(3), 233–242.CrossRefGoogle Scholar
  26. Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Transactions on Graphics (Proc. SIGGRAPH 2008), 27(3), 73-1–73-10.Google Scholar
  27. Tai, Y.-W., Du, H., Brown, M. S., & Lin, S. (2008). Image/video deblurring using a hybrid camera. CVPR: In Proc.Google Scholar
  28. Tai, Y.-W., Tan, P., & Brown, M. S. (2011). Richardson-Lucy deblurring for scenes under a projective motion path. IEEE PAMI, 33(8), 1603–1618.CrossRefGoogle Scholar
  29. Wang, Y., Yang, J., Yin, W., & Zhang, Y. (2008). A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences, 1(3), 248–272.MathSciNetCrossRefMATHGoogle Scholar
  30. Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.CrossRefGoogle Scholar
  31. Welk, M. (2010). Robust variational approaches to positivity-constrained image deconvolution. Technical Report 261, Saarland University, Saarbrücken, Germany.Google Scholar
  32. Whyte, O. (2012). Removing camera shake blur and unwanted occluders from photographs. PhD thesis, ENS Cachan.Google Scholar
  33. Whyte, O., Sivic, J., Zisserman, A. (2011). Deblurring shaken and partially saturated images. In: Proc. CPCV, with ICCV.Google Scholar
  34. Whyte, O., Sivic, J., Zisserman, A., Ponce, J. (2010). Non-uniform deblurring for shaken images. In: Proc. CVPR.Google Scholar
  35. Whyte, O., Sivic, J., Zisserman, A., & Ponce, J. (2012). Non-uniform deblurring for shaken images. International Journal of Computer Vision, 98(2), 168–186.MathSciNetCrossRefMATHGoogle Scholar
  36. Wiener, N. (1949). Extrapolation, interpolation, and smoothing of stationary time series. London: MIT Press.MATHGoogle Scholar
  37. Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Proc. ECCVGoogle Scholar
  38. Yang, J., Zhang, Y., & Yin, W. (2009). An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. The SIAM Journal on Scientific Computing, 31(4), 2842–2865. doi:10.1137/080732894.MathSciNetCrossRefMATHGoogle Scholar
  39. Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007). Image deblurring with blurred/noisy image pairs. ACM Transsactions on Graphics (Proc. SIGGRAPH 2007), 26(3), 1-1–1-10.Google Scholar
  40. Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2008). Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Transactions on Graphics (Proc. SIGGRAPH 2008), 27(3), 74-1–74-10.Google Scholar
  41. Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole image restoration. In Proc. ICCVGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Microsoft CorporationRedmondUSA
  2. 2.INRIA - Willow Project Laboratoire d’Informatique de l’Ecole Normale Supérieure (CNRS/ENS/INRIA UMR 8548)ParisFrance
  3. 3.Visual Geometry Group, Department of Engineering ScienceUniversity of OxfordOxfordUK

Personalised recommendations