Abstract
We address the problem of deblurring images degraded by camera shake blur and saturated (over-exposed) pixels. Saturated pixels violate the common assumption that the image-formation process is linear, and often cause ringing in deblurred outputs. We provide an analysis of ringing in general, and show that in order to prevent ringing, it is insufficient to simply discard saturated pixels. We show that even when saturated pixels are removed, ringing is caused by attempting to estimate the values of latent pixels that are brighter than the sensor’s maximum output. Estimating these latent pixels is likely to cause large errors, and these errors propagate across the rest of the image in the form of ringing. We propose a new deblurring algorithm that locates these error-prone bright pixels in the latent sharp image, and by decoupling them from the remainder of the latent image, greatly reduces ringing. In addition, we propose an approximate forward model for saturated images, which allows us to estimate these error-prone pixels separately without causing artefacts. Results are shown for non-blind deblurring of real photographs containing saturated regions, demonstrating improved deblurred image quality compared to previous work.
Similar content being viewed by others
Notes
We note that the regulariser \(\varvec{\phi }\) in Eq. (26) is not continuously differentiable, however this can be avoided by using the standard approximation \( |x|\simeq \sqrt{\epsilon +x^2}\), for some small number \(\epsilon \).
http://cg.postech.ac.kr/research/deconv_outliers/ (accessed November 12, 2011).
References
Afonso, M., Bioucas-Dias, J., & Figueiredo, M. (2010). Fast image recovery using variable splitting and constrained optimization. IEEE Transactions on Image Processing, 19(9), 2345–2356.
Almeida, M. S. C., & Figueiredo, M. A. T. (2013). Deconvolving images with unknown boundaries using the alternating direction method of multipliers. IEEE Transactions on Image Processing, 22(8), 3074–3086.
Bar, L., Sochen, N., & Kiryati, N. (2006). Image deblurring in the presence of impulsive noise. International Journal of Computer Vision, 70(3), 279–298.
Blake, A., & Zisserman, A. (1987). Visual Reconstruction. London: MIT Press.
Bouman, C., & Sauer, K. (1993). A generalized Gaussian image model for edge-preserving MAP estimation. IEEE Transactions on Image Processing, 2(3), 296–310.
Cai, J.-F., Ji, H., Liu, C., Shen, Z (2009). Blind motion deblurring from a single image using sparse approximation. In: Proc. CVPR
Chen, C., & Mangasarian, O. L. (1996). A class of smoothing functions for nonlinear and mixed complementarity problems. Computational Optimization and Applications, 5(2), 97–138.
Cho, S., & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics, 28(5), 145–148.
Cho, S., Wang, J., & Lee, S. (2011). Handling outliers in non-blind image deconvolution. In:Proc ICCV
Chou, P. B., & Brown, C. M. (1990). The theory and practice of Bayesian image labeling. International Journal of Computer Vision, 4(3), 185–210.
Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794.
Gamelin, T. W. (2001). Complex Analysis. New York: Springer-Verlag.
Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M., Curless, B. (2010). Single image deblurring using motion density functions. In: Proc. ECCV
Harmeling, S., Hirsch, M., Schölkopf, B (2010a) Space-variant single-image blind deconvolution for removing camera shake. In:NIPS
Harmeling, S., Sra, S., Hirsch, M., Schölkopf, B (2010b) Multiframe blind deconvolution, super-resolution, and saturation correction via incremental EM. In: Proc. ICIP.
Ji, H., & Wang, K. (2012). Robust image deblurring with an inaccurate blur kernel. IEEE Transactions on Image Processing, 21(4), 1624–1634.
Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. ACM Transactions on Graphics, 29(4), 30–39.
Krishnan, D., Fergus, R. (2009). Fast image deconvolution using hyper-Laplacian priors. In NIPS.
Krishnan, D., Tay, T., Fergus, R (2011). Blind deconvolution using a normalized sparsity measure. In: Proc. CVPR.
Levin, A., Fergus, R., Durand, F., & Freeman, W. T. (2007). Image and depth from a conventional camera with a coded aperture. ACM Transactions on Graphics, 26(3), 70–79.
Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. MIT: Technical report.
Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011). Efficient marginal likelihood optimization in blind deconvolution. CVPR: In Proc.
Lucy, L. B. (1974). An iterative technique for the rectification of observed distributions. Astronomical Journal, 79(6), 745–754.
Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. The Journal of the Optical Society of America, 62(1), 55–59.
Schultz, R. R., & Stevenson, R. L. (1994). A bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing, 3(3), 233–242.
Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Transactions on Graphics (Proc. SIGGRAPH 2008), 27(3), 73-1–73-10.
Tai, Y.-W., Du, H., Brown, M. S., & Lin, S. (2008). Image/video deblurring using a hybrid camera. CVPR: In Proc.
Tai, Y.-W., Tan, P., & Brown, M. S. (2011). Richardson-Lucy deblurring for scenes under a projective motion path. IEEE PAMI, 33(8), 1603–1618.
Wang, Y., Yang, J., Yin, W., & Zhang, Y. (2008). A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences, 1(3), 248–272.
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.
Welk, M. (2010). Robust variational approaches to positivity-constrained image deconvolution. Technical Report 261, Saarland University, Saarbrücken, Germany.
Whyte, O. (2012). Removing camera shake blur and unwanted occluders from photographs. PhD thesis, ENS Cachan.
Whyte, O., Sivic, J., Zisserman, A. (2011). Deblurring shaken and partially saturated images. In: Proc. CPCV, with ICCV.
Whyte, O., Sivic, J., Zisserman, A., Ponce, J. (2010). Non-uniform deblurring for shaken images. In: Proc. CVPR.
Whyte, O., Sivic, J., Zisserman, A., & Ponce, J. (2012). Non-uniform deblurring for shaken images. International Journal of Computer Vision, 98(2), 168–186.
Wiener, N. (1949). Extrapolation, interpolation, and smoothing of stationary time series. London: MIT Press.
Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Proc. ECCV
Yang, J., Zhang, Y., & Yin, W. (2009). An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. The SIAM Journal on Scientific Computing, 31(4), 2842–2865. doi:10.1137/080732894.
Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007). Image deblurring with blurred/noisy image pairs. ACM Transsactions on Graphics (Proc. SIGGRAPH 2007), 26(3), 1-1–1-10.
Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2008). Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Transactions on Graphics (Proc. SIGGRAPH 2008), 27(3), 74-1–74-10.
Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole image restoration. In Proc. ICCV
Acknowledgments
This work was partly supported by the MSR-INRIA laboratory, the EIT ICT labs and ERC grant VisRec no. 228180.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Dr. Srinivas Narasimhan, Dr. Frédo Durand and Dr. Wolfgang Heidrich.
Parts of this work were previously published in the IEEE Workshop on Color and Photometry in Computer Vision, with ICCV 2011 Whyte et al. (2011).
Rights and permissions
About this article
Cite this article
Whyte, O., Sivic, J. & Zisserman, A. Deblurring Shaken and Partially Saturated Images. Int J Comput Vis 110, 185–201 (2014). https://doi.org/10.1007/s11263-014-0727-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-014-0727-3