International Journal of Computer Vision

, Volume 98, Issue 2, pp 168–186 | Cite as

Non-uniform Deblurring for Shaken Images

  • Oliver Whyte
  • Josef Sivic
  • Andrew Zisserman
  • Jean Ponce
Article

Abstract

Photographs taken in low-light conditions are often blurry as a result of camera shake, i.e. a motion of the camera while its shutter is open. Most existing deblurring methods model the observed blurry image as the convolution of a sharp image with a uniform blur kernel. However, we show that blur from camera shake is in general mostly due to the 3D rotation of the camera, resulting in a blur that can be significantly non-uniform across the image. We propose a new parametrized geometric model of the blurring process in terms of the rotational motion of the camera during exposure. This model is able to capture non-uniform blur in an image due to camera shake using a single global descriptor, and can be substituted into existing deblurring algorithms with only small modifications. To demonstrate its effectiveness, we apply this model to two deblurring problems; first, the case where a single blurry image is available, for which we examine both an approximate marginalization approach and a maximum a posteriori approach, and second, the case where a sharp but noisy image of the scene is available in addition to the blurry image. We show that our approach makes it possible to model and remove a wider class of blurs than previous approaches, including uniform blur as a special case, and demonstrate its effectiveness with experiments on synthetic and real images.

Keywords

Motion blur Blind deconvolution Camera shake Non-uniform/spatially-varying blur 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11263_2011_502_MOESM1_ESM.pdf (301 kb)
Non-uniform Deblurring for Shaken Images: Derivation of parameter update equations for blind de-blurring (PDF 301 kB)

References

  1. Banham, M. R., & Katsaggelos, A. K. (1997). Digital image restoration. IEEE Signal Processing Magazine, 14(2), 24–41. CrossRefGoogle Scholar
  2. Bishop, C. M. (2006). Pattern recognition and machine learning (information science and statistics). Berlin: Springer. ISBN 0387310738. Google Scholar
  3. Cai, J.-F., Ji, H., Liu, C., & Shen, Z. (2009). Blind motion deblurring from a single image using sparse approximation. In Proc. CVPR. Google Scholar
  4. Chakrabarti, A., Zickler, T., & Freeman, W. T. (2010). Analyzing spatially-varying blur. In Proc. CVPR. Google Scholar
  5. Chan, T. F., & Wong, C.-K. (1998). Total variation blind deconvolution. IEEE Transactions on Image Processing, 7(3). Google Scholar
  6. Chen, J., Yuan, L., Tang, C.-K., & Quan, L. (2008). Robust dual motion deblurring. In Proc. CVPR. Google Scholar
  7. Cho, S., & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics, 28(5), 145:1–145:8 (Proc. SIGGRAPH Asia 2009). CrossRefGoogle Scholar
  8. Cho, S., Matsushita, Y., & Lee, S. (2007). Removing non-uniform motion blur from images. In Proc. ICCV. Google Scholar
  9. Couzinie-Devy, F., Mairal, J., Bach, F., & Ponce, J. (2011). Dictionary learning for deblurring and digital zoom (submitted). Preprint HAL: inria-00627402. Google Scholar
  10. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2008). Image restoration by sparse 3D transform-domain collaborative filtering. In SPIE electronic imaging. Google Scholar
  11. Efron, B., Hastie, T., Johnstone, L., & Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32(2), 407–499. MathSciNetMATHCrossRefGoogle Scholar
  12. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794 (Proc. SIGGRAPH 2006). CrossRefGoogle Scholar
  13. Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M., & Curless, B. (2010). Single image deblurring using motion density functions. In Proc. ECCV. Google Scholar
  14. Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd edn.). Cambridge: CUP. ISBN 0521540518. MATHCrossRefGoogle Scholar
  15. Hirsch, M., Sra, S., Schölkopf, B., & Harmeling, S. (2010). Efficient filter flow for space-variant multiframe blind deconvolution. In Proc. CVPR. Google Scholar
  16. Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. ACM Transactions on Graphics, 29(4), 30:1–30:9 (Proc. SIGGRAPH 2010). CrossRefGoogle Scholar
  17. Kim, S.-J., Koh, K., Lustig, M., Boyd, S., & Gorinevsky, D. (2007). An interior-point method for large-scale 1-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 1(4), 606–617. CrossRefGoogle Scholar
  18. Klein, G., & Drummond, T. (2005). A single-frame visual gyroscope. In Proc. BMVC. Google Scholar
  19. Krishnan, D., & Fergus, R. (2009). Fast image deconvolution using hyper-Laplacian priors. In NIPS. Google Scholar
  20. Lee, D. D., & Seung, H. S. (2001). Algorithms for non-negative matrix factorization. In NIPS. Google Scholar
  21. Levin, A. (2006). Blind motion deblurring using image statistics. In NIPS. Google Scholar
  22. Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. In Proc. CVPR. Google Scholar
  23. Lim, S. H., & Silverstein, A. (2008). Estimation and removal of motion blur by capturing two images with different exposures. Technical Report HPL-2008-170, HP Laboratories. Google Scholar
  24. Lucy, L. B. (1974). An iterative technique for the rectification of observed distributions. Astronomical Journal, 79(6), 745–754. CrossRefGoogle Scholar
  25. Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11, 19–60. MathSciNetGoogle Scholar
  26. Miskin, J. W., & MacKay, D. J. C. (2000). Ensemble learning for blind image separation and deconvolution. In M. Girolani (Ed.), Advances in independent component analysis. Berlin: Springer. Google Scholar
  27. Nagy, J. G., & O’Leary, D. P. (1998). Restoring images degraded by spatially variant blur. SIAM Journal on Scientific Computing, 19(4), 1063–1082. MathSciNetMATHCrossRefGoogle Scholar
  28. Osher, S., & Rudin, L. I. (1990). Feature oriented image enhancement using shock filters. SIAM Journal on Numerical Analysis, 27(4), 919–940. MATHCrossRefGoogle Scholar
  29. Pérez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. ACM Transactions on Graphics, 22(3), 313–318 (Proc. SIGGRAPH 2003). CrossRefGoogle Scholar
  30. Puetter, R. C., Gosnell, T. R., & Yahil, A. (2005). Digital image reconstruction: deblurring and denoising. Annual Review of Astronomy and Astrophysics, 43, 139–194. CrossRefGoogle Scholar
  31. Rav-Acha, A., & Peleg, S. (2005). Two motion-blurred images are better than one. Pattern Recognition Letters, 26(3). Google Scholar
  32. Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. Journal of the Optical Society of America, 62(1), 55–59. CrossRefGoogle Scholar
  33. Sawchuk, A. A. (1974). Space-variant image restoration by coordinate transformations. Journal of the Optical Society of America, 64(2), 138–144. CrossRefGoogle Scholar
  34. Shan, Q., Xiong, W., & Jia, J. (2007). Rotational motion deblurring of a rigid object from a single image. In Proc. ICCV. Google Scholar
  35. Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Transactions on Graphics, 27(3) (Proc. SIGGRAPH 2008). Google Scholar
  36. Tai, Y.-W., Tan, P., Gao, L., & Brown, M. S. (2009). Richardson-Lucy deblurring for scenes under projective motion path. Technical report, KAIST. Google Scholar
  37. Tai, Y.-W., Du, H., Brown, M. S., & Lin, S. (2010a). Correction of spatially varying image and video motion blur using a hybrid camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(6), 1012–1028. CrossRefGoogle Scholar
  38. Tai, Y.-W., Kong, N., Lin, S., & Shin, S. Y. (2010b). Coded exposure imaging for projective motion deblurring. In Proc. CVPR. Google Scholar
  39. Tai, Y.-W., Tan, P., & Brown, M. S. (2011). Richardson-Lucy deblurring for scenes under a projective motion path. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1603–1618. CrossRefGoogle Scholar
  40. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Joutnal of the Royal Statistical Society, Series B, 58(1), 267–288. MathSciNetMATHGoogle Scholar
  41. Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In Proc. ICCV. Google Scholar
  42. Vio, R., Nagy, J., Tenorio, L., & Wamsteker, W. (2005). Multiple image deblurring with spatially variant PSFs. Astronomy & Astrophysics, 434, 795–800. CrossRefGoogle Scholar
  43. Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Proc. ECCV. Google Scholar
  44. Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007a). Image deblurring with blurred/noisy image pairs. ACM Transactions on Graphics, 26(3) (Proc. SIGGRAPH 2007). Google Scholar
  45. Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007b). Blurred/non-blurred image alignment using sparseness prior. In Proc. ICCV. Google Scholar
  46. Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2008). Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Transactions on Graphics, 27(3) (Proc. SIGGRAPH 2008) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Oliver Whyte
    • 1
    • 4
  • Josef Sivic
    • 1
    • 4
  • Andrew Zisserman
    • 2
    • 4
  • Jean Ponce
    • 3
    • 4
  1. 1.INRIAParisFrance
  2. 2.Department of Engineering ScienceUniversity of OxfordOxfordUK
  3. 3.Département d’InformatiqueEcole Normale SupérieureParisFrance
  4. 4.Willow Project, Laboratoire d’Informatique de l’Ecole Normale SupérieureCNRS/ENS/INRIA UMR 8548ParisFrance

Personalised recommendations