Skip to main content

Stripformer: Strip Transformer for Fast Image Deblurring

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13679))

Included in the following conference series:

Abstract

Images taken in dynamic scenes may contain unwanted motion blur, which significantly degrades visual quality. Such blur causes short- and long-range region-specific smoothing artifacts that are often directional and non-uniform, which is difficult to be removed. Inspired by the current success of transformers on computer vision and image processing tasks, we develop, Stripformer, a transformer-based architecture that constructs intra- and inter-strip tokens to reweight image features in the horizontal and vertical directions to catch blurred patterns with different orientations. It stacks interlaced intra-strip and inter-strip attention layers to reveal blur magnitudes. In addition to detecting region-specific blurred patterns of various orientations and magnitudes, Stripformer is also a token-efficient and parameter-efficient transformer model, demanding much less memory usage and computation cost than the vanilla transformer but works better without relying on tremendous training data. Experimental results show that Stripformer performs favorably against state-of-the-art models in dynamic scene deblurring.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The authors from the universities in Taiwan completed the experiments.

References

  1. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Proceedings of the European Conference on Computer Vision (2020)

    Google Scholar 

  2. Chen, H., et al.: Pre-trained image processing transformer. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  3. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: Proceedings of the Conference on Machine Learning (2020)

    Google Scholar 

  4. Chi, Z., Wang, Y., Yu, Y., Tang, J.: Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  5. Cho, S.J., Ji, S.W., Hong, J.P., Jung, S.W., Ko, S.J.: Rethinking coarse-to-fine approach in single image deblurring. In: Proceedings of the Conference on Computer Vision (2021)

    Google Scholar 

  6. Cho, S., Lee, S.: Fast motion deblurring. In: ACM Trans. Graphic. (2009)

    Google Scholar 

  7. Chu, X., et al.: Twins: Revisiting the design of spatial attention in vision transformers. In: Proceedings of the Conference on Neural Information Processing Systems (2021)

    Google Scholar 

  8. Chu, X., et al.: Conditional positional encodings for vision transformers. In: arxiv preprint arXiv:2102.10882 (2021)

  9. Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: Proceedings of the Conference on Learning Representations (2021)

    Google Scholar 

  10. Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. ACM Trans. Graphic. 25(3), 787–794 (2006)

    Google Scholar 

  11. Gao, H., Tao, X., Shen, X., Jia, J.: Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  12. Hou, Q., Zhang, L., Cheng, M.M., Feng, J.: Strip Pooling: Rethinking spatial pooling for scene parsing. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  13. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proc. Conf. Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  14. Hu, X., et al.: Pyramid architecture search for real-time image deblurring. In: Proceedings of the Conference on Computer Vision (2021)

    Google Scholar 

  15. Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: Ccnet: Criss-cross attention for semantic segmentation. In: Proceedings of the Conference on Computer Vision (2019)

    Google Scholar 

  16. Joshi, N., Zitnick, C.L., Szeliski, R., Kriegman, D.J.: Image deblurring and denoising using color priors. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2009)

    Google Scholar 

  17. Kupyn, O., Martyniuk, T., Wu, J., Wang, Z.: Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the Conference on Computer Vision (2019)

    Google Scholar 

  18. Li, J., Tan, W., Yan, B.: Perceptual variousness motion deblurring with light global context refinement. In: Proceedings of the Conference on Computer Vision (2021)

    Google Scholar 

  19. Liu, Z., et al.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the Conference on Computer Vision (2021)

    Google Scholar 

  20. Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  21. Pan, J., Hu, Z., Su, Z., Yang, M.H.: Deblurring text images via l0-regularized intensity and gradient prior. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2014)

    Google Scholar 

  22. Pan, J., Sun, D., Pfister, H., Yang, M.H.: Deblurring images via dark channel prior. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  23. Park, D., Kang, D.U., Kim, J., Chun, S.Y.: Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: Proceedings of the European Conference on Computer Vision (2020)

    Google Scholar 

  24. Parmar, N., Vaswani, A., Uszkoreit, J., Kaiser, Ł., Shazeer, N., Ku, A., Tran, D.: Image transformer. arXiv preprint arXiv:1802.05751 (2018)

  25. Purohit, K., Rajagopalan, A.N.: Region-adaptive dense network for efficient motion deblurring. In: Proceedings of the Conference on Artificial Intelligence (2020)

    Google Scholar 

  26. Purohit, K., Suin, M., Rajagopalan, A.N., Boddeti, V.N.: Spatially-adaptive image restoration using distortion-guided networks. In: Proceedings of the Conference on Computer Vision (2021)

    Google Scholar 

  27. Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: Proceedings of the Conference on Computer Vision (2021)

    Google Scholar 

  28. Rim, J., Lee, H., Won, J., Cho, S.: Real-world blur dataset for learning and benchmarking deblurring algorithms. In: Proceedings of the European Conference Computer Vision (2020)

    Google Scholar 

  29. Shen, Z., Wang, W., Shen, J., Ling, H., Xu, T., Shao, L.: Human-aware motion deblurring. In: Proceedings of the Conference on Computer Vision (2019)

    Google Scholar 

  30. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the Conference on Learning Representations (2015)

    Google Scholar 

  31. Suin, M., Purohit, K., Rajagopalan, A.N.: Spatially-attentive patch-hierarchical network for adaptive motion deblurring. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  32. Sun, J., Cao, W., Xu, Z., Ponce, J.: Learning a convolutional neural network for non-uniform motion blur removal. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  33. Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  34. Vaswani, A., et al.: Attention is all you need. In: Proceedings of the Conference on Neural Information Processing Systems (2017)

    Google Scholar 

  35. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  36. Wu, H., et al.: Contrastive learning for compact single image dehazing. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  37. Yang, F., Yang, H., Fu, J., Lu, H., Guo, B.: Learning texture transformer network for image super-resolution. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  38. Yuan, Y., Su, W., Ma, D.: Efficient dynamic scene deblurring using spatially variant deconvolution network with optical flow guided training. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  39. Zamir, S.W., et al.: Multi-stage progressive image restoration. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  40. Zeng, Y., Fu, J., Chao, H.: Learning joint spatial-temporal transformations for video inpainting. In: Proceedings of the European Conference on Computer Vision (2020)

    Google Scholar 

  41. Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. In: arXiv preprint arXiv:1805.08318 (2019)

  42. Zhang, H., Dai, Y., Li, H., Koniusz, P.: Deep stacked hierarchical multi-patch network for image deblurring. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  43. Zhang, K., et al.: Deblurring by realistic blurring. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Ministry of Science and Technology (MOST) under grants 109–2221-E-009–113- MY3, 111–2628-E-A49-025-MY3, 111–2634-F-007–002, 110–2634-F-002–050, 110–2634-F-006–022, 110–2622-E-004–001, and 111–2221-E-004–010. This work was funded in part by Qualcomm through a Taiwan University Research Collaboration Project and by MediaTek. We thank the National Center for High-performance Computing (NCHC) of National Applied Research Laboratories (NARLabs) in Taiwan for providing computational and storage resources.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chia-Wen Lin .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 4548 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tsai, FJ., Peng, YT., Lin, YY., Tsai, CC., Lin, CW. (2022). Stripformer: Strip Transformer for Fast Image Deblurring. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13679. Springer, Cham. https://doi.org/10.1007/978-3-031-19800-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19800-7_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19799-4

  • Online ISBN: 978-3-031-19800-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics