Abstract
When smartphone cameras are used to take photos of digital screens, usually moiré patterns result, severely degrading photo quality. In this paper, we design a wavelet-based dual-branch network (WDNet) with a spatial attention mechanism for image demoiréing. Existing image restoration methods working in the RGB domain have difficulty in distinguishing moiré patterns from true scene texture. Unlike these methods, our network removes moiré patterns in the wavelet domain to separate the frequencies of moiré patterns from the image content. The network combines dense convolution modules and dilated convolution modules supporting large receptive fields. Extensive experiments demonstrate the effectiveness of our method, and we further show that WDNet generalizes to removing moiré artifacts on non-screen images. Although designed for image demoiréing, WDNet has been applied to two other low-level vision tasks, outperforming state-of-the-art image deraining and deraindrop methods on the Rain100h and Raindrop800 data sets, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bao, L., Song, Y., Yang, Q., Yuan, H., Wang, G.: Tree filtering: efficient structure-preserving smoothing with a minimum spanning tree. TIP 23, 555–569 (2013)
Bell, S., Lawrence Zitnick, C., Bala, K., Girshick, R.: Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks. In: CVPR (2016)
Blau, Y., Michaeli, T.: The perception-distortion tradeoff. In: CVPR (2018)
Chang, Y., Yan, L., Zhong, S.: Transformed low-rank model for line pattern noise removal. In: ICCV (2017)
Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: CVPR (2013)
Fu, J., et al.: Dual attention network for scene segmentation. In: CVPR (2019)
Fujieda, S., Takayama, K., Hachisuka, T.: Wavelet convolutional neural networks for texture classification. arXiv preprint arXiv:1707.07394 (2017)
Gao, F., Huang, T., Wang, J., Sun, J., Hussain, A., Yang, E.: Dual-branch deep convolution neural network for polarimetric SAR image classification. Appl. Sci. 7, 447 (2017)
Gharbi, M., Chaurasia, G., Paris, S., Durand, F.: Deep joint demosaicking and denoising. TOG 35, 1–12 (2016)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall, Upper Saddle River (2007)
Gueguen, L., Sergeev, A., Kadlec, B., Liu, R., Yosinski, J.: Faster neural networks straight from JPEG. In: NeurIPS (2018)
Guo, T., Cherukuri, V., Monga, V.: Dense123’color enhancement dehazing network. In: CVPRW (2019)
Ham, B., Cho, M., Ponce, J.: Robust image filtering using joint static and dynamic guidance. In: CVPR (2015)
He, B., Wang, C., Shi, B., Duan, L.Y.: Mop moire patterns using MopNet. In: ICCV (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Hu, X., Fu, C.W., Zhu, L., Heng, P.A.: Depth-attentional features for single-image rain removal. In: CVPR (2019)
Hu, X., Fu, C.W., Zhu, L., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection and removal. TPAMI (2019)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR (2017)
Huang, H., He, R., Sun, Z., Tan, T.: Wavelet-SRNet: a wavelet-based CNN for multi-scale face super resolution. In: ICCV (2017)
Isobe, T., et al.: Video super-resolution with temporal group attention. In: CVPR (2020)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)
Jang, D.W., Park, R.H.: DenseNet with deep residual channel-attention blocks for single image super resolution. In: CVPRW (2019)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Karacan, L., Erdem, E., Erdem, A.: Structure-preserving image smoothing via region covariances. TOG 32, 1–11 (2013)
Kim, T.H., Park, S.I.: Deep context-aware descreening and rescreening of halftone images. TOG 37, 1–12 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lefkimmiatis, S.: Universal denoising networks: a novel CNN architecture for image denoising. In: CVPR (2018)
Levinskis, A.: Convolutional neural network feature reduction using wavelet transform. Elektronika ir Elektrotechnika 19, 61–64 (2013)
Li, Q., Shen, L., Guo, S., Lai, Z.: Wavelet integrated CNNs for noise-robust image classification. In: CVPR (2020)
Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV (2018)
Lin, H., Fu, X., Jing, C., Ding, X., Huang, Y.: A\(^2\)Net: Adjacent aggregation networks for image raindrop removal. arXiv preprint arXiv:1811.09780 (2018)
Liu, B., Shu, X., Wu, X.: Demoiréing of camera-captured screen images using deep convolutional neural network. arXiv preprint arXiv:1804.03809 (2018)
Liu, F., Yang, J., Yue, H.: Moiré pattern removal from texture images via low-rank and sparse matrix decomposition. In: VCIP (2015)
Liu, L., Jia, X., Liu, J., Tian, Q.: Joint demosaicing and denoising with self guidance. In: CVPR (2020)
Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-CNN for image restoration. In: CVPRW (2018)
Liu, Y., Li, Q., Sun, Z.: Attribute-aware face aging with wavelet-based generative adversarial networks. In: CVPR (2019)
Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: ICCV (2015)
Oyallon, E., Belilovsky, E., Zagoruyko, S.: Scaling the scattering transform: deep hybrid networks. In: ICCV (2017)
Qian, R., Tan, R.T., Yang, W., Su, J., Liu, J.: Attentive generative adversarial network for raindrop removal from a single image. In: CVPR (2018)
Qin, J., Xie, Z., Shi, Y., Wen, W.: Difficulty-aware image super resolution via deep adaptive dual-network. In: ICME (2019)
Quan, Y., Deng, S., Chen, Y., Ji, H.: Deep learning for seeing through window with raindrops. In: ICCV (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sasada, R., Yamada, M., Hara, S., Takeo, H., Shimura, K.: Stationary grid pattern removal using 2D technique for moiré-free radiographic image display. In: Medical Imaging (2003)
Shen, L., Yue, Z., Chen, Q., Feng, F., Ma, J.: Deep joint rain and haze removal from a single image. In: ICPR (2018)
Si, J., et al.: Dual attention matching network for context-aware feature sequence based person re-identification. In: CVPR (2018)
Siddiqui, H., Boutin, M., Bouman, C.A.: Hardware-friendly descreening. TIP 19, 746–757 (2009)
Sidorov, D.N., Kokaram, A.C.: Removing moire from degraded video archives. In: ESPC (2002)
Sidorov, D.N., Kokaram, A.C.: Suppression of moiré patterns via spectral analysis. In: VCIP (2002)
Su, Z., Luo, X., Deng, Z., Liang, Y., Ji, Z.: Edge-preserving texture suppression filter based on joint filtering schemes. TMM 15, 535–548 (2012)
Subr, K., Soler, C., Durand, F.: Edge-preserving multiscale image decomposition based on local extrema. TOG 28, 1–9 (2009)
Sun, B., Li, S., Sun, J.: Scanned image descreening with image redundancy and adaptive filtering. TIP 23, 3698–3710 (2014)
Sun, Y., Schaefer, S., Wang, W.: Image structure retrieval via \( l\_0 \) minimization. TVCG 24, 2129–2139 (2017)
Sun, Y., Yu, Y., Wang, W.: Moiré photo restoration using multiresolution convolutional neural networks. TIP 27, 4160–4172 (2018)
Tai, Y., Yang, J., Liu, X., Xu, C.: MemNet: a persistent memory network for image restoration. In: ICCV (2017)
Wang, P., et al.: Understanding convolution for semantic segmentation. In: WACV (2018)
Wei, Z., Wang, J., Nichol, H., Wiebe, S., Chapman, D.: A median-Gaussian filtering framework for Moiré pattern noise removal from X-ray microscopy image. Micron 43, 170–176 (2012)
Williams, T., Li, R.: Wavelet pooling for convolutional neural networks. In: ICLR (2018)
Xu, L., Lu, C., Xu, Y., Jia, J.: Image smoothing via L0 gradient minimization. TOG 30, 11 (2011)
Xu, L., Yan, Q., Xia, Y., Jia, J.: Structure extraction from texture via relative total variation. TOG 31, 1–10 (2012)
Xu, X., et al.: Rendering portraitures from monocular camera and beyond. In: ECCV (2018)
Yang, J., Liu, F., Yue, H., Fu, X., Hou, C., Wu, F.: Textured image demoiréing via signal decomposition and guided filtering. TIP 26, 3528–3541 (2017)
Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: CVPR (2017)
Yoo, J., Uh, Y., Chun, S., Kang, B., Ha, J.W.: Photorealistic style transfer via wavelet transforms. In: CVPR (2019)
Yuan, S., Timofte, R., Leonardis, A., Slabaugh, G.: NTIRE 2020 challenge on image demoireing: methods and results. In: CVPRW (2020)
Yuan, S., Timofte, R., Slabaugh, G., Leonardis, A.: AIM 2019 challenge on image demoireing: dataset and study. In: ICCVW (2019)
Yuan, S., et al.: AIM 2019 challenge on image demoireing: methods and results. In: ICCVW (2019)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. TIP 26, 3142–3155 (2017)
Zhang, Q., Shen, X., Xu, L., Jia, J.: Rolling guidance filter. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 815–830. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_53
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR (2018)
Zheng, B., Yuan, S., Slabaugh, G., Leonardis, A.: Image demoireing with learnable bandpass filters. In: CVPR (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, L. et al. (2020). Wavelet-Based Dual-Branch Network for Image Demoiréing. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12358. Springer, Cham. https://doi.org/10.1007/978-3-030-58601-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-58601-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58600-3
Online ISBN: 978-3-030-58601-0
eBook Packages: Computer ScienceComputer Science (R0)