Advertisement

Deep Inverse Halftoning via Progressively Residual Learning

  • Menghan Xia
  • Tien-Tsin WongEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11366)

Abstract

Inverse halftoning as a classic problem has been investigated in the last two decades, however, it is still a challenge to recover the continuous version with accurate details from halftone images. In this paper, we present a statistic learning based method to address it, leveraging Convolutional Neural Network (CNN) as a nonlinear mapping function. To exploit features as completely as possible, we propose a Progressively Residual Learning (PRL) network that synthesizes the global tone and subtle details from the halftone images in a progressive manner. Particularly, it contains two modules: Content Aggregation that removes the halftone patterns and reconstructs the continuous tone firstly, and Detail Enhancement that boosts the subtle structures incrementally via learning a residual image. Benefiting from this efficient architecture, the proposed network is superior to all the candidate networks employed in our experiments for inverse halftoning. Also, our approach outperforms the state of the art with a large margin.

Keywords

Inverse halftoning Progressive learning 

Notes

Acknowledgement

This project is supported by Shenzhen Science and Technology Program (No. JCYJ20160429190300857) and Shenzhen Key Laboratory (No. ZDSYS201605101739178), and the Research Grants Council of the Hong Kong Special Administrative Region, under RGC General Research Fund (Project No. CUHK14201017).

Supplementary material

484523_1_En_33_MOESM1_ESM.mp4 (15.4 mb)
Supplementary material 1 (mp4 15750 KB)

References

  1. 1.
    Analoui, M., Allebach, J.: New results on reconstruction of continuous-tone from halftone. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASS) (1992)Google Scholar
  2. 2.
    Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  3. 3.
    Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: the Advances in Neural Information Processing Systems (NIPS) (2014)Google Scholar
  4. 4.
    Floyd, R.W.: An adaptive algorithm for spatial gray-scale. In: Society of Information Display (1976)Google Scholar
  5. 5.
    Freitas, P.G., Farias, M.C., Araujo, A.P.: Enhancing inverse halftoning via coupled dictionary training. Sig. Process.: Image Commun. 49, 1–8 (2016)Google Scholar
  6. 6.
    Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS) (2015)Google Scholar
  7. 7.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS) (2014)Google Scholar
  8. 8.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  9. 9.
    Hein, S., Zakhor, A.: Halftone to continuous-tone conversion of error-diffusion coded images. Sigma Delta Modulators 213, 133–154 (1993)CrossRefGoogle Scholar
  10. 10.
    Hou, X., Qiu, G.: Image companding and inverse halftoning using deep convolutional neural networks. arXiv preprint arXiv:1707.00116 (2017)
  11. 11.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  12. 12.
    Kim, Y.T., Gonzalo, R.A., Nikolai, G.: Inverse halftoning using binary permutation filters. IEEE Trans. Image Process. (TIP) 4(9), 1296–1311 (1995)CrossRefGoogle Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1511.06349 (2014)
  14. 14.
    Kipphan, H.: Handbook of Print Media. Springer, Heidelberg (2001).  https://doi.org/10.1007/978-3-540-29900-4CrossRefGoogle Scholar
  15. 15.
    Kite, T.D., Niranjan, D.V., Brian, L.E., Alan, C.B.: A high quality, fast inverse halftoning algorithm for error diffused halftones. In: Proceedings of the IEEE International Conference on Image Processing (ICIP) (1998)Google Scholar
  16. 16.
    Larsson, G., Maire, M., Shakhnarovich, G.: Learning representations for automatic colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 577–593. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_35CrossRefGoogle Scholar
  17. 17.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  18. 18.
    Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017)Google Scholar
  19. 19.
    Lizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. (TOG) 35(4), 110–121 (2016)Google Scholar
  20. 20.
    Loffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: IEEE International Conference on Machine Learning (ICML) (2015)Google Scholar
  21. 21.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  22. 22.
    Mese, M., Palghat, P.V.: Look-up table (LUT) method for inverse halftoning. IEEE Trans. Image Process. (TIP) 10(10), 1566–1578 (2001)CrossRefGoogle Scholar
  23. 23.
    Mese, M., Vaidyanathan, P.P.: Recent advances in digital halftoning and inverse halftoning methods. IEEE Trans. Circ. Syst. 49(6), 790–806 (2002)CrossRefGoogle Scholar
  24. 24.
    Mese, M., Vaidyanathan, P.P.: Recent advances in digital halftoning and inverse halftoning methods. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 49(6), 790–805 (2002)CrossRefGoogle Scholar
  25. 25.
    Mitsa, T., Parker, K.J.: Digital halftoning technique using a blue-noise mask. JOSA A 9(11), 1920–1929 (1992)CrossRefGoogle Scholar
  26. 26.
    Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  27. 27.
    Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  28. 28.
    Seldowitz, M.A., Allebach, J.P., Sweeney, D.W.: Synthesis of digital holograms by direct binary search. Appl. Opt. 26(14), 2788–2798 (1987)CrossRefGoogle Scholar
  29. 29.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  30. 30.
    Son, C.H.: Inverse halftoning based on sparse representation. Opt. Lett. 37(12), 2352–2354 (2012)CrossRefGoogle Scholar
  31. 31.
    Son, C.H., Choo, H.: Local learned dictionaries optimized to edge orientation for inverse halftoning. IEEE Trans. Image Process. (TIP) 23(6), 2542–2557 (2014)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Ulichney, R.A.: Dithering with blue noise. In: Proceedings of the IEEE (1988)Google Scholar
  33. 33.
    Wong, P.W.: Inverse halftoning and kernel estimation for error diffusion. IEEE Trans. Image Process. (TIP) 4(4), 486–498 (1995)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  35. 35.
    Xiong, Z., Michael, T.O., Kannan, R.: Inverse halftoning using wavelets. IEEE Trans. Image Process. (TIP) 8(10), 1479–1483 (1999)CrossRefGoogle Scholar
  36. 36.
    Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. (TIP) 26(7), 3142–3155 (2017)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.The Chinese University of Hong KongSha TinHong Kong
  2. 2.Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, SIATShenzhenChina

Personalised recommendations