Skip to main content
SpringerLink
Log in

Text-Aware Single Image Specular Highlight Removal

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13022))

Included in the following conference series:

Abstract

Removing undesirable specular highlight from a single input image is of crucial importance to many computer vision and graphics tasks. Existing methods typically remove specular highlight for medical images and specific-object images, however, they cannot handle the images with text. In addition, the impact of specular highlight on text recognition is rarely studied by text detection and recognition community. Therefore, in this paper, we first raise and study the text-aware single image specular highlight removal problem. The core goal is to improve the accuracy of text detection and recognition by removing the highlight from text images. To tackle this challenging problem, we first collect three high-quality datasets with fine-grained annotations, which will be appropriately released to facilitate the relevant research. Then, we design a novel two-stage network, which contains a highlight detection network and a highlight removal network. The output of highlight detection network provides additional information about highlight regions to guide the subsequent highlight removal network. Moreover, we suggest a measurement set including the end-to-end text detection and recognition evaluation and auxiliary visual quality evaluation. Extensive experiments on our collected datasets demonstrate the superior performance of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chineseocr: Ctpn plus densenet plus ctc based chinese ocr. https://github.com/YCG09/chinese_ocr. Accessed 30 Apr 2021

  2. Paddleocr: Awesome multilingual ocr toolkits based on paddlepaddle. https://github.com/PaddlePaddle/PaddleOCR. Accessed 30 Apr 2021

  3. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)

    Article  Google Scholar 

  4. Arnold, M., Ghosh, A., Ameling, S., Lacey, G.: Automatic segmentation and inpainting of specular highlights for endoscopic imaging. J. Image Video Process. (2010)

    Google Scholar 

  5. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: ACM SIGGRAPH, pp. 417–424 (2000)

    Google Scholar 

  6. Fleyeh, H.: Shadow and highlight invariant colour segmentation algorithm for traffic signs. In: IEEE Conference on Cybernetics and Intelligent Systems (2006)

    Google Scholar 

  7. Fu, G., Zhang, Q., Song, C., Lin, Q., Xiao, C.: Specular highlight removal for real-world images. Comput. Graph. Forum 38(7), 253–263 (2019)

    Article  Google Scholar 

  8. Funke, I., Bodenstedt, S., Riediger, C., Weitz, J., Speidel, S.: Generative adversarial networks for specular highlight removal in endoscopic images. In: Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10576, pp. 8–16 (2018)

    Google Scholar 

  9. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)

    Google Scholar 

  10. Guo, X., Chen, Z., Li, S., Yang, Y., Yu, J.: Deep eyes: binocular depth-from-focus on focal stack pairs. In: Chinese Conference on Pattern Recognition and Computer Vision, pp. 353–365 (2019)

    Google Scholar 

  11. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the European Conference on Computer Vision, pp. 694–711 (2016)

    Google Scholar 

  12. Khanian, M., Boroujerdi, A.S., Breuß, M.: Photometric stereo for strong specular highlights. arXiv preprint arXiv:1709.01357 (2017)

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)

    Google Scholar 

  14. Lee, S.T., Yoon, T.H., Kim, K.S., Kim, K.D., Park, W.: Removal of specular reflections in tooth color image by perceptron neural nets. In: International Conference on Signal Processing Systems, vol. 1, pp. V1–285-V1-289 (2010)

    Google Scholar 

  15. Lin, J., El Amine Seddik, M., Tamaazousti, M., Tamaazousti, Y., Bartoli, A.: Deep multi-class adversarial specularity removal. In: Image Analysis, pp. 3–15 (2019)

    Google Scholar 

  16. Long, S., He, X., Yao, C.: Scene text detection and recognition: The deep learning era. Int. J. Comput. Vis. 129(1), 161–184 (2021)

    Article  Google Scholar 

  17. Lucas, S., Panaretos, A., Sosa, L., Tang, A., Wong, S., Young, R.: Icdar 2003 robust reading competitions. In: International Conference on Document Analysis and Recognition, pp. 682–687 (2003)

    Google Scholar 

  18. Meslouhi, O.E., Kardouchi, M., Allali, H., Gadi, T., Benkaddour, Y.A.: Automatic detection and inpainting of specular reflections for colposcopic images. Central Eur. J. Comput. Sci. 1 (2011)

    Google Scholar 

  19. Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  20. Muhammad, S., Dailey, M.N., Farooq, M., Majeed, M.F., Ekpanyapong, M.: Spec-net and spec-cgan: Deep learning models for specularity removal from faces. Image Vis. Comput. 93, 103823 (2020)

    Google Scholar 

  21. Ortiz, F., Torres, F.: A new inpainting method for highlights elimination by colour morphology. In: International Conference on Pattern Recognition and Image Analysis, pp. 368–376 (2005)

    Google Scholar 

  22. Park, J.W., Lee, K.H.: Inpainting highlights using color line projection. IEICE Trans. Inf. Syst. 90(1), 250–257 (2007)

    Article  Google Scholar 

  23. Ren, W., Tian, J., Tang, Y.: Specular reflection separation with color-lines constraint. IEEE Trans. Image Process. 26(5), 2327–2337 (2017)

    Article  MathSciNet  Google Scholar 

  24. Shafer, S.A.: Using color to separate reflection components. Color. Res. Appl. 10(4), 210–218 (1985)

    Article  Google Scholar 

  25. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  26. Son, M., Lee, Y., Chang, H.S.: Toward specular removal from natural images based on statistical reflection models. IEEE Trans. Image Process. (2020)

    Google Scholar 

  27. Tan, P., Lin, S., Quan, L., Shum, H.Y.: Highlight removal by illumination-constrained inpainting. In: IEEE International Conference on Computer Vision, pp. 164–169 (2003)

    Google Scholar 

  28. Tan, R.T., Nishino, K., Ikeuchi, K.: Separating reflection components based on chromaticity and noise analysis. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1373–1379 (2004)

    Article  Google Scholar 

  29. Tian, Z., Huang, W., He, T., He, P., Qiao, Y.: Detecting text in natural image with connectionist text proposal network. In: European Conference on Computer Vision, pp. 56–72 (2016)

    Google Scholar 

  30. Wang, T.C., Efros, A.A., Ramamoorthi, R.: Occlusion-aware depth estimation using light-field cameras. In: IEEE International Conference on Computer Vision, pp. 3487–3495 (2015)

    Google Scholar 

  31. Wang, W., Deng, R., Li, L., Xu, X.: Image aesthetic assessment based on perception consistency. In: Chinese Conference on Pattern Recognition and Computer Vision, pp. 303–315 (2019)

    Google Scholar 

  32. Yang, Q., Wang, S., Ahuja, N.: Real-time specular highlight removal using bilateral filtering. In: European Conference on Computer Vision, pp. 87–100 (2010)

    Google Scholar 

  33. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2242–2251 (2017)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China (2019YFB2204104), and the National Natural Science Foundation of China (Nos. 6210071649, 62172415 and 61772523).

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Shiyu Hou, Chaoqun Wang, Weize Quan, Jingen Jiang & Dong-Ming Yan

  2. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, 100049, China

    Shiyu Hou, Chaoqun Wang, Weize Quan, Jingen Jiang & Dong-Ming Yan

Authors
  1. Shiyu Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoqun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weize Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jingen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dong-Ming Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dong-Ming Yan .

Editor information

Editors and Affiliations

  1. University of Science and Technology Beijing, Beijing, China

    Huimin Ma

  2. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  3. Tsinghua University, Beijing, China

    Changshui Zhang

  4. Zhejiang University, Hangzhou, China

    Fei Wu

  5. Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  6. Hunan University, Changsha, China

    Yaonan Wang

  7. Sun Yat-Sen University, Guangzhou, Guangdong, China

    Jianhuang Lai

  8. Beijing Jiaotong University, Beijing, China

    Yao Zhao

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hou, S., Wang, C., Quan, W., Jiang, J., Yan, DM. (2021). Text-Aware Single Image Specular Highlight Removal. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13022. Springer, Cham. https://doi.org/10.1007/978-3-030-88013-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88013-2_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88012-5

  • Online ISBN: 978-3-030-88013-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation