Abstract
Specular highlight usually causes serious information degradation, which leads to the failure of many computer vision algorithms. We have proposed a novel bifurcated convolution neural network to tackle the problem of high reflectivity image information degradation. A two-stage process is proposed for the extraction and elimination of the specular highlight features, with the procedure starting at a coarse level and progressing towards a finer level, to ensure the generated diffuse images are less affected by visual artifacts and information distortions. A bifurcated feature selection module is designed to remove the specular highlight features, thereby enhancing the detection capability of the network. The experiments on two types of challenging datasets demonstrate that our method outperforms state-of-the-art approaches for specular highlight detection and removal. The effectiveness of the proposed bifurcated feature selection module and the overall network is also verified.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
SHI J, DONG Y, SU H, et al. Learning non-Lambertian object intrinsics across ShapeNet categories[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 21–26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 1063–6919.
WEI X, XU X B, ZHANG J W, et al. Specular highlight reduction with known surface geometry[J]. Computer vision and image understanding, 2018, 168: 132–144.
YANG Q X, TANG J H, AHUJA N. Efficient and robust specular highlight removal[J]. IEEE transaction on pattern analysis and machine intelligence, 2014, 37(6): 1304–1311.
WEI X, XU X B, ZHANG J W, et al. Specular highlight reduction with known surface geometry[J]. Computer vision and image understanding, 2018, 168: 132–144.
LI R Y, PAN J J, SI Y Q, et al. Specular refmections removal for endoscopic image sequences with adaptive-RPCA decomposition[J]. IEEE transactions on medical imaging, 2020, 39(2): 328–340.
WU Z Q, ZHUANG C Q, SHI J, et al. Single-image specular highlight removal via real-world dataset construction[C]//Proceedings of the IEEE Transactions on Multimedia, August 27, 2021. New York: IEEE, 2021: 3782–3793.
WANG X C, TAO C N, TAO X, et al. SIHRNet: a fully convolutional network for single image highlight removal with a real-world dataset[J]. Journal of electronic imaging, 2022, 31: 033013.
FU G, ZHANG Q, ZHU L, et al. A multi-task network for joint specular highlight detection and removal[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 20–25, 2021, Nashville, TN, USA. New York: IEEE, 2021: 7748–7757.
XU J T, LIU S, CHEN G Z, et al. Highlight detection and removal method based on bifurcated-CNN[C]//Proceedings of the EI Conference on Intelligent Robotics and Applications, August 10, 2022, Harbin, China. New York: EI, 2022: 307–318.
WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision, August 22–25, 2018, Munich, Germany. Berlin, Heidelberg: Springer-Verlag, 2018: 3–19.
ZHANG X, NG R, CHEN Q. Single image reflection separation with perceptual losses[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 18–22, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 4786–4794.
WEI K, YANG J, FU Y, et al. Single image reflection removal exploiting misaligned training data and network enhancements[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 16–20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 8178–8187.
LIN J, EL AMINE SEDDIK M, TAMAAZOUSTI M, et al. Deep multi-class adversarial specularity removal[C]//Proceedings of the 21st Scandinavian Conference on Image Analysis, June 11–13, 2019, Norrköping, Sweden. Berlin, Heidelberg: Springer-Verlag, 2019: 11482.
FUNKE I, SEBASTIAN B, CARINA R, et al. Generative adversarial networks for specular highlight removal in endoscopic images[C[//Proceedings of the SPIE, March, 2018, Houston, Texas, USA. Washington: SPIE, 2018: 10576(9).
YAMAMOTO T, KITAJIMA T, KAWAUCHI R. Efficient improvement method for separation of reflection components based on an energy function[C]//Proceedings of the IEEE International Conference on Image Processing, September 17–20, 2017, Beijing, China. Beijing: IEEE, 2017: 4222–4226.
FAN D P, ZHAI Y G, ALI B, et al. BBS-Net: RGB-D salient object detection with a bifurcated backbone strategy network[C]//Proceedings of the European Conference on Computer Vision, October 23–27, 2020, Tel Aviv, Israel. Berlin, Heidelberg: Springer-Verlag, 2020: 275–292.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare no conflict of interest.
Additional information
This work has been supported by the National Key R&D Program of China (No.2018YFB1305200).
Rights and permissions
About this article
Cite this article
Xu, J., Liu, S., Chen, G. et al. Bifurcated convolutional network for specular highlight removal. Optoelectron. Lett. 19, 756–761 (2023). https://doi.org/10.1007/s11801-023-3029-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11801-023-3029-6