Analytical Survey of Highlight Detection in Color and Spectral Images

  • Haris Ahmad Khan
  • Jean-Baptiste Thomas
  • Jon Yngve Hardeberg
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10213)

Abstract

Detection of highlights is a prominent issue in computer vision, graphics and image processing. Applications which require object properties measurement or rendering are affected by specular reflection since the models assume matte diffusing surfaces most of the time. Hence, detection, and sometimes removal, of specular reflection (highlights) in an image may be critical. Several methods are proposed for addressing this issue. In this paper, we present a review and analysis of these techniques in color and spectral images.

Keywords

Image analysis Highlights detection Specular reflection Diffuse reflection Spectral imaging 

References

  1. 1.
    Barrow, H., Tanenbaum, J.: Recovering intrinsic scene characteristic from images. In: Computer Vision System, pp. 3–26 (1978)Google Scholar
  2. 2.
    Lee, H.C., Breneman, E.J., Schulte, C.P.: Modeling light reflection for computer color vision. IEEE Trans. Pattern Anal. Mach. Intell. 12, 402–409 (1990)CrossRefGoogle Scholar
  3. 3.
    Tan, P., Quan, L., Lin, S.: Separation of highlight reflections on textured surfaces. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2006, Washington, D.C., vol. 2, pp. 1855–1860. IEEE Computer Society (2006)Google Scholar
  4. 4.
    Tan, R.T., Ikeuchi, K.: Separating Reflection Components of Textured Surfaces Using a Single Image. Digitally Archiving Cultural Objects, pp. 353–384. Springer, Boston (2008)Google Scholar
  5. 5.
    Artusi, A., Banterle, F., Chetverikov, D.: A survey of specularity removal methods. Comput. Graph. Forum 30(8), 2208–2230 (2011)CrossRefGoogle Scholar
  6. 6.
    Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10(4), 210–218 (1985)CrossRefGoogle Scholar
  7. 7.
    Park, J.W., Lee, K.H.: Inpainting highlights using color line projection. IEICE Trans. Inf. Syst. E90–D, 250–257 (2007)CrossRefGoogle Scholar
  8. 8.
    Budianto, Lun, D.P.K.: Inpainting for fringe projection profilometry based on geometrically guided iterative regularization. IEEE Trans. Image Process. 24, 5531–5542 (2015)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Ortiz, F., Torres, F.: A new inpainting method for highlights elimination by colour morphology. In: Singh, S., Singh, M., Apte, C., Perner, P. (eds.) ICAPR 2005. LNCS, vol. 3687, pp. 368–376. Springer, Heidelberg (2005). doi:10.1007/11552499_42 CrossRefGoogle Scholar
  10. 10.
    Tan, P., Lin, S., Quan, L., Shum, H.-Y.: Highlight removal by illumination-constrained inpainting. In: Proceedings of 9th IEEE International Conference on Computer Vision, vol. 1, pp. 164–169, October 2003Google Scholar
  11. 11.
    Klinker, G., Shafer, S., Kanade, T.: Using a color reflection model to separate highlights from object color. In: Proceedings of 1st International Conference on Computer Vision, pp. 145–150 (1991)Google Scholar
  12. 12.
    Klinker, G.J., Shafer, S.A., Kanade, T.: The measurement of highlights in color images. Int. J. Comput. Vis. 2(1), 7–32 (1988)CrossRefGoogle Scholar
  13. 13.
    Klinker, G.J., Shafer, S.A., Kanade, T.: A physical approach to color image understanding. Int. J. Comput. Vis. 4(1), 7–38 (1990)CrossRefGoogle Scholar
  14. 14.
    Schlüns, K., Teschner, M.: Analysis of 2D color spaces for highlight elimination in 3D shape reconstruction. In: Proceedings of the Asian Conference on Computer Vision, vol. 2, pp. 801–805 (1995)Google Scholar
  15. 15.
    Schlüns, K., Teschner, M.: Fast separation of reflection components and its application in 3D shape recovery. Color Imaging Conf. (1), 48–51 (1995)Google Scholar
  16. 16.
    Bajcsy, R., Lee, S.W., Leonardis, A.: Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation. Int. J. Comput. Vis. 17(3), 241–272 (1996)CrossRefGoogle Scholar
  17. 17.
    Mallick, S.P., Zickler, T., Belhumeur, P.N., Kriegman, D.J.: Specularity removal in images and videos: a PDE approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 550–563. Springer, Heidelberg (2006). doi:10.1007/11744023_43 CrossRefGoogle Scholar
  18. 18.
    Yang, J., Cai, Z., Wen, L., Lei, Z., Guo, G., Li, S.Z.: A new projection space for separation of specular-diffuse reflection components in color images. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7727, pp. 418–429. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37447-0_32 CrossRefGoogle Scholar
  19. 19.
    Yang, J., Liu, L., Li, S.Z.: Separating specular and diffuse reflection components in the HSI color space. In: IEEE International Conference on Computer Vision Workshops, pp. 891–898, December 2013Google Scholar
  20. 20.
    Akashi, Y., Okatani, T.: Separation of reflection components by sparse non-negative matrix factorization. Comput. Vis. Image Underst. 146, 77–85 (2016)CrossRefGoogle Scholar
  21. 21.
    Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 178–193 (2005)CrossRefGoogle Scholar
  22. 22.
    Yoon, K.J., Choi, Y., Kweon, I.S.: Fast separation of reflection components using a specularity-invariant image representation. In: International Conference on Image Processing, pp. 973–976, October 2006Google Scholar
  23. 23.
    Shen, H.-L., Cai, Q.-Y.: Simple and efficient method for specularity removal in an image. Appl. Opt. 48, 2711–2719 (2009)CrossRefGoogle Scholar
  24. 24.
    Shen, H.-L., Zhang, H.-G., Shao, S.-J., Xin, J.H.: Chromaticity-based separation of reflection components in a single image. Pattern Recogn. 41(8), 2461–2469 (2008)CrossRefMATHGoogle Scholar
  25. 25.
    Liu, Y., Yuan, Z., Zheng, N., Wu, Y.: Saturation-preserving specular reflection separation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3725–3733, June 2015Google Scholar
  26. 26.
    Yang, Q., Wang, S., Ahuja, N.: Real-time specular highlight removal using bilateral filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 87–100. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15561-1_7 CrossRefGoogle Scholar
  27. 27.
    Yang, Q., Tang, J., Ahuja, N.: Efficient and robust specular highlight removal. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1304–1311 (2015)CrossRefGoogle Scholar
  28. 28.
    Kim, H., Jin, H., Hadap, S., Kweon, I.: Specular reflection separation using dark channel prior. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1460–1467, June 2013Google Scholar
  29. 29.
    Suo, J., An, D., Ji, X., Wang, H., Dai, Q.: Fast and high quality highlight removal from a single image. IEEE Trans. Image Process. 25, 5441–5454 (2016)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Shen, H.-L., Zheng, Z.-H.: Real-time highlight removal using intensity ratio. Appl. Opt. 52, 4483–4493 (2013)CrossRefGoogle Scholar
  31. 31.
    Lee, S.W., Bajcsy, R.: Detection of specularity using colour and multiple views. Image Vis. Comput. 10(10), 643–653 (1992)CrossRefGoogle Scholar
  32. 32.
    Sato, Y., Ikeuchi, K.: Temporal-color space analysis of reflection. J. Opt. Soc. Am. A 11, 2990–3002 (1994)CrossRefGoogle Scholar
  33. 33.
    Lin, S., Shum, H.-Y.: Separation of diffuse and specular reflection in color images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 341–346 (2001)Google Scholar
  34. 34.
    Lin, S., Li, Y., Kang, S.B., Tong, X., Shum, H.-Y.: Diffuse-specular separation and depth recovery from image sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 210–224. Springer, Heidelberg (2002). doi:10.1007/3-540-47977-5_14 CrossRefGoogle Scholar
  35. 35.
    Weiss, Y.: Deriving intrinsic images from image sequences. In: Proceedings Eighth IEEE International Conference on Computer Vision, ICCV, vol. 2, pp. 68–75 (2001)Google Scholar
  36. 36.
    Feris, R., Raskar, R., Tan, K.-H., Turk, M.: Specular highlights detection and reduction with multi-flash photography. J. Braz. Comput. Soci. 12(1), 35–42 (2006)CrossRefGoogle Scholar
  37. 37.
    Agrawal, A., Raskar, R., Nayar, S.K., Li, Y.: Removing photography artifacts using gradient projection and flash-exposure sampling. ACM Trans. Graph. 24, 828–835 (2005)CrossRefGoogle Scholar
  38. 38.
    Chen, T., Goesele, M., Seidel, H.P.: Mesostructure from specularity. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 1825–1832 (2006)Google Scholar
  39. 39.
    Yang, Q., Wang, S., Ahuja, N., Yang, R.: A uniform framework for estimating illumination chromaticity, correspondence, and specular reflection. IEEE Trans. Image Process. 20, 53–63 (2011)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Wang, C., Kamata, S.I., Ma, L.: A fast multi-view based specular removal approach for pill extraction. In: IEEE International Conference on Image Processing, pp. 4126–4130, September 2013Google Scholar
  41. 41.
    Prinet, V., Werman, M., Lischinski, D.: Specular highlight enhancement from video sequences. In: IEEE International Conference on Image Processing, pp. 558–562, September 2013Google Scholar
  42. 42.
    Wang, H., Xu, C., Wang, X., Zhang, Y., Peng, B.: Light field imaging based accurate image specular highlight removal. PLOS ONE 11, 1–17 (2016)Google Scholar
  43. 43.
    Nayar, S.K., Fang, X.-S., Boult, T.: Separation of reflection components using color and polarization. Int. J. Comput. Vis. 21(3), 163–186 (1997)CrossRefGoogle Scholar
  44. 44.
    Wolff, L.B.: Classification of material surfaces using the polarization of specular highlights. In: Proceedings of SPIE - The International Society for Optical Engineering, vol. 1005, pp. 206–213, January 1988Google Scholar
  45. 45.
    Wolff, L.B.: Polarization-based material classification from specular reflection. IEEE Trans. Pattern Anal. Mach. Intell. 12, 1059–1071 (1990)CrossRefGoogle Scholar
  46. 46.
    Kim, D.W., Lin, S., Hong, K.-S., Shum, H.: Variational specular separation using color and polarization. In: Proceedings of the IAPR Workshop on Machine Vision Applications (2002)Google Scholar
  47. 47.
    Atkinson, G.A., Hancock, E.R.: Recovery of surface orientation from diffuse polarization. IEEE Trans. Image Process. 15, 1653–1664 (2006)CrossRefGoogle Scholar
  48. 48.
    Atkinson, G.A., Hancock, E.R.: Two-dimensional BRDF estimation from polarisation. Comput. Vis. Image Underst. 111(2), 126–141 (2008)CrossRefGoogle Scholar
  49. 49.
    Müller, V.: Polarization-based separation of diffuse and specular surface-reflection. In: Sagerer, G., Posch, S., Kummert, F. (eds.) Mustererkennung 1995: Verstehen akustischer und visueller Informationen. Informatik aktuell, pp. 202–209. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  50. 50.
    Umeyama, S., Godin, G.: Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images. IEEE Trans. Pattern Anal. Mach. Intell. 26, 639–647 (2004)CrossRefGoogle Scholar
  51. 51.
    Lamond, B., Peers, P., Debevec, P.: Fast image-based separation of diffuse and specular reflections. In: ACM SIGGRAPH Sketches, SIGGRAPH 2007. ACM, New York (2007)Google Scholar
  52. 52.
    Zhang, L., Hancock, E.R., Atkinson, G.A.: Reflection component separation using statistical analysis and polarisation. In: Vitrià, J., Sanches, J.M., Hernández, M. (eds.) IbPRIA 2011. LNCS, vol. 6669, pp. 476–483. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21257-4_59 CrossRefGoogle Scholar
  53. 53.
    Bochko, V., Parkkinen, J.: Highlight analysis using a mixture model of probabilistic PCA. In: Proceedings of the 4th WSEAS International Conference on Signal Processing, Robotics and Automation, ISPRA 2005, Stevens Point, Wisconsin, pp. 15:1–15:5. World Scientific and Engineering Academy and Society (WSEAS) (2005)Google Scholar
  54. 54.
    Fu, Z., Tan, R.T., Caelli, T.: Specular free spectral imaging using orthogonal subspace projection. In: 18th International Conference on Pattern Recognition (ICPR), vol. 1, pp. 812–815 (2006)Google Scholar
  55. 55.
    Koirala, P., Pant, P., Hauta-Kasari, M., Parkkinen, J.: Highlight detection and removal from spectral image. J. Opt. Soc. Am. A 28, 2284–2291 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Haris Ahmad Khan
    • 1
    • 2
  • Jean-Baptiste Thomas
    • 1
    • 2
  • Jon Yngve Hardeberg
    • 1
  1. 1.The Norwegian Colour and Visual Computing LaboratoryNTNU - Norwegian University of Science and TechnologyGjøvikNorway
  2. 2.Le2i, FRE CNRS 2005, Univ. Bourgogne Franche-ComtéDijonFrance

Personalised recommendations