Embedded Polarizing Filters to Separate Diffuse and Specular Reflection

  • Laurent Valentin Jospin
  • Gilles BaechlerEmail author
  • Adam Scholefield
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11362)


Polarizing filters provide a powerful way to separate diffuse and specular reflection; however, traditional methods rely on several captures and require proper alignment of the filters. Recently, camera manufacturers have proposed to embed polarizing micro-filters in front of the sensor, creating a mosaic of pixels with different polarizations. In this paper, we investigate the advantages of such camera designs. In particular, we consider different design patterns for the filter arrays and propose an algorithm to demosaic an image generated by such cameras. This essentially allows us to separate the diffuse and specular components using a single image. The performance of our algorithm is compared with a color-based method using synthetic and real data. Finally, we demonstrate how we can recover the normals of a scene using the diffuse images estimated by our method.


Polarizing micro-filter Diffuse and specular separation Photometric stereo 


  1. 1.
    Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19, 191139 (1980)CrossRefGoogle Scholar
  2. 2.
    Artusi, A., Banterle, F., Chetverikov, D.: A survey of specularity removal methods. Comput. Graph. Forum 30, 2208–2230 (2011)CrossRefGoogle Scholar
  3. 3.
    Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10, 210–218 (1985)CrossRefGoogle Scholar
  4. 4.
    Klinker, G.J., Shafer, S.A., Kanade, T.: The measurement of highlights in color images. Int. J. Comput. Vis. 2, 7–32 (1988)CrossRefGoogle Scholar
  5. 5.
    Gershon, R.: The use of color in computational vision. Technical report, Department of Computer Science, University of Toronto (1987)Google Scholar
  6. 6.
    Klinker, G.J., Shafer, S.A., Kanade, T.: A physical approach to color image understanding. Int. J. Comput. Vis. 4, 7–38 (1990)CrossRefGoogle Scholar
  7. 7.
    Mallick, S.P., Zickler, T., Belhumeur, P.N., Kriegman, D.J.: Specularity removal in images and videos: a PDE approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 550–563. Springer, Heidelberg (2006). Scholar
  8. 8.
    Yang, J., Liu, L., Li, S.Z.: Separating specular and diffuse reflection components in the HSI color space. In: IEEE International Conference on Computer Vision Workshops, pp. 891–898 (2013)Google Scholar
  9. 9.
    Mallick, S.P., Zickler, T.E., Kriegman, D.J., Belhumeur, P.N.: Beyond lambert: reconstructing specular surfaces using color. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 619–626. IEEE (2005)Google Scholar
  10. 10.
    Bajcsy, R., Lee, S.W., Leonardis, A.: Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation. Int. J. Comput. Vis. 17, 241–272 (1996)CrossRefGoogle Scholar
  11. 11.
    Shen, H.L., Zheng, Z.H.: Real-time highlight removal using intensity ratio. Appl. Opt. 52, 4483–4493 (2013)CrossRefGoogle Scholar
  12. 12.
    Tan, R.T., Nishino, K., Ikeuchi, K.: Separating reflection components based on chromaticity and noise analysis. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1373–1379 (2004)CrossRefGoogle Scholar
  13. 13.
    Mallick, S.P., Zickler, T., Belhumeur, P., Kriegman, D.: Dichromatic separation: specularity removal and editing. In: ACM SIGGRAPH 2006 Sketches, p. 166. ACM (2006)Google Scholar
  14. 14.
    Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 178–193 (2005)CrossRefGoogle Scholar
  15. 15.
    Yoon, K.J., Choi, Y., Kweon, I.S.: Fast separation of reflection components using a specularity-invariant image representation. In: IEEE International Conference on Image Processing, pp. 973–976. IEEE (2006)Google Scholar
  16. 16.
    Lamond, B., Peers, P., Ghosh, A., Debevec, P.: Image-based separation of diffuse and specular reflections using environmental structured illumination. In: IEEE Computational Photography (2009)Google Scholar
  17. 17.
    Ma, W.C., Hawkins, T., Peers, P., Chabert, C.F., Weiss, M., Debevec, P.: Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In: Eurographics Symposium on Rendering, pp. 183–194. Eurographics Association (2007)Google Scholar
  18. 18.
    Nayar, S.K., Krishnan, G., Grossberg, M.D., Raskar, R.: Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans. Graph. 25, 935–944 (2006)CrossRefGoogle Scholar
  19. 19.
    Jaklič, A., Solina, F.: Separating diffuse and specular component of image irradiance by translating a camera. In: Chetverikov, D., Kropatsch, W.G. (eds.) CAIP 1993. LNCS, vol. 719, pp. 428–435. Springer, Heidelberg (1993). Scholar
  20. 20.
    Lin, S., Li, Y., Kang, S.B., Tong, X., Shum, H.Y.: Diffuse-specular separation and depth recovery from image sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 210–224. Springer, Heidelberg (2002). Scholar
  21. 21.
    Meng, L., Lu, L., Bedard, N., Berkner, K.: Single-shot specular surface reconstruction with gonio-plenoptic imaging. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3433–3441 (2015)Google Scholar
  22. 22.
    Wang, H., Xu, C., Wang, X., Zhang, Y., Peng, B.: Light field imaging based accurate image specular highlight removal. PLOS ONE 11, e0156173 (2016)CrossRefGoogle Scholar
  23. 23.
    Nayar, S.K., Fang, X.S., Boult, T.: Removal of specularities using color and polarization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 583–590 (1993)Google Scholar
  24. 24.
    Kim, D.W., Lin, S., Hong, K.S., Shum, H.Y.: Variational specular separation using color and polarization. In: MVA, pp. 176–179 (2002)Google Scholar
  25. 25.
    Nayar, S.K., Fang, X.S., Boult, T.: Separation of reflection components using color and polarization. Int. J. Comput. Vis. 21, 163–186 (1997)CrossRefGoogle Scholar
  26. 26.
    Umeyama, S., Godin, G.: Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images. IEEE Trans. Pattern Anal. Mach. Intell. 26, 639–647 (2004)CrossRefGoogle Scholar
  27. 27.
    Debevec, P., Hawkins, T., Tchou, C., Duiker, H.P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: SIGGRAPH, pp. 145–156 (2000)Google Scholar
  28. 28.
    4D Technologies: Polarization camera for image enhancement.
  29. 29.
    FluxData: Polarization imaging camera FD-1665P.
  30. 30.
    Photonic Lattice: Polarization imaging camera PI-110.
  31. 31.
    Ricoh Imaging Company Ltd.: Polarization camera.
  32. 32.
    Cui, Z., Gu, J., Shi, B., Tan, P., Kautz, J.: Polarimetric multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1558–1567 (2017)Google Scholar
  33. 33.
    Goldstein, T., Osher, S.: The split bregman method for l1-regularized problems. SIAM J. Img. Sci. 2, 323–343 (2009)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Blender Online Community: Blender - A 3D modelling and rendering package. Blender Foundation, Blender Institute, Amsterdam (2017)Google Scholar
  35. 35.
    Stanford Graphics Lab: The Stanford 3D Scanning Repository.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer and Communication Sciences, Audiovisual Communications LaboratoryEcole Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations