Polarimetric Three-View Geometry

  • Lixiong Chen
  • Yinqiang ZhengEmail author
  • Art Subpa-asa
  • Imari Sato
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11220)


This paper theorizes the connection between polarization and three-view geometry. It presents a ubiquitous polarization-induced constraint that regulates the relative pose of a system of three cameras. We demonstrate that, in a multi-view system, the polarization phase obtained for a surface point is induced from one of the two pencils of planes: one by specular reflections with its axis aligned with the incident light; one by diffusive reflections with its axis aligned with the surface normal. Differing from the traditional three-view geometry, we show that this constraint directly encodes camera rotation and projection, and is independent of camera translation. In theory, six polarized diffusive point-point-point correspondences suffice to determine the camera rotations. In practise, a cross-validation mechanism using correspondences of specularites can effectively resolve the ambiguities caused by mixed polarization. The experiments on real world scenes validate our proposed theory.


  1. 1.
    Atkinson, G.A.: Two-source surface reconstruction using polarisation. In: Sharma, P., Bianchi, F.M. (eds.) SCIA 2017. LNCS, vol. 10270, pp. 123–135. Springer, Cham (2017). Scholar
  2. 2.
    Atkinson, G.A., Hancock, E.R.: Multi-view surface reconstruction using polarization. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, vol. 1, pp. 309–316. IEEE (2005)Google Scholar
  3. 3.
    Atkinson, G.A., Hancock, E.R.: Recovery of surface orientation from diffuse polarization. IEEE Trans. Image Process. 15(6), 1653–1664 (2006)CrossRefGoogle Scholar
  4. 4.
    Atkinson, G.A., Hancock, E.R.: Shape estimation using polarization and shading from two views. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 2001–2017 (2007)CrossRefGoogle Scholar
  5. 5.
    Atkinson, G.A., Hancock, E.R.: Surface reconstruction using polarization and photometric stereo. In: Kropatsch, W.G., Kampel, M., Hanbury, A. (eds.) CAIP 2007. LNCS, vol. 4673, pp. 466–473. Springer, Heidelberg (2007). Scholar
  6. 6.
    Berger, K., Voorhies, R., Matthies, L.H.: Depth from stereo polarization in specular scenes for urban robotics. In: IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 1966–1973. IEEE (2017)Google Scholar
  7. 7.
    Canterakis, N.: A minimal set of constraints for the trifocal tensor. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 84–99. Springer, Heidelberg (2000). Scholar
  8. 8.
    Cui, Z., Gu, J., Shi, B., Tan, P., Kautz, J.: Polarimetric multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1558–1567 (2017)Google Scholar
  9. 9.
    Drbohlav, O., Sara, R.: Unambiguous determination of shape from photometric stereo with unknown light sources. In: Proceedings. Eighth IEEE International Conference on Computer Vision, 2001. ICCV 2001, vol. 1, pp. 581–586. IEEE (2001)Google Scholar
  10. 10.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)Google Scholar
  11. 11.
    Hartley, R.I.: Lines and points in three views and the trifocal tensor. Int. J. Comput. Vis. 22(2), 125–140 (1997)CrossRefGoogle Scholar
  12. 12.
    Hartley, R.I., et al.: Projective reconstruction from line correspondences. In: CVPR, pp. 903–907 (1994)Google Scholar
  13. 13.
    Hecht, E.: Optics. Pearson Education (2016)Google Scholar
  14. 14.
    Heitz, E.: Understanding the masking-shadowing function in microfacet-based brdfs. J. Comput. Graph. Tech. 3(2), 32–91 (2014)Google Scholar
  15. 15.
    Huynh, C.P., Robles-Kelly, A., Hancock, E.: Shape and refractive index recovery from single-view polarisation images. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1229–1236. IEEE (2010)Google Scholar
  16. 16.
    Huynh, C.P., Robles-Kelly, A., Hancock, E.R.: Shape and refractive index from single-view spectro-polarimetric images. Int. J. Comput. Vis. 101(1), 64–94 (2013)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Polarized 3d: High-quality depth sensing with polarization cues. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3370–3378 (2015)Google Scholar
  18. 18.
    Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Depth sensing using geometrically constrained polarization normals. Int. J. Comput. Vis. 125(1–3), 34–51 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Kileel, J.: Minimal problems for the calibrated trifocal variety. SIAM J. Appl. Algebr. Geom. 1(1), 575–598 (2017)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Kneip, L., Siegwart, R., Pollefeys, M.: Finding the exact rotation between two images independently of the translation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 696–709. Springer, Heidelberg (2012). Scholar
  21. 21.
    Larsson, V., Aström, K., Oskarsson, M.: Efficient solvers for minimal problems by syzygy-based reduction. In: Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 3 (2017)Google Scholar
  22. 22.
    Leonardos, S., Tron, R., Daniilidis, K.: A metric parametrization for trifocal tensors with non-colinear pinholes. In: CVPR, pp. 259–267 (2015)Google Scholar
  23. 23.
    Mahmoud, A.H., El-Melegy, M.T., Farag, A.A.: Direct method for shape recovery from polarization and shading. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 1769–1772. IEEE (2012)Google Scholar
  24. 24.
    Martyushev, E.: On some properties of calibrated trifocal tensors. J. Math. Imaging Vis. 58(2), 321–332 (2017)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Miyazaki, D., Ikeuchi, K.: Shape estimation of transparent objects by using inverse polarization ray tracing. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 2018–2030 (2007)CrossRefGoogle Scholar
  26. 26.
    Miyazaki, D., Kagesawa, M., Ikeuchi, K.: Transparent surface modeling from a pair of polarization images. IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 73–82 (2004)CrossRefGoogle Scholar
  27. 27.
    Miyazaki, D., Shigetomi, T., Baba, M., Furukawa, R., Hiura, S., Asada, N.: Surface normal estimation of black specular objects from multiview polarization images. Opt. Eng. 56(4), 041303 (2016)CrossRefGoogle Scholar
  28. 28.
    Miyazaki, D., Tan, R.T., Hara, K., Ikeuchi, K.: Polarization-based inverse rendering from a single view. In: null, p. 982. IEEE (2003)Google Scholar
  29. 29.
    Morel, O., Stolz, C., Meriaudeau, F., Gorria, P.: Active lighting applied to three-dimensional reconstruction of specular metallic surfaces by polarization imaging. Appl. Opt. 45(17), 4062–4068 (2006)CrossRefGoogle Scholar
  30. 30.
    Murez, Z., Treibitz, T., Ramamoorthi, R., Kriegman, D.: Photometric stereo in a scattering medium. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3415–3423 (2015)Google Scholar
  31. 31.
    Nayar, S.K., Fang, X.S., Boult, T.: Separation of reflection components using color and polarization. Int. J. Comput. Vis. 21(3), 163–186 (1997)CrossRefGoogle Scholar
  32. 32.
    Ngo Thanh, T., Nagahara, H., Taniguchi, R.i.: Shape and light directions from shading and polarization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2310–2318 (2015)Google Scholar
  33. 33.
    Ponce, J., Hebert, M.: Trinocular geometry revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 17–24 (2014)Google Scholar
  34. 34.
    Quan, L.: Invariants of six points and projective reconstruction from three uncalibrated images. IEEE Trans. Pattern Anal. Mach. Intell. 17(1), 34–46 (1995)CrossRefGoogle Scholar
  35. 35.
    Rahmann, S., Canterakis, N.: Reconstruction of specular surfaces using polarization imaging. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol. 1, pp. I-I. IEEE (2001)Google Scholar
  36. 36.
    Shashua, A.: Algebraic functions for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 17(8), 779–789 (1995)CrossRefGoogle Scholar
  37. 37.
    Smith, W.A., Ramamoorthi, R., Tozza, S.: Linear depth estimation from an uncalibrated, monocular polarisation image. In: European Conference on Computer Vision. pp. 109–125. Springer, Berlin (2016)CrossRefGoogle Scholar
  38. 38.
    Taamazyan, V., Kadambi, A., Raskar, R.: Shape from mixed polarization. arXiv preprint arXiv:1605.02066 (2016)
  39. 39.
    Torr, P.H., Zisserman, A.: Robust parameterization and computation of the trifocal tensor. Image Vis. Comput. 15(8), 591–605 (1997)CrossRefGoogle Scholar
  40. 40.
    Tozza, S., Smith, W.A., Zhu, D., Ramamoorthi, R., Hancock, E.R.: Linear differential constraints for photo-polarimetric height estimation. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society Press (2017)Google Scholar
  41. 41.
    Walter, B., Marschner, S.R., Li, H., Torrance, K.E.: Microfacet models for refraction through rough surfaces. In: Proceedings of the 18th Eurographics Conference on Rendering Techniques, pp. 195–206. Eurographics Association (2007)Google Scholar
  42. 42.
    Weng, J., Huang, T.S., Ahuja, N.: Motion and structure from line correspondences; closed-form solution, uniqueness, and optimization. IEEE Trans. Pattern Anal. Mach. Intell. 3, 318–336 (1992)CrossRefGoogle Scholar
  43. 43.
    Wolff, L.B., Boult, T.E.: Constraining object features using a polarization reflectance model. IEEE Trans. Pattern Anal. Mach. Intell. 13(7), 635–657 (1991)CrossRefGoogle Scholar
  44. 44.
    Yang, L., Tan, F., Li, A., Cui, Z., Furukawa, Y., Tan, P.: Polarimetric dense monocular slam, pp. 3857–3866 (2018)Google Scholar
  45. 45.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Lixiong Chen
    • 1
  • Yinqiang Zheng
    • 1
    Email author
  • Art Subpa-asa
    • 2
  • Imari Sato
    • 1
  1. 1.National Institute of InformaticsTokyoJapan
  2. 2.Tokyo Institute of TechnologyTokyoJapan

Personalised recommendations