Stereo Image Analysis of Non-Lambertian Surfaces

Article

Abstract

Stereo image analysis is based on establishing correspondences between a pair of images by determining similarity measures for potentially corresponding image parts. Such similarity criteria are only strictly valid for surfaces with Lambertian (diffuse) reflectance characteristics. Specular reflections are viewpoint dependent and may thus cause large intensity differences at corresponding image points. In the presence of specular reflections, traditional stereo approaches are often unable to establish correspondences at all, or the inferred disparity values tend to be inaccurate, or the established correspondences do not belong to the same physical surface point. The stereo image analysis framework for non-Lambertian surfaces presented in this contribution combines geometric cues with photometric and polarimetric information into an iterative scheme that allows to establish stereo correspondences in accordance with the specular reflectance behaviour and at the same time to determine the surface gradient field based on the known photometric and polarimetric reflectance properties. The described approach yields a dense 3D reconstruction of the surface which is consistent with all observed geometric and photopolarimetric data. Initially, a sparse 3D point cloud of the surface is computed by traditional blockmatching stereo. Subsequently, a dense 3D profile of the surface is determined in the coordinate system of camera 1 based on the shape from photopolarimetric reflectance and depth technique. A synthetic image of the surface is rendered in the coordinate system of camera 2 using the illumination direction and reflectance properties of the surface material. Point correspondences between the rendered image and the observed image of camera 2 are established with the blockmatching technique. This procedure yields an increased number of 3D points of higher accuracy, compared to the initial 3D point cloud. The improved 3D point cloud is used to compute a refined dense 3D surface profile. These steps are iterated until convergence of the 3D reconstruction. An experimental evaluation of our method is provided for areas of several square centimetres of forged and cast iron objects with rough surfaces displaying both diffuse and significant specular reflectance components, where traditional stereo image analysis largely fails. A comparison to independently measured ground truth data reveals that the root-mean-square error of the 3D reconstruction results is typically of the order 30–100 μm at a lateral pixel resolution of 86 μm. For two example surfaces, the number of stereo correspondences established by the specular stereo algorithm is several orders of magnitude higher than the initial number of 3D points. For one example surface, the number of stereo correspondences decreases by a factor of about two, but the 3D point cloud obtained with the specular stereo method is less noisy, contains a negligible number of outliers, and shows significantly more surface detail than the initial 3D point cloud. For poorly known reflectance parameters we observe a graceful degradation of the accuracy of 3D reconstruction.

Keywords

Stereo vision Specular reflectance Shape from shading Shape from polarisation Photometric stereo 

References

  1. Atkinson, G. A., & Hancock, E. R. (2005). Multi-view surface reconstruction using polarization. In Proc. int. conf. on computer vision (pp. 309–316). Beijing, China. Google Scholar
  2. Batlle, J., Mouaddib, E., & Salvi, J. (1998). Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. Pattern Recognition, 31(7), 963–982. CrossRefGoogle Scholar
  3. Bhat, D. N., & Nayar, S. K. (1998). Stereo and specular reflection. International Journal of Computer Vision, 26(2), 91–106. CrossRefGoogle Scholar
  4. Bouguet, J.-Y. (2007). Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/.
  5. Bruhn, A., & Weickert, J. (2005). Towards ultimate motion estimation: combining highest accuracy with real-time performance. In Proc. IEEE int. conf. on computer vision (Vol. 1, pp. 749–755). Beijing, China. Google Scholar
  6. Cox, I., Hingorani, S., & Rao, S. (1996). A maximum likelihood stereo algorithm. Computer Vision and Image Understanding, 63(3), 542–567. CrossRefGoogle Scholar
  7. Cryer, J. E., Tsai, P.-S., & Shah, M. (1995). Integration of shape from shading and stereo. Pattern Recognition, 28(7), 1033–1043. CrossRefGoogle Scholar
  8. D’Angelo, P., & Wöhler, C. (2005a). 3D reconstruction of metallic surfaces by photopolarimetric analysis. In H. Kalviainen et al. (Eds.), LNCS : Vol. 3540. Proc. 14th Scand. conf. on image analysis (pp. 689–698). Berlin: Springer. Google Scholar
  9. D’Angelo, P., & Wöhler, C. (2005b). 3D surface reconstruction based on combined analysis of reflectance and polarisation properties: a local approach. In ISPRS workshop towards benchmarking automated calibration, orientation and surface reconstruction from images. Beijing, China. Google Scholar
  10. D’Angelo, P., & Wöhler, C. (2006). Image-based 3D surface reconstruction by combination of sparse depth data with shape from shading and polarisation. In ISPRS conf. on photogrammetric computer vision. Bonn, Germany. Google Scholar
  11. D’Angelo, P., & Wöhler, C. (2008). Image-based 3D surface reconstruction by combination of photometric, geometric, and real-aperture methods. ISPRS Journal of Photogrammetry and Remote Sensing, 63(3), 297–321. CrossRefGoogle Scholar
  12. Davis, J., Nehab, D., Ramamoorthi, R., & Rusinkiewicz, S. (2005). Spacetime stereo: a unifying framework for depth from triangulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2), 296–302. CrossRefGoogle Scholar
  13. Fassold, H., Danzl, R., Schindler, K., & Bischof, H. (2004). Reconstruction of archaeological finds using shape from stereo and shape from shading. In Proc. 9th computer vision winter workshop (pp. 21–30). Piran, Slovenia. Google Scholar
  14. Faugeras, O. (1993). Three-dimensional computer vision: a geometric viewpoint. Cambridge: MIT Press. Google Scholar
  15. Franke, U., & Joos, A. (2000). Real-time stereo vision for urban traffic scene understanding. In Proc. IEEE conf. on intelligent vehicles. Detroit, USA. Google Scholar
  16. Hirschmüller, H. (2001). Improvements in real-time correlation-based stereo vision. In Proc. int. conf. on computer vision and pattern recognition, stereo workshop. Hawaii, USA. Google Scholar
  17. Hirschmüller, H. (2006). Stereo vision in structured environments by consistent semi-global matching. In Proc. IEEE conf. on computer vision and pattern recognition (Vol. 2, pp. 2386–2393). Google Scholar
  18. Horn, B. K. P. (1986). Robot vision. Cambridge: MIT Press. Google Scholar
  19. Horn, B. K. P. (1989). Height and gradient from shading (MIT technical report 1105A). http://people.csail.mit.edu/people/bkph/AIM/AIM-1105A-TEX.pdf.
  20. Jin, H., Soatto, S., & Yezzi, A. (2003). Multi-view stereo beyond Lambert. In Proc. IEEE conf. on computer vision and pattern recognition (pp. 171–178). Google Scholar
  21. Klette, R., Kozera, R., & Schlüns, K. (1999). Shape from shading and photometric stereo methods. In: B. Jähne, H. Haussecker, & P. Geissler (Eds.), Handbook of computer vision and applications : Vol. 2. Signal processing and pattern recognition (pp. 532–590). San Diego: Academic Press. Google Scholar
  22. Krüger, L., Wöhler, C., Würz-Wessel, A., & Stein, F. (2004). In-factory calibration of multiocular camera systems. In Proc. SPIE photonics Europe (Optical metrology in production engineering) (pp. 126–137). Strasbourg. Google Scholar
  23. Lim, J., Ho., J., Yang, M.-H., & Kriegman, D. (2005). Passive photometric stereo from motion. In Proc. int. conf. on computer vision (pp. 1635–1642). Beijing, China. Google Scholar
  24. Lohse, V., Heipke, C., & Kirk, R. L. (2006). Derivation of planetary topography using multi-image shape-from-shading. Planetary and Space Science, 54, 661–674. CrossRefGoogle Scholar
  25. Lowitzsch, S., Kaminski, J., Knauer, M. C., & Häusler, G. (2005). Vision and modeling of specular surfaces. In Proc. 10th int. fall workshop on vision, modeling, and visualization. Erlangen, Germany. Google Scholar
  26. Miyazaki, D., Tan, R. T., Hara, K., & Ikeuchi, K. (2003). Polarization-based inverse rendering from a single view. In Proc. IEEE int. conf. on computer vision (Vol. 2, pp. 982–987). Nice. Google Scholar
  27. Nayar, S. K., Ikeuchi, K., & Kanade, T. (1991). Surface reflection: physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 611–634. CrossRefGoogle Scholar
  28. Nayar, S. K., Fang, X.-S., & Boult, T. (1993). Removal of specularities using color and polarization. In Proc. int. conf. on computer vision and pattern recognition (pp. 583–590). New York. Google Scholar
  29. Rahmann, S. (1999). Inferring 3D scene structure from a single polarization image. In SPIE: Vol. 3826. Polarization and color techniques in industrial inspection (pp. 22–33). Google Scholar
  30. Roy, S., & Cox, L. (1998). A maximum-flow formulation of the N-camera stereo correspondence problem. In Proc. int. conf. on computer vision (pp. 492–499). Bombay. Google Scholar
  31. Samaras, D., Metaxas, D., Fua, P., & Leclerc, Y. G. (2000). Variable albedo surface reconstruction from stereo and shape from shading. In Proc. int. conf. on computer vision and pattern recognition (Vol. I, pp. 480–487). Google Scholar
  32. Scharstein, D., & Szeliski, R. (2001). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1–3), 7–42. Google Scholar
  33. Shi, J., & Tomasi, C. (1994). Good features to track. In Proc. IEEE conf. on computer vision and pattern recognition. Seattle, USA. Google Scholar
  34. Simchony, T., Chellappa, R., & Shao, M. (1990). Direct analytic methods for solving Poisson equations in computer vision problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(5), 435–556. CrossRefGoogle Scholar
  35. Wang, L., Yang, R., & Davis, J. E. (2007). BRDF invariant stereo using light transport constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(9), 1616–1626. CrossRefGoogle Scholar
  36. Wolff, L. B., & Angelopoulou, E. (1994). Three-dimensional stereo by photometric ratios. Journal of the Optical Society of America, 11(11), 3069–3078. CrossRefGoogle Scholar
  37. Zhang, Z. (1999). Flexible camera calibration by viewing a plane from unknown orientations. In Proc. int. conf. on computer vision (pp. 666–673). Google Scholar
  38. Zickler, T., Belhumeur, P. N., & Kriegman, D. J. (2002). Helmholtz stereopsis: exploiting reciprocity for surface reconstruction. In Proc. Eur. conf. on computer vision (pp. 869–884). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Daimler Group Research, Environment PerceptionUlmGermany
  2. 2.German Aerospace Center (DLR) OberpfaffenhofenWeßlingGermany

Personalised recommendations