Abstract
Real-time active 3D range cameras based on time-of-flight technology using the Photonic Mixer Device (PMD) can be considered as a complementary technique for stereo-vision based depth estimation. Since those systems directly yield 3D measurements, they can also be used for initializing vision based approaches, especially in highly dynamic environments. Fusion of PMD depth images with passive intensity-based stereo is a promising approach for obtaining reliable surface reconstructions even in weakly textured surface regions.
In this work a PMD-stereo fusion algorithm for the estimation of patchlets from a combined PMD-stereo camera rig will be presented. As patchlet we define an oriented small planar 3d patch with associated surface normal. Least-squares estimation schemes for estimating patchlets from PMD range images as well as from a pair of stereo images are derived. It is shown, how those two approaches can be fused into one single estimation, that yields results even if either of the two single approaches fails.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Scharstein, D., Szeliski, R., Zabih, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms (2001)
Hoppe, H., DeRose, T., Duchamp, T., Halstead, M., Jin, H., McDonald, J., Schweitzer, J., Stuetzle, W.: Piecewise smooth surface reconstruction. Computer Graphics 28(Annual Conference Series), 295–302 (1994)
Lindner, M., Kolb, A.: Lateral and depth calibration of pmd-distance sensors. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Remagnino, P., Nefian, A., Meenakshisundaram, G., Pascucci, V., Zara, J., Molineros, J., Theisel, H., Malzbender, T. (eds.) ISVC 2006. LNCS, vol. 4292, pp. 524–533. Springer, Heidelberg (2006)
Streckel, B., Bartczak, B., Koch, R., Kolb, A.: Supporting structure from motion with a 3d-range-camera. In: Ersbøll, B.K., Pedersen, K.S.(eds.) SCIA 2007. LNCS, vol. 4522. Springer, Heidelberg (2007)
Grest, D., Koch, R.: Single view motion tracking by depth and silhouette information. In: Scandinavian Conference on Image Analysis (SCIA 2007) (2007)
Gokturk, S., Tomasi, C.: 3d head tracking based on recognition and interpolation using a time-of-flight depth sensor. In: Proc. CVPR, pp. 211–217 (2004)
Kuhnert, K., Stommel, M.: Fusion of stereo-camera and pmd-camera data for real-time suited precise 3d environment reconstruction. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (October 2006)
Blank for review
Murray, D.R.: Patchlets: a method of interpreting correlation stereo 3D data. PhD thesis, The Univeristy of British Columbia, Vancouver, Canada (2004)
Murray, D., Little, J.J.: Segmenting correlation stereo range images using surface elements. In: 3DPVT 2004: Proceedings of the 3D Data Processing, Visualization, and Transmission, 2nd International Symposium on (3DPVT 2004), Washington, DC, USA, pp. 656–663. IEEE Computer Society Press, Los Alamitos (2004)
Szeliski, R., Tonnesen, D.: Surface modeling with oriented particle systems. Computer Graphics 26(2), 185–194 (1992)
Molton, N.D., Davison, A.J., Reid, I.D.: Locally planar patch features for real-time structure from motion. In: Proc. British Machine Vision Conference, BMVC (September 2004)
Pietzsch, T., Grossmann, A.: A method of estimating oriented surface elements from stereo images. In: Proc. British Machine Vision Conference (2005)
Pritchett, P., Zisserman, A.: Wide baseline stereo matching. In: Proceedings of the International Conference on Computer Vision, pp. 754–760 (1998)
Baker, S., Matthews, I.: Lucas-kanade 20 years on: A unifying framework: Part (2002)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI 1981, pp. 674–679 (1981)
Triggs, B.: Detecting keypoints with stable position, orientation, and scale under illumination changes. In: Proceedings of European Conference on Computer Vision, pp. 100–113 (2004)
Xu, Z., Schwarte, R., Heinol, H., Buxbaum, B., Ringbeck, T.: Smart pixels -photonic mixer device (pmd). In: Mechatronics and Machine Vision in Practice, pp. 259–264 (1998)
Lange, R., Seitz, P., Biber, A., Schwarte, R.: Time-of-flight range imaging with a custom solid state image sensor. In: Tiziani, H.J., Rastogi, P.K. (eds.) Proc. SPIE. September 1999. Laser Metrology and Inspection, vol. 3823, pp. 180–191 (1999)
Zhang, Z.: Untersuchung und Charakterisierung von Photomischdetektor-Strukturen und ihren Grundschaltungen. PhD thesis, Department of Electrical Engineering And Computer Science at Univeristy of Siegen (December 2003)
Kahlmann, T., Remondino, F., Ingensand, H.: Calibration for increased accuracy of the range imaging camera swissrangertm. In: IEVM 2006Â (2006)
Förstner, W., Wrobel, B.: Mathematical concepts in photogrammetry. In: McGlone, J.C., Mikhail, E.M., Bethel, J. (eds.) Manual of Photogrammetry. ASPRS, pp. 15–180 (2004)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Beder, C., Bartczak, B., Koch, R. (2007). A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds) Pattern Recognition. DAGM 2007. Lecture Notes in Computer Science, vol 4713. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74936-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-74936-3_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74933-2
Online ISBN: 978-3-540-74936-3
eBook Packages: Computer ScienceComputer Science (R0)