Pose Refinement of Transparent Rigid Objects with a Stereo Camera

  • Ilya Lysenkov
  • Victor Eruhimov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7870)


We propose a new method for refining 6-DOF pose of rigid transparent objects. The algorithm is based on minimizing the distance between edges in a test image and a set of edges produced by the training model with a specific pose. The model is scanned with a monocular camera and a 3D sensor such as a Kinect device. The pose is estimated from a monocular image or a stereo pair. The method does not require a CAD model of the object. We demonstrate experimental results on a set of kitchen items essential for any home and office environment.


pose estimation localization transparent objects 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Collet, A., Berenson, D., Srinivasa, S., Ferguson, D.: Object recognition and full pose registration from a single image for robotic manipulation. In: IEEE International Conference on Robotics and Automation (2009)Google Scholar
  2. 2.
    Rosenhahn, B., Brox, T., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose tracking. International Journal of Computer Vision 73(3), 243–262 (2007)CrossRefGoogle Scholar
  3. 3.
    Ihrke, I., Kutulakos, K.N., Lensch, H.P.A., Magnor, M., Heidrich, W.: State of the Art in Transparent and Specular Object Reconstruction. In: STAR Proceedings of Eurographics, pp. 87–108 (2008)Google Scholar
  4. 4.
    Klank, U., Carton, D., Beetz, M.: Transparent Object Detection and Reconstruction on a Mobile Platform. In: IEEE International Conference on Robotics and Automation (2011)Google Scholar
  5. 5.
    Phillips, C., Derpanis, K., Daniilidis, K.: A Novel Stereoscopic Cue for Figure-Ground Segregation of Semi-Transparent Objects. In: 1st IEEE Workshop on Challenges and Opportunities in Robot Perception (2011)Google Scholar
  6. 6.
    Lysenkov, I., Eruhimov, V., Bradski, G.: Recognition and pose estimation of rigid transparent objects with a kinect sensor. In: Robotics: Science and Systems Conference (2012)Google Scholar
  7. 7.
    Lagger, P., Salzmann, M., Lepetit, V., Fua, P.: 3d pose refinement from reflections. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE (2008)Google Scholar
  8. 8.
    Chang, J., Raskar, R., Agrawal, A.: 3d pose estimation and segmentation using specular cues. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE (2009)Google Scholar
  9. 9.
    Netz, A., Osadchy, M.: Using specular highlights as pose invariant features for 2d-3d pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  10. 10.
    Shotton, J., Blake, A., Cipolla, R.: Multiscale categorical object recognition using contour fragments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1270–1281 (2007)Google Scholar
  11. 11.
    Biederman, I., Ju, G.: Surface versus edge-based determinants of visual recognition. Cognitive Psychology 20(1), 38–64 (1988)CrossRefGoogle Scholar
  12. 12.
    Rosenhahn, B.: Pose estimation revisited. PhD thesis, Universität Kiel (September 2003)Google Scholar
  13. 13.
    Hinterstoisser, S., Lepetit, V., Ilic, S., Fua, P., Navab, N.: Dominant orientation templates for real-time detection of texture-less objects. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2257–2264 (2010)Google Scholar
  14. 14.
    Gavrila, D.: A bayesian, exemplar-based approach to hierarchical shape matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1408–1421 (2007)Google Scholar
  15. 15.
    Reinbacher, C., Ruther, M., Bischof, H.: Pose estimation of known objects by efficient silhouette matching. In: IEEE International Conference on Pattern Recognition, pp. 1080–1083 (2010)Google Scholar
  16. 16.
    Lowe, D.: Three-dimensional object recognition from single two-dimensional images. Artificial Intelligence 31(3), 355–395 (1987)CrossRefGoogle Scholar
  17. 17.
    Liu, M., Tuzel, O., Veeraraghavan, A., Chellappa, R., Agrawal, A., Okuda, H.: Pose estimation in heavy clutter using a multi-flash camera. In: IEEE International Conference on Robotics and Automation (2010)Google Scholar
  18. 18.
    Ulrich, M., Wiedemann, C., Steger, C.: CAD-based recognition of 3d objects in monocular images. In: International Conference on Robotics and Automation, vol. 1191 (2009)Google Scholar
  19. 19.
    Canny, J.: A computational approach to edge detection. Readings in Computer Vision: Issues, Problems, Principles, and Paradigms 184, 87–116 (1987)Google Scholar
  20. 20.
    Besl, P., McKay, N.: A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence (1992)Google Scholar
  21. 21.
    Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. International Journal of Computer Vision 13(2), 119–152 (1994)CrossRefGoogle Scholar
  22. 22.
    Pulli, K.: Multiview registration for large data sets. In: IEEE Second International Conference on 3-D Digital Imaging and Modeling, pp. 160–168 (1999)Google Scholar
  23. 23.
    Bergevin, R., Soucy, M., Gagnon, H., Laurendeau, D.: Towards a general multi-view registration technique. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(5), 540–547 (1996)CrossRefGoogle Scholar
  24. 24.
    Fitzgibbon, A.: Robust registration of 2D and 3D point sets. Image and Vision Computing (2003)Google Scholar
  25. 25.
    Hazan, E., Safra, S., Schwartz, O.: On the hardness of approximating k-dimensional matching. In: Electronic Colloquium on Computational Complexity, TR03-020 (2003)Google Scholar
  26. 26.
    Singh, R., Xu, J., Berger, B.: Global alignment of multiple protein interaction networks. In: Proc. Pacific Symp. Biocomputing, vol. 13, pp. 303–314. Citeseer (2008)Google Scholar
  27. 27.
    Hubert, M., Rousseeuw, P., Van Aelst, S.: High-breakdown robust multivariate methods. Statistical Science 23(1), 92–119 (2008)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Rousseeuw, P.: Multivariate estimation with high breakdown point. Mathematical Statistics and Applications 8, 283–297 (1985)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Douglas, D., Peucker, T.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization 10(2), 112–122 (1973)CrossRefGoogle Scholar
  30. 30.
    Matas, J., Shao, Z., Kittler, J.: Estimation of curvature and tangent direction by median filtered differencing. In: Braccini, C., Vernazza, G., DeFloriani, L. (eds.) ICIAP 1995. LNCS, vol. 974, pp. 83–88. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  31. 31.
    Huttenlocher, D., Klanderman, G., Rucklidge, W.: Comparing images using the hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence, 850–863 (1993)Google Scholar
  32. 32.
    Jones, D., Perttunen, C., Stuckman, B.: Lipschitzian optimization without the Lipschitz constant. Journal of Optimization Theory and Applications 79(1), 157–181 (1993)MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Johnson, S.G.: The nlopt nonlinear-optimization package, http://ab-initio.mit.edu/nlopt
  34. 34.
    De Las Rivas, J., Fontanillo, C.: Protein–protein interactions essentials: key concepts to building and analyzing interactome networks. PLoS Computational Biology 6(6), e1000807 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ilya Lysenkov
    • 1
  • Victor Eruhimov
    • 1
  1. 1.ItseezNizhny NovgorodRussia

Personalised recommendations