International Journal of Computer Vision

, Volume 34, Issue 2–3, pp 123–145 | Cite as

3-D to 2-D Pose Determination with Regions

  • David Jacobs
  • Ronen Basri


This paper presents a novel approach to parts-based object recognition in the presence of occlusion. We focus on the problem of determining the pose of a 3-D object from a single 2-D image when convex parts of the object have been matched to corresponding regions in the image. We consider three types of occlusions: self-occlusion, occlusions whose locus is identified in the image, and completely arbitrary occlusions. We show that in the first two cases this is a convex optimization problem, derive efficient algorithms, and characterize their performance. For the last case, we prove that the problem of finding valid poses is computationally hard, but provide an efficient, approximate algorithm. This work generalizes our previous work on region-based object recognition, which focused on the case of planar models.

object recognition pose determination linear programming line traversal occlusion regions parts convexity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alter, T.D. and Jacobs, D. 1998. Uncertainty propagation in model-based recognition. International Journal of Computer Vision, 27(2):127–159.Google Scholar
  2. Amenta, N. 1992. Finding a line traversal of axial objects in three dimensions. In Proc. of the Third ACM-SIAM Symp. on Discrete Alg., pp. 66–71.Google Scholar
  3. Amenta, N. Personal communication.Google Scholar
  4. Amenta, N. 1994. Bounded boxes, Hausdorff distance, and a new proof of an interesting Helly-type theorem. In Proceedings of the 10th Annual ACM Symposium on Computational Geometry, pp. 340–347.Google Scholar
  5. Basri, R. 1996. Paraperspective ≡ affine. International Journal of Computer Vision, 19(2):169–179.Google Scholar
  6. Basri, R. and Jacobs, D.W. 1995. Recognition using region correspondences. Technical Report CS95-33, The Weizmann Institute of Science.Google Scholar
  7. Basri, R. and Jacobs, D. 1996. Matching convex polygons and polyhedra, allowing for occlusion. First ACM Workshop on Applied Computational Geometry, pp. 57–66.Google Scholar
  8. Basri, R. and Jacobs, D. 1997. Recognition using region correspondences. International Journal of Computer Vision, 25(2):145–166.Google Scholar
  9. Bergevin, R. and Levine, M. 1993. Generic object recognition: Building and matching coarse descriptions from line drawings. IEEE Trans. on Pattern Analysis and Machine Intelligence, 15(1):19–36.Google Scholar
  10. Biederman, I. 1985. Human image understanding: Recent research and a theory. Computer Graphics, Vision, and Image Processing, (32):29–73.Google Scholar
  11. Binford, T. 1971. Visual perception by computer. In IEEE Conf. on Systems and Control.Google Scholar
  12. Brooks, R. 1981. Symbolic reasoning among 3-D models and 2-D images. Artificial Intelligence, 17:285–348.Google Scholar
  13. Canny, J. 1986. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6):679–698.Google Scholar
  14. Clemens, D. 1991. Region-based feature interpretation for recognizing 3D models in 2D images. MIT AI TR-1307.Google Scholar
  15. Conway, J.B. 1990. A Course in Functional Analysis. Springer-Verlag.Google Scholar
  16. Duda, R.O. and Hart, P.E. 1973. Pattern Classification and Scene Analysis. Wiley-Interscience Publication, John Wiley and Sons, Inc.Google Scholar
  17. Dudani, S.A., Breeding, K.J., and McGhee, R.B. 1977. Aircraft identification by moments invariants. IEEE Transactions on Computations, C-26(1):39–46.Google Scholar
  18. Fischler, M.A. and Bolles, R.C. 1981. Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography. Com. of the A.C.M., 24(6):381–395.Google Scholar
  19. Forsyth, D. 1996. Recognizing algebraic surfaces from their outlines. International Journal of Computer Vision, 18(1):21–40.Google Scholar
  20. Forsyth, D., Mundy, J., Zisserman, A., and Rothwell, C. 1992. Recognising rotationally symmetric surfaces from their outlines. European Conf. on Comp. Vis., pp. 639–647.Google Scholar
  21. Gross, A. and Boult, T. 1990. Recovery of generalized cylinders from a single intensity image. In Proc. of the DARPA IU Workshop, pp. 557–564.Google Scholar
  22. Horaud, R. 1987. New methods for matching 3-D objects with single perspective views. IEEE Trans. Pattern Anal. Machine Intell., 9(3):401–412.Google Scholar
  23. Hu, M.K. 1962. Visual pattern recognition by moment invariants. IRE Transactions on Information Theory, IT-8:169–187.Google Scholar
  24. Huttenlocher, D., Klanderman, G., and Rucklidge, W. 1993a. Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850–863.Google Scholar
  25. Huttenlocher, D., Noh, J., and Rucklidge, W. 1993b. Tracking non-rigid objects in complex scenes. In 4th Int. Conf. on Computer Vision, pp. 93–101.Google Scholar
  26. Huttenlocher, D.P. and Ullman, S. 1990. Recognizing solid objects by alignment with an image. Int. J. Computer Vision, 5(2):195–212.Google Scholar
  27. Jacobs, D. 1992. Recognizing 3-D objects using 2-D images. MIT AI Memo 1416.Google Scholar
  28. Jacobs, D. 1996. Robust and efficient detection of salient convex groups. IEEE Trans. Pattern Anal. Machine Intell., 18(1):23–37.Google Scholar
  29. Jacobs, D. 1997. Matching 3-D models to 2-D images. Int. J. Computer Vision, 21(1/2):123–153.Google Scholar
  30. Koenderink, J. and van Doorn, A. 1991. Affine structure from motion. Journal of the Optical Society of America, 8(2):377–385.Google Scholar
  31. Kriegman, D. and Ponce, J. 1990. On recognizing and positioning curved 3-D objects from image contours. IEEE Trans. Pattern Anal. Machine Intell., 12(12):1127–1137.Google Scholar
  32. Lamdan, Y. and Wolfson, H.J. 1988. Geometric hashing: A general and efficient model-based recognition scheme. In Second International Conference Computer Vision, pp. 238–249.Google Scholar
  33. Lowe, D. 1985. Perceptual Organization and Visual Recognition. Kluwer Academic Publishers: The Netherlands.Google Scholar
  34. Marr, D. and Nishihara, H. 1978. Representation and recognition of the spatial organization of three dimensional structure. In Proceedings of the Royal Society of London B, 200:269–294.Google Scholar
  35. Megiddo, N. 1996. Finding a line of sight thru boxes in d-space in linear time. Research Report RJ 10018, IBM Almaden Research Center, San Jose, California.Google Scholar
  36. Nagao, K. and Grimson, W. 1994. Object recognition by alignment using invariant projections of planar surfaces. In 12th Int. Conf. on Pattern Rec., pp. 861–864.Google Scholar
  37. Nayar, S. and Bolle, R. 1996. Reflectance based object recognition. Int. J. of Comp. Vis., 17(3):219–240.Google Scholar
  38. Pellegrini, M. 1990. Stabbing and ray shooting in three dimensional space. In Proc. of the 6th Annual Symp. on Comp. Geometry, pp. 177–186.Google Scholar
  39. Pentland, A. 1987. Recognition by parts. In Proceedings of the First International Conference on Computer Vision, pp. 612–620.Google Scholar
  40. Persoon, E. and Fu, K.S. 1977. Shape discrimination using Fourier descriptors. IEEE Transactions on Systems, Man and Cybernetics, 7:534–541.Google Scholar
  41. Poelman, C.J. and Kanade, T. 1997. A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3):206–218.Google Scholar
  42. Ponce, J., Chelberg, D., and Mann, W. 1989. Invariant properties of straight homogeneous generalized cylinders and their contours. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(9):951–966.Google Scholar
  43. Reeves, A.P., Prokop, R.J., Andrews, S.E., and Kuhl, F.P. 1984. Three-dimensional shape analysis using moments and fourier descriptors. In Proc. of Int. Conf. on Pattern Recognition, pp. 447–450.Google Scholar
  44. Richard, C.W. and Hemami, H. 1974. Identification of three dimensional objects using fourier descriptors of the boundry curve. IEEE Transactions on Systems, Man and Cybernetics, 4(4):371–378.Google Scholar
  45. Rivlin, E., Dickinson, S.J., and Rosenfeld, A. 1995. Recognition by functional parts. Computer Vision: Image Understanding, 62(2):164–176.Google Scholar
  46. Rock, I. 1983. The Logic of Perception. MIT Press: Cambridge, MA.Google Scholar
  47. Rothwell, C., Forsyth, D., Zisserman, A., and Mundy, J. 1993. Extracting projective structure from single perspective views of 3D point sets. In Third Int. Conf. on Comp. Vis., pp. 573–582.Google Scholar
  48. Rucklidge, W. 1997. Efficiently locating objects using the Hausdorff distance. Int. J. of Comp. Vis., 24(3):251–270.Google Scholar
  49. Sadjadi, F.A. and Hall, E.L. 1980. Three-dimensional moment invariants. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2(2):127–136.Google Scholar
  50. Seidel, R. 1990. Linear programming and convex hulls made easy. In Proc. of the Sixth Annual Symp. on Comp. Geometry, pp. 211–215.Google Scholar
  51. Shafer, S. and Kanade, T. 1983. The theory of stright homogeneous generalized cylinders. Tech. Report CS-083-105, Carnegie Mellon Univ.Google Scholar
  52. Solina, F. and Bajcsy, R. 1990. Recovery of parametric models from range images: The case for superquadrics with global deformations. IEEE Trans. on PAMI, 12(2):131–146.Google Scholar
  53. Sugimoto, A. 1996. Object recognition by combining paraperspective images. Int. J. of Comp. Vis., 19(2):181–201.Google Scholar
  54. Terzopoulos, D. and Metaxas, D. 1991. Dynamic 3D models with local and global deformations: Deformable superquadrics. IEEE Trans. on PAMI, 13(7):703–714.Google Scholar
  55. Thompson, D. and Mundy, J. 1987. Three-dimensional model matching from an unconstrained viewpoint. In Proceedings IEEE Conference Rob. Aut., pp. 208–220.Google Scholar
  56. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137–154.Google Scholar
  57. Ulipinar, F. and Nevatia, R. 1995. Shape from contour: Straight generalized cylinders and constant cross section generalized cylinders. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(2):120–135.Google Scholar
  58. Ullman, S. and Basri, R. 1991. Recognition by linear combinations of models. IEEE Trans. on PAMI, 13(10):992–1006.Google Scholar
  59. Vijayakumar, B., Kriegman, D., and Ponce, J. 1995. Invariant-based recognition of complex curved 3D objects from image contours. In Fifth Int. Conf. on Comp. Vis., pp. 508–514.Google Scholar
  60. Zerroug, M. and Nevatia, R. 1994. Using invariance and quasi-invariance for the segmentation and recovery of curved objects. In Applications of Invariance in Computer Vision, J. Mundy and Z. Zisserman (Eds.), Springer-Verlag: Berlin, Heidelberg.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • David Jacobs
  • Ronen Basri

There are no affiliations available

Personalised recommendations