Advertisement

International Journal of Computer Vision

, Volume 60, Issue 3, pp 241–265 | Cite as

On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image

  • Wei Hong
  • Allen Yang Yang
  • Kun Huang
  • Yi Ma
Article

Abstract

In this paper, we provide a principled explanation of how knowledge in global 3-D structural invariants, typically captured by a group action on a symmetric structure, can dramatically facilitate the task of reconstructing a 3-D scene from one or more images. More importantly, since every symmetric structure admits a “canonical” coordinate frame with respect to which the group action can be naturally represented, the canonical pose between the viewer and this canonical frame can be recovered too, which explains why symmetric objects (e.g., buildings) provide us overwhelming clues to their orientation and position. We give the necessary and sufficient conditions in terms of the symmetry (group) admitted by a structure under which this pose can be uniquely determined. We also characterize, when such conditions are not satisfied, to what extent this pose can be recovered. We show how algorithms from conventional multiple-view geometry, after properly modified and extended, can be directly applied to perform such recovery, from all “hidden images” of one image of the symmetric structure. We also apply our results to a wide range of applications in computer vision and image processing such as camera self-calibration, image segmentation and global orientation, large baseline feature matching, image rendering and photo editing, as well as visual illusions (caused by symmetry if incorrectly assumed).

structure from symmetry multiple-view geometry symmetry group reflective symmetry rotational symmetry translational symmetry 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Criminisi, I. and Zisserman, A. 1999. Single view metrology. In IEEE International Conference on Computer Vision, pp. 434–441.Google Scholar
  2. A. Criminisi, I. and Zisserman, A. 2000. Single view metrology. Int. Journal on Computer Vision.Google Scholar
  3. Bieberbach, L. 1910. Uber die bewegungsgruppen des ndimensionalen euklidischen raumes mit einem endlichen fundamentalbereich. Nachr, 75–84.Google Scholar
  4. Bulatov (www). Polyhedra collection at http://www.physics.orst.edu/~bulatov/polyhedra/index.html.Google Scholar
  5. Carlsson, S. 1998. Symmetry in perspective. In European Conference on Computer Vision.Google Scholar
  6. Cham, T.J. and Cipolla, R. 1996. Geometric saliency of curve correspondances and grouping of symmetric comntours. In European Conference on Computer Vision, pp. 385-398.Google Scholar
  7. Comanicu, D. and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. In IEEE Transactions on Pattern Analysis & Machine Intelligence, pp. 603–619.Google Scholar
  8. Fedorov, E.S. 1885. The elements of the study of figures. Zapiski Imperatorskogo S. Peterburgskogo Mineralogichesgo Obshchestva [Proc. S. Peterb. Mineral. Soc.], 2(21):1–289.Google Scholar
  9. Fedorov, E.S. 1891a. Symmetry in the plane. Zapiski Imperatorskogo S. Peterburgskogo Mineralogichesgo Obshchestva [Proc. S. Peterb. Mineral. Soc.], 2(28):345–390.Google Scholar
  10. Fedorov, E.S. 1891b. Symmetry of finite figures. Zapiski Imperatorskogo S. Peterburgskogo Mineralogichesgo Obshchestva [Proc. S. Peterb. Mineral. Soc.], 2(28):1–146.Google Scholar
  11. Fedorov, E.S. 1971. Symmetry of Crystals. Translated from the 1949 Russian Edition. American Crystallographic Association.Google Scholar
  12. Fishler, M. and Bolles, R. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395.CrossRefGoogle Scholar
  13. Garding, J. 1992. Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision, 2(4):327–350.Google Scholar
  14. Garding, J. 1993. Shape from texture and contour by weak isotropy. J. of Artificial Intelligence, 64(2):243–297.CrossRefGoogle Scholar
  15. Gibson, J. 1950. The Perception of the Visual World. Houghton Mifflin.Google Scholar
  16. Gool, L.V., Moons, T., and Proesmans, M. 1996. Mirror and point symmetry under perspective skewing. In Int. Conference on Computer Vision & Pattern Recognition, pp. 285–292.Google Scholar
  17. Grünbaum, B. and Shephard, G.C. 1987. Tilings and Patterns. W. H. Freeman and Company.Google Scholar
  18. Hong,W., Ma, Y., and Yu, Y. 2004. Reconstruction of 3-D deformed symmetric curves from perspective images without discrete features. In European Conference on Computer Vision.Google Scholar
  19. Huang, K., Hong, W., and Ma, Y. 2003. Symmetry-based photoediting. In 9th International Conference on Computer Vision, Workshop on Higher Level Knowledge, Nice, France.Google Scholar
  20. Huang, K., Hong, W., Yang, A.Y., and Ma, Y. 2004. Large baseline matching and reconstruction from symmetry cells. In IEEE Int. Conference on Robotics & Automation.Google Scholar
  21. Kanatani, K. 1997. Comments on symmetry as a continuous feature. IEEE Transactions on Pattern Analysis & Machine Intelligence, 19(2):246–247.Google Scholar
  22. Kiryati, N. and Gofman, Y. 1998. Detecting symmetry in grey level images: The global optimization approach. Int. Journal on Computer Vision, 29(1):29–45.CrossRefGoogle Scholar
  23. Leung, T. and Malik, J. 1997. On perpendicular texture or: Why do we see more flowers in the distance? In Int. Conference on Computer Vision & Pattern Recognition, pp. 807–813.Google Scholar
  24. Liebowitz, D. and Zisserman, A. 1998. Detecting rotational symmetries using normalized convolution. In Int. Conference on Computer Vision & Pattern Recognition, pp. 482–488.Google Scholar
  25. Liu, J., Walker, E., and Mundy, J. 1995. Characterizing the stability of 3D invariants derived from 3D translational symmetry. In S.Z. Li, D.P. Mital, E.K. Teoh, and H. Wang (eds.), Recent Developments in Computer Vision, Springer, Berlin, pp. 415–424.Google Scholar
  26. Liu, Y. and Colline, R. 1998. Frieze and wallpaper symmetry groups classification under affine and perspective distortion. Technical Report, CMU-RI-TR-98-37, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
  27. Ma, Y., KoŠecká, J., and Huang, K. 2002. Rank deficiency condition of the multiple-view matrix for mixed point and line features. In Asian Conference on Computer Vision.Google Scholar
  28. Ma, Y., KoŠecká, J., Soatto, S., and Sastry, S. 2003. An Invitation to 3-D Vision: From Images to Geometric Models. Springer-Verlag.Google Scholar
  29. Ma, Y., Vidal, R., KoŠecká, J., and Sastry, S. 2000. Kruppa's equations revisited: Its degeneracy, renormalization, and relations to chirality. In European Conference on Computer Vision.Google Scholar
  30. Malik, J. and Rosenholtz, R. 1997. Computing local surface orientation and shape from texture for curved surfaces. Int. Journal on Computer Vision, 23:149–168.CrossRefGoogle Scholar
  31. Marola, G. 1989. On the detection of the axes of symmetry of symmetric and almost symmetric planar images. IEEE Transactions on Pattern Analysis & Machine Intelligence, 11(1):104–108.Google Scholar
  32. Marr, D. 1982. Vision. W. H. Freeman and Company.Google Scholar
  33. Martin, G.E. 1975. The Foundations of Geometry and the Non-Euclidean Plane. Springer-Verlag.Google Scholar
  34. Mitsumoto, H., Tamura, S., Okazaki, K., and Fukui, Y. 1992. 3-D reconstruction using mirror images based on a plane symmetry recovering method. IEEE Transactions on Pattern Analysis & Machine Intelligence, 14(9):941–946.Google Scholar
  35. Mukherjee, D.P., Zisserman, A., and Brady, J.M. 1995. Shape from symmetry—Detecting and exploiting symmetry in affine images. Phil. Trans. Royal Soc. London A, 351:77–106.Google Scholar
  36. Plamer, S.E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.Google Scholar
  37. Rosenholtz, R. and Malik, J. 1997. Surface orientation from texture: Isotropy or homogeneity(or both)? Vision Research, 37(16):2283–2293.CrossRefGoogle Scholar
  38. Rothwell, C.A., Forsyth, D.A., Zisserman, A., and Mundy, J.L. 1993. Extracting projective structure from single perspective views of3D point sets. In IEEE International Conference on Computer Vision, pp. 573–582.Google Scholar
  39. Schaffalitzky, F. and Zisserman, A. 2000. Planar grouping for automatic detection of vanishing lines and points. Image and Vision Computing, 18:647–658.CrossRefGoogle Scholar
  40. Sun, C. and Sherrah, J. 1997. 3D symmetry detection using the extended gaussian image. IEEE Transactions on Pattern Analysis & Machine Intelligence, 19(2):164–169.Google Scholar
  41. Vetter, T. and Poggio, T. 1994. Symmetric 3D objects are an easy case for 2D object recognition. Spatial Vision, 8:443–453.Google Scholar
  42. Weng, J., Huang, T.S., and Ahuja, N. 1993. Motion and Structure from Image Sequences. Springer Verlag.Google Scholar
  43. Weyl, H. 1952. Symmetry. Princeton Univ. Press.Google Scholar
  44. Witkin, A.P. 1988. Recovering surface shape and orientation from texture. J. of Artificial Intelligence, 17:17–45.CrossRefGoogle Scholar
  45. Yang, A.Y., Rao, S., Huang, K., Hong, W., and Ma, Y. 2003. Geometric segmentation of perspective images based on symmetry groups. In IEEE International Conference on Computer Vision.Google Scholar
  46. Zabrodsky, H., Peleg, S., and Avnir, D. 1995. Symmetry as a continuous feature. IEEE Transactions on Pattern Analysis & Machine Intelligence, 17(12):1154–1166.Google Scholar
  47. Zabrodsky, H. and Weinshall, D. 1997. Using bilateral symmetry to improve 3D reconstruction from image sequences. Comp. Vision and Image Understanding, 67:48–57.CrossRefGoogle Scholar
  48. Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q.-T. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknownepipolar geometry. Artificial Intelligence, 78(1/2):87–119.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Wei Hong
    • 1
  • Allen Yang Yang
    • 1
  • Kun Huang
    • 1
  • Yi Ma
    • 1
  1. 1.Department of Electrical & Computer EngineeringUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations