Advertisement

International Journal of Computer Vision

, Volume 40, Issue 3, pp 199–233 | Cite as

The Geometry and Matching of Lines and Curves Over Multiple Views

  • Cordelia Schmid
  • Andrew Zisserman
Article

Abstract

This paper describes the geometry of imaged curves in two and three views. Multi-view relationships are developed for lines, conics and non-algebraic curves. The new relationships focus on determining the plane of the curve in a projective reconstruction, and in particular using the homography induced by this plane for transfer from one image to another. It is shown that given the fundamental matrix between two views, and images of the curve in each view, then the plane of a conic may be determined up to a two fold ambiguity, but local curvature of a curve uniquely determines the plane. It is then shown that given the trifocal tensor between three views, this plane defines a homography map which may be used to transfer a conic or the curvature from two views to a third. Simple expressions are developed for the plane and homography in each case.

A set of algorithms are then described for automatically matching individual line segments and curves between images. The algorithms use both photometric information and the multiple view geometric relationships. For image pairs the homography facilitates the computation of a neighbourhood cross-correlation based matching score for putative line/curve correspondences. For image triplets cross-correlation matching scores are used in conjunction with line/curve transfer based on the trifocal geometry to disambiguate matches. Algorithms are developed for both short and wide baselines. The algorithms are robust to deficiencies in the segment extraction and partial occlusion.

Experimental results are given for image pairs and triplets, for varying motions between views, and for different scene types. The methods are applicable to line/curve matching in stereo and trinocular rigs, and as a starting point for line/curve matching through monocular image sequences.

line curve matching curve conic transfer 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ayache, N. 1990. Stereovision and Sensor Fusion. MIT Press.Google Scholar
  2. Ayache, N. and Faverjon, B. 1987. Efficient registration of stereo images by matching graph descriptions of edge segments. International Journal of Computer Vision, 1(2):107–132.Google Scholar
  3. Ayache, N. and Lustman, F. 1987. Fast and reliable passive stereovision using three cameras. In International Conference on Computer Vision, pp. 422–427.Google Scholar
  4. Baillard, C., Schmid, C., Zisserman, A., and Fitzgibbon, A. 1999. Automatic line matching and 3D reconstruction of buildings from multiple views. In ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery, IAPRS vol. 32, Part 3-2W5, pp. 69–80.Google Scholar
  5. Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In European Conference on Computer Vision, pp. 683–695.Google Scholar
  6. Bignone, F., Henricsson, O., Fua, P., and Stricker, M. 1996. Automatic extraction of generic house roofs from high resolution aerial imagery. In European Conference on Computer Vision, pp. 85–96.Google Scholar
  7. Blake, A. and Cipolla, R. 1990. Robust estimation of surface curvature from deformation of apparent contours. In European Conference on Computer Vision, pp. 465–474.Google Scholar
  8. Bookstein, F.L. 1979. Fitting conic sections to scattered data. Computer Graphics and Image Processing, 9:56–71.Google Scholar
  9. Brint, A.T. and Brady, M. 1989. Stereo matching of curves by least deformation. In InternationalWorkshop on Intelligent Robots and Systems, pp. 163–170.Google Scholar
  10. Carlsson, S., Mohr, R., Morin, L., Rothwell, C., Van Gool L., Veillon, F., and Zisserman, A. 1996. Semi-local projective invariants for the recognition of smooth plane curves. International Journal of Computer Vision, 19(3):211–236.Google Scholar
  11. Chung, R.C.K. and Nevatia, R. 1991. Use of monocular groupings and occlusion analysis in a hierarchical stereo system. In Conference on Computer Vision and Pattern Recognition, pp. 50–56.Google Scholar
  12. Cipolla, R., Astrom, K., and Giblin, P. 1995. Motion from the frontier of curved surfaces. In International Conference on Computer Vision, pp. 269–275.Google Scholar
  13. Cipolla, R. and Zisserman, A. 1992. Qualitative shape from deformation of image curves. International Journal of Computer Vision, 8(1):53–69.Google Scholar
  14. Cross, G. and Zisserman, A. 1998. Quadric reconstruction from dualspace geometry. In International Conference on Computer Vision, pp. 25–31.Google Scholar
  15. Crowley, J.L. and Stelmazyk, P. 1990. Measurement and integration of 3D structures by tracking edges lines. In European Conference on Computer Vision, pp. 269–280.Google Scholar
  16. Deriche, R. and Faugeras, O. 1990. Tracking line segments. In European Conference on Computer Vision, pp. 259–267.Google Scholar
  17. Faugeras, O. 1992. What can be seen in three dimensions with an uncalibrated stereo rig? In European Conference on Computer Vision, pp. 563–578.Google Scholar
  18. Faugeras, O. 1993. Three-Dimensional Computer Vision– A Geometric Viewpoint. Artificial intelligence. MIT Press.Google Scholar
  19. Faugeras, O. and Keriven, R. 1998. Complete dense stereovision using level set methods. In European Conference on Computer Vision, pp. 379–393.Google Scholar
  20. Faugeras, O. and Papadopoulo, T. 1993. A theory of the motion fields of curves. International Journal of Computer Vision, 10(2):125–156.Google Scholar
  21. Faugeras, O. and Robert, L. 1996. What can two images tell us about a third one? International Journal of Computer Vision, 18(1):5–19.Google Scholar
  22. Gdalyahu, Y. and Weinshall, D. 1999. Flexible syntactic matching of curves and its application to automatic hierarchical classification of silhouettes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12):1312–1328.Google Scholar
  23. Gros, P. 1995. Matching and clustering: Two steps towards object modelling in computer vision. International Journal of Robotics Research, 14(6):633–642.Google Scholar
  24. Gruen, A.W. 1985. Adaptative least squares correlation: A powerful image matching technique. S. Afr. Journal of Photogrammetry, Remote Sensing and Cartography, 14(3):175–187.Google Scholar
  25. Hartley, R.I. 1995. A linear method for reconstruction from lines and points. In International Conference on Computer Vision, pp. 882–887.Google Scholar
  26. Hartley, R.I. 1997. Lines and points in three views and the trifocal tensor. International Journal of Computer Vision, 22(2):125–140.Google Scholar
  27. Hartley, R.I., Gupta, R., and Chang, T. 1992. Stereo from uncalibrated cameras. In Conference on Computer Vision and Pattern Recognition, pp. 761–764.Google Scholar
  28. Hartley, R.I. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press.Google Scholar
  29. Havaldar, P. and Medioni, G. 1995. Segmented shape descriptions from 3-view stereo. In International Conference on Computer Vision, pp. 102–108.Google Scholar
  30. Horaud, R. and Skordas, T. 1989. Stereo correspondence through feature grouping and maximal cliques. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(11):1168–1180.Google Scholar
  31. Huttenlocher, D.P., Klanderman, G.A., and Rucklidge, W.J. 1993. Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850–863.Google Scholar
  32. Kahl, F. and Heyden, A. 1998. Using conic correspondences in two images to estimate the epipolar geometry. In International Conference on Computer Vision, pp. 761–766.Google Scholar
  33. Laurent, P.J. 1987. Courbes ouvertes ou fermées par B-splines régularis ées. Technical Report RR 652, IMAG, Grenoble, France.Google Scholar
  34. Luong, Q.T. and Viéeville, T 1996. Canonical representations for the geometries of multiple projective views. Computer Vision and Image Understanding, 64(2):193–229.Google Scholar
  35. Ma, S. 1993. Conics-based stereo, motion estimation, and pose determination. International Journal of Computer Vision, 10(1):7–25.Google Scholar
  36. Maybank, S.J. and Faugeras, O. 1992. A theory of self calibration of a moving camera. International Journal of Computer Vision, 8(2):123–151.Google Scholar
  37. Medioni, G. and Nevatia, R. 1985. Segment-based stereo matching. Computer Vision, Graphics and Image Processing, 31:2–18.Google Scholar
  38. Nayar, S.K. and Bolle, R.M. 1996. Reflectance based object recognition. International Journal of Computer Vision, 17(3):219–240.Google Scholar
  39. Pollard, S.B., Pridmore, T.P., Porrill, J., Mayhew, J.E.W., and Frisby, J.P. 1989. Geometrical modeling from multiple stereo views. International Journal of Robotics Research, 8(4):132–138.Google Scholar
  40. Pollard, S.B., Mayhew, J.E.W., and Frisby, J.P. 1985. PMF: A stereo correspondence algorithm using a disparity gradient constraint. Perception, 14:449–470.Google Scholar
  41. Porrill, J. and Pollard, S.B. 1991. Curve matching and stereo calibration. Image and Vision Computing, 9(1):45–50.Google Scholar
  42. Pritchett, P. and Zisserman, A. 1998. Wide baseline stereo matching. In International Conference on Computer Vision, pp. 754–760.Google Scholar
  43. Quan, L. 1996. Conic reconstruction and correspondence from two views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(2):151–160.Google Scholar
  44. Remagnino, P., Brand, P., and Mohr, R. 1994. Correlation techniques in adaptative template matching with uncalibrated cameras. In Vision Geometry III, SPIE's international symposium on photonic sensors & control for commercial applications, vol. 2356, pp. 252–253.Google Scholar
  45. Robert, L. and Faugeras, O. 1991. Curve-based stereo: Figural continuity and curvature. In Conference on Computer Vision and Pattern Recognition, pp. 57–62.Google Scholar
  46. Rothwell, C., Mundy, J.L., and Hoffman, B. 1996. Representing objects using topology. In Object Representation in Computer Vision II, LNCS 1144, pp. 79–108. Springer-Verlag.Google Scholar
  47. Rothwell, C., Zisserman, A., Forsyth, D., and Mundy, J. 1992. Fast recognition using algebraic invariants. In Geometric Invariance in Computer Vision, MIT Press, pp. 398–407.Google Scholar
  48. Rothwell, C.A. 1993. Recognition using Projective Invariance. Ph.D. Thesis, Department of Engineering Science, University of Oxford.Google Scholar
  49. Safaee-Rad, R., Tchoukanov, I., Benhabib, B., and Smit, K.C. 1992. 3D pose estimation from a quadratic curved feature in two perspective views. In International Confernce on Pattern Recognition, pp. 341–344.Google Scholar
  50. Schmid, C. and Zisserman, A. 1997. Automatic line matching across views. In Conference on Computer Vision and Pattern Recognition, pp. 666–671.Google Scholar
  51. Schmid, C. and Zisserman, A. 1998. The geometry and matching of curves in multiple views. In European Conference on Computer Vision, vol. I, pp. 394–409.Google Scholar
  52. Semple, J. and Kneebone, G. 1979. Algebraic Projective Geometry. Oxford University Press.Google Scholar
  53. Shashua, A. 1994. Trilinearity in visual recognition by alignment. In European Conference on Computer Vision, pp. 479–484.Google Scholar
  54. Spetsakis, M.E. and Aloimonos, J. 1990. Structure from motion using line correspondences. International Journal of Computer Vision, 4(3):171–183.Google Scholar
  55. Torr, P.H.S. and Murray, D.W. 1997. The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, 24(3):271–300.Google Scholar
  56. Torr, P.H.S. and Zisserman, A. 1997. Robust parameterization and computation of the trifocal tensor. Image and Vision Computing, 15:591–605.Google Scholar
  57. Triggs, B. 1997. Auto calibration and the absolute quadric. In Conference on Computer Vision and Pattern Recognition, pp. 609–614.Google Scholar
  58. Vaillant, R. 1990. Using occluding contours for 3D object modeling. In European Conference on Computer Vision, pp. 454–464.Google Scholar
  59. Van Gool, L., Brill, M., Barrett, E., Moons, T., and Pauwels, E. 1992. Semi-differential invariants for nonplanar curves. In Geometric Invariance in Computer Vision, MIT Press, pp. 293–309.Google Scholar
  60. Van Gool, L., Moons, L.T., Pauwels, E. and Oosterlinck, A. 1992. Semi-differential invariants. In Geometric Invariance in Computer Vision, MIT Press, pp. 157–192.Google Scholar
  61. Venkateswar, V. and Chellappa, R. 1995. Hierarchical stereo and motion correspondence using feature groupings. International Journal of Computer Vision, 15(3):245–269.Google Scholar
  62. Zhang, Y. and Gerbrands, J.J. 1995. Method for matching general stereo planar curves. Image and Vision Computing, 13(8):645–655.Google Scholar
  63. Zhang, Z. 1994. Token tracking in a cluttered scene. Image and Vision Computing, 12(2):110–120.Google Scholar
  64. Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q.T. 1995. Arobust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87–119.Google Scholar
  65. Zisserman, A. and Maybank, S. 1994. A case against epipolar geometry. In Applications of Invariance in Computer Vision LNCS 825, Springer-Verlag, pp. 69–88.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Cordelia Schmid
    • 1
  • Andrew Zisserman
    • 2
  1. 1.INRIA Rhône-AlpesMontbonnotFrance
  2. 2.Dept. of Engineering ScienceOxfordUK

Personalised recommendations