# Three-Dimensional Reconstruction of Points and Lines with Unknown Correspondence across Images

- 205 Downloads
- 8 Citations

## Abstract

Three-dimensional reconstruction from a set of images is an important and difficult problem in computer vision. In this paper, we address the problem of determining image feature correspondences while simultaneously reconstructing the corresponding 3D features, given the camera poses of disparate monocular views. First, two new affinity measures are presented that capture the degree to which candidate features from different images consistently represent the projection of the same 3D point or 3D line. An affinity measure for point features in two different views is defined with respect to their distance from a hypothetical projected 3D pseudo-intersection point. Similarly, an affinity measure for 2D image line segments across three views is defined with respect to a 3D pseudo-intersection line. These affinity measures provide a foundation for determining unknown correspondences using weighted bipartite graphs representing candidate point and line matches across different images. As a result of this graph representation, a standard graph-theoretic algorithm can provide an optimal, simultaneous matching and triangulation of points across two views, and lines across three views. Experimental results on synthetic and real data demonstrate the effectiveness of the approach.

## Preview

Unable to display preview. Download preview PDF.

## References

- Amerinex Artificial Intelligence. 1991.
*The KBVision System User's Guide*. AAI: Amherst, MA.Google Scholar - Aggarwal, J.K., Davis, L.S., and Martin, W.N. 1981. Correspondence processes in dynamic scene analysis. In
*Proc. of IEEE 69*, pp. 562–572.Google Scholar - Aloimonos, J. and Rigoutsos, I. 1986. Determining 3-D motion of a rigid surface patch without correspondences under perspective projection. In
*Proc. of AAAI*, pp. 681–688.Google Scholar - ARPA. 1994. RADIUS, Section III. In
*Arpa Image Understanding Workshop.*Monterey, CA.Google Scholar - Ayache, N. 1991.
*Artificial Vision for Mobile Robots: Stereo Vision and Multisensory Perception*. The MIT Press: Cambridge, MA.Google Scholar - Basu, A. and Aloimonos J. 1987. A robust algorithm for determining the translation of a rigidly moving surface without correspondence for robotics applications. In
*Proc. of IJCAI*, pp. 815–818.Google Scholar - Bedekar, A.S. and Haralick, R.M. 1996. Finding corresponding points based on Bayesian triangulation. In
*Proc. of CVPR'96*, San Francisco, CA, pp. 61–66.Google Scholar - Blostein, S.D. and Huang, T.S. 1987. Quantization errors in stereo triangulation. In
*Proc. of First International Conf. on Computer Vision*, pp. 325–334.Google Scholar - Boldt, M., Weiss, R., and Riseman, E.M. 1989. Token-based extraction of straight lines.
*IEEE Trans. SMC*, 19(6):1581–1594.Google Scholar - Burns, J.B., Hanson, A.R., and Riseman, E.M. 1986. Extracting straight lines.
*IEEE Trans. PAMI*, 8(4):425–455.Google Scholar - Chang, S.F. and McCormick, S.T. 1989. A fast implementation of a bipartite matching algorithm. Technical Report, Columbia University, New York.Google Scholar
- Cheng, Y.Q. 1996. Acquisition of 3D Models from a Set of 2D Images. Ph.D. Thesis, CMPSCI TR96–91, Computer Science Department, University of Massachusetts.Google Scholar
- Cheng, Y.Q., Collins, R., Hanson, A.R., and Riseman, E.M. 1994. Triangulation without correspondences.
*Arpa Image Understanding Workshop*, Monterey, CA, pp. 993–1000.Google Scholar - Cheng, Y.Q., Wu, V., Collins, R.T., Hanson, A.R., and Riseman, E.M. 1996. Maximum-weight bipartite matching technique and its application in image feature matching. In
*Proc. SPIE VCIP*.Google Scholar - Cheriyan, J. and Maheshwari, S.N. 1989. Analysis of preflowpush algorithms for maximum network flow.
*SIAM J. Comput.*, 18:1057–1086.Google Scholar - Chou, T.C. and Kanatani, K. 1987. Recovering 3D rigid motions without correspondences. In
*Proc. ICCV*, pp. 534–538.Google Scholar - Collins, R.T. 1996. A space-sweep approach to true multi-image matching. In
*Proc. of CVPR'96*, San Francisco, CA, pp. 358–363.Google Scholar - Collins, R.T., Cheng, Y.Q., Jayes, C., Stolle, F., Wang, X.G., Hanson, A.R., and Riseman, E.M. 1998. The ASCENDER system automated site modeling from multiple aerial images.
*Computer Vision and Image Understanding*, 72(2):143–162.Google Scholar - Collins, R.T., Cheng, Y.Q., Jayes, C., Stolle, F., Wang, X.G., Hanson, A.R., and Riseman, E.M. 1996. Site model acquisition and Three-Dimensional Reconstruction of Points and Lines 155 extension from aerial images. In
*Proc. ICCV*, IEEE, pp. 888–893.Google Scholar - Collins, R.T., Hanson, A.R., Riseman, E.M., and Cheng, Y.Q. 1993. Model matching and extension for automated 3D modeling. In
*Proc. IUW*, pp. 197–203.Google Scholar - Collins, R.T., Hanson, A.R., Riseman, E.M., Jayes, C.O., Stolle, F., Wang, X.G., and Cheng, Y.Q. 1996. UMass progress in 3D building model acquisition. In
*Arpa Image Understanding Workshop*, Palm Springs, CA, pp. 305–316.Google Scholar - Collins, R.T., Jayes, C.O., Stolle, F., Wang, X.G., Cheng, Y.Q., Hanson, A.R., and Riseman, E.M. 1995. A system for automated site model acquisition. In
*Integrating Photogrammetric Techniques with Scene Analysis and Machine Vision II*, SPIE Proceeding. Vol. 7617, Orlando, FL, pp. 244–254.Google Scholar - Deriche, R., Vaillant, R., and Faugeras, O.D. 1992. From noisy edge points to 3D reconstruction of a scene: A robust approach and its uncertainty analysis. In
*Proc. of the Second Europeian Conf. on Computer Vision*.Google Scholar - Edmonds, J. 1965. Maximum matching and polyhedron with 0,1 vertices.
*J. Research of the National Bureau of Standards*, 69B.Google Scholar - Edmonds, J. and Karp, R.M. 1972. Theoretical improvements in algorithmic efficiency for network flowproblems.
*J. Assoc. Comput. Math.*, 19:248–264.Google Scholar - Ford, L.R. and Fulkerson, E. 1962.
*Flows in Networks*. Princeton Univ. Press: Princeton, NJ.Google Scholar - Gabow, H.N. 1985. A scaling algorithm for weighted matching on general graphs. In
*Proc. 26th Annual Symp. of the Foundations of Computer Science*, pp. 90–100.Google Scholar - Gabow, H.N. 1990. Data structure for weighted matching and nearest common ancestors with linking.
*Proceedings 1st Annual ACMSIAM Symposium on Discrete Algorithms*, pp. 434–443.Google Scholar - Ganapathy, S. 1984. Decomposition of transformation matrices for robot vision. In
*Proc. IEEE Int. Conf. Robotics and Automation*, Atlanta, GA, pp. 130–139.Google Scholar - Gold, S. and Rangarajan, A. 1996. Graph matching by graduated assignment. In
*Proc. of CVPR'96*, San Francisco, CA, pp. 239–244.Google Scholar - Goldberg, A.V. 1987. Efficient graph algorithms for sequential and parallel computers. Ph.D thesis, MIT, Cambridge, MA.Google Scholar
- Goldberg, A.V. and Tarjan, R.E. 1988. A new approach to the maximum-flow problem.
*J. Assoc. Comput. Mach.*, 35:921–940.Google Scholar - Goldgof, D.B., Lee, H., and Huang, T.S. 1992. Matching and motion estimation of three-dimensional point and line sets using eigenstructure without correspondences.
*Pattern Recognition*, 25(3):271–286.Google Scholar - Griffin, P.M. 1989. Correspondence of 2-D projections by bipartite matching.
*Pattern Recognition Letter*, 9:361–366.Google Scholar - Gruen, A. and Baltsavias, E. 1988. Geometrically constrained multiphoto matching.
*Photogrammetric Engineering and Remote Sensing*, 54(6):633–641.Google Scholar - Hao, J. and Kocur, G. 1993. An implementation of a shortest augmenting path algorithm for the assignment problem.
*Network flows and matching: First DIMACS Implementation Challenge*, Series in DIMACS, American Mathematical Society: Rhode Island, Vol. 12, pp. 453–468.Google Scholar - Hartley, R.I. 1994. Triangulation. In
*Proc. IUW*, pp. 957–966.Google Scholar - Hopcroft, J.E. and Karp, R.M. 1973. An
*n*^{2.5}algorithm for maximum matching in bipartite graphs.*J. ACM*, 23:225–231.Google Scholar - Ito, E. and Aloimonos, J. 1988. Is correspondence necessary for the perception of structure from motion?. In
*Image Understanding Workshop*, pp. 921–929.Google Scholar - Jaynes, C., Stoll, F., and Collins, R. 1994. Task driven perceptual organization for extraction of rooftop polygons. In
*Proceeding ARPA Image Understanding Workshop*, Monterey, CA, pp. 359–365.Google Scholar - Johnson, D.S. and McGeoch, C.C. 1993.
*Network flows and matching: First DIMACS Implementation Challenge*, Series in DIMACS, American Mathematical Society: R.I., Vol. 12.Google Scholar - Jonker, R. and Volgenant, A. 1987. A shortest augmenting path algorithm for dense and sparse linear assignment problems.
*Computing*, 38:325–340.Google Scholar - Kim, W. and Kak, A. 1991. 3D object recognition using bipartite matching embedded in discrete relaxation.
*IEEE PAMI*, 13(3):224–251.Google Scholar - Kahn, P., Kitchen, L., and Riseman, E. 1990. Real-time feature extraction: A fast line finder for vision-guided robot navigation.
*IEEE Pattern Analysis and machine Intelligence*, 12(11):1098–1102.Google Scholar - Kuhn, H.W. 1955. The Hungarian method for assignment problem.
*Naval Research Logistics*, Quarterly, Vol. 2, pp. 83–97.Google Scholar - Kumar, R. 1992.
*Model dependent Inference of 3D information from a sequence of 2D images*. Ph.D. Thesis, CMPSCI TR92–04, Computer Science Department, University of Massachusetts.Google Scholar - Kumar, R. and Hanson, A.R. 1992a. Model extension and refinement using pose recovery techniques.
*Journal of Robotic Systems*, 9(6):753–771.Google Scholar - Kumar, R. and Hanson, A.R. 1992b. Application of pose determination techniques to model extension and refinement.
*Proceedings Darpa Image UnderstandingWorkshop*, San Diego, CA, pp. 727–744.Google Scholar - Kumar, R., Anandan, P., and Hanna, K. 1994. Shape recovery from multiple views: A parallax based approach. In
*Proc. of ARPA Image Understanding*, Monterey, CA, pp. 947–955.Google Scholar - Lee, C.-H. and Joshi, A. 1993. Correspondence problem in image sequence analysis.
*Pattern Recognition*, 26(1):47–61.Google Scholar - Lee, H., Lin, Z., and Huang, T.S. 1986. Finding 3-D point correspondences in motion estimation. In
*Proc. of Eighth Int. Conf. on Pattern Recognition*, pp. 303–305.Google Scholar - Lee, H., Lin, Z., and Huang, T.S. 1990. Estimating rigid-body motion from three-dimentional data without matching point correspondences.
*Int. J. Imaging Systems Technol.*, 2:55–62.Google Scholar - Lessard, R., Rousseau, J.-M., and Minoux, M. 1989.Anewalgorithm for general matching problems using network flow subproblems.
*Networks*, 19:459–479.Google Scholar - Martin Marietta and SRI International. 1993.
*RCDE User's Guide*. Martin Marietta, Management and Data Systems: Philadelpha, PA.Google Scholar - Micali, S. and Vazirani, V. 1980. An \(O\left( {\sqrt {\left| V \right|.\left| E \right|} } \right)\)algorithm for finding maximum matchings in general graphs. In
*Proc. 21st Symp. Foundations of Computer Science*, pp. 17–27.Google Scholar - Pentland, A. and Horowitz, B. 1991. Recovery of non-rigid motion and structure.
*IEEE PAMI*, 13(7):730–742.Google Scholar - Pentland, A. and Sclaroff, S. 1991. Closed-form solutions for physically-based shape modeling and recognition.
*IEEE PAMI*, 13(7):715–729.Google Scholar - Roy, S. and Cox, I.J. 1998. A maximum-flow formulation of the Ncamera stereo correspondence problem. In
*ICCV98*, pp. 492–499.Google Scholar - Sclaroff, S. and Pentland, A. 1993. A modal framework for correspondence and description. In
*Proc. of IEEE*, pp. 308–313.Google Scholar - Scott, G.L. and Longuet-Higgins, H.C. 1991. An algorithm for associating the features of two patterns.
*Proc. Roy Soc Lond*, B244:21–26.Google Scholar - Shapiro, L.S. and Brady, J.M. 1992. Feature-based correspondence: An eigenvector approach.
*Image and Vision Computing*, 10(5):283–288.Google Scholar - Ullman, S. 1979.
*The Interpretation of Visual Motion*.MIT Press: Cambridge, MA.Google Scholar - Wang, X., Cheng, Y.Q., Collins, R.T., and Hanson, A.R. 1996. Determining correspondences and rigid motion of 3-D point sets with missing data. In
*Proc. CVPR*, IEEE, pp. 252–257.Google Scholar - Wu, Z. and Leahy, R. 1993. An optimal graph theoretic approach to data clustering: Theory and its application to image segmentation.
*IEEE PAMI*, 13(7):1101–1113.Google Scholar - Wu, M.S. and Leou, J.J. 1995. A bipartite matching approach to feature correspondence in stereo vision.
*Pattern Recognition Letter*, 16:23–31.Google Scholar - Zhang, Z. and Faugeras, O.D. 1990a. Building a 3D world model with a mobile robot: 3D line segment representation and integration. In
*Proc. of the IEEE International Conf. on Pattern Recognition*, Atlantic City, New Jersey, pp. 38–42.Google Scholar - Zhang, Z. and Faugeras, O.D. 1990b. Tracking and grouping 3D line segments. In
*Proc. of 3rd International Conf. on Computer Vision*, Osaka, Japan, pp. 577–580.Google Scholar