# A Generalized Projective Reconstruction Theorem and Depth Constraints for Projective Factorization

## Abstract

This paper presents a generalized version of the classic projective reconstruction theorem which helps to choose or assess depth constraints for projective depth estimation algorithms. The theorem shows that projective reconstruction is possible under a much weaker constraint than requiring all estimated projective depths to be nonzero. This result enables us to present classes of depth constraints under which any reconstruction of cameras and points projecting into given image points is projectively equivalent to the true camera-point configuration. It also completely specifies the possible wrong configurations allowed by other constraints. We demonstrate the application of the theorem by analysing several constraints used in the literature, as well as presenting new constraints with desirable properties. We mention some of the implications of our results on iterative depth estimation algorithms and projective reconstruction via rank minimization. Our theory is verified by running experiments on both synthetic and real data.

## Keywords

Multiple view geometry Projective reconstruction Projective reconstruction theorem Projective factorization Projective depths Constraints on projective depths## References

- Angst, R., Zach, C., & Pollefeys, M. (2011). The generalized trace-norm and its application to structure-from-motion problems. In:
*Proceedings of the IEEE International Conference on Computer Vision*(ICCV) 2011, (pp. 2502–2509). doi: 10.1109/ICCV.2011.6126536. - Buchanan, T. (1988). The twisted cubic and camera calibration.
*Computer Vision, Graphics, and Image Processing*,*42*(1), 130–132.MATHMathSciNetCrossRefGoogle Scholar - Dai, Y., Li, H., & He, M. (2010). Element-wise factorization for n-view projective reconstruction. In:
*Proceedings of the 11th European conference on Computer vision: Part IV*, ECCV’10, (pp. 396–409). Berlin: Springer.Google Scholar - Dai, Y., Li, H., & He, M. (2013). Projective multi-view structure and motion from element-wise factorization.
*IEEE Transactions on Pattern Analysis and Machine Intelligence 35*(9), 2238–2251. doi: 10.1109/TPAMI.2013.20. - Hartley, R., & Kahl, F. (2007). Critical configurations for projective reconstruction from multiple views.
*International Journal of Computer Vision*,*71*(1), 5–47. doi: 10.1007/s11263-005-4796-1.CrossRefGoogle Scholar - Hartley, R., & Schaffalizky, F. (2003). PowerFactorization: 3D reconstruction with missing or uncertain data. In: Proceedings of the Australia-Japan Advanced Workshop on Computer Vision.Google Scholar
- Hartley, R. I., & Zisserman, A. (2004).
*Multiple View Geometry in Computer Vision*(2nd ed.). Cambridge, MA: Cambridge University Press. ISBN: 0521540518.MATHCrossRefGoogle Scholar - Heyden, A., Berthilsson, R., & Sparr, G. (1999). An iterative factorization method for projective structure and motion from image sequences.
*Image and Vision Computing*,*17*(13), 981–991.CrossRefGoogle Scholar - Lin, Z., Chen, M., & Wu, L. (2010). The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. Analysis math.OC (Technical Report UILU-ENG-09-2215):-09-2215.Google Scholar
- Luenberger, D. G. (1984).
*Linear and Nonlinear Programming*. Boston, MA: Addison-Wesley Publishing Company.MATHGoogle Scholar - Mahamud, S., Hebert, M., Omori, Y., & Ponce, J. (2001). Provably-convergent iterative methods for projective structure from motion. In:
*Proceedings of the Computer Vision and Pattern Recognition*(CVPR), (pp. 1018–1025). IEEEGoogle Scholar - Oliensis, J., & Hartley, R. (2007). Iterative extensions of the Sturm/Triggs algorithm: Convergence and nonconvergence.
*Pattern Analysis and Machine Intelligence*,*29*(12), 2217–2233. doi: 10.1109/TPAMI.2007.1132.CrossRefGoogle Scholar - Semple, J., & Kneebone, G. (1952).
*Algebraic Projective Geometry*., Oxford Classic Texts in the Physical Sciences Series Oxford: Clarendon Press.MATHGoogle Scholar - Sinkhorn, R. (1964). A relationship between arbitrary positive matrices and doubly stochastic matrices.
*The Annals of Mathematical Statistics*,*35*(2), 876–879.MATHMathSciNetCrossRefGoogle Scholar - Sinkhorn, R. (1967). Diagonal equivalence to matrices with prescribed row and column sums.
*The American Mathematical Monthly*,*74*(4), 402–405.MATHMathSciNetCrossRefGoogle Scholar - Sturm, P.F., & Triggs, B. (1996). A factorization based algorithm for multi-image projective structure and motion. In:
*Proceedings of the 4th European Conference on Computer Vision*, ECCV ’96, (vol. 2, pp 709–720). London: Springer.Google Scholar - Triggs, B. (1996). Factorization methods for projective structure and motion. In:
*Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition*(CVPR ’96), (pp. 845). IEEE Computer Society, Washington, DC.Google Scholar - Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000). Bundle adjustment - a modern synthesis. In:
*Proceedings of the International Workshop on Vision Algorithms: Theory and Practice*, ICCV ’99, (pp 298–372). London: Springer.Google Scholar - Ueshiba, T., & Tomita, F. (1998). A factorization method for projective and euclidean reconstruction from multiple perspective views via iterative depth estimation.
*Computer*,*I*, 296–310.Google Scholar - Yang, J., & Yuan, X. (2013). Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization.
*Mathematics of Computation*,*82*(281), 301–329. doi: 10.1090/S0025-5718-2012-02598-1.MATHMathSciNetCrossRefGoogle Scholar - Zangwill, W. (1969).
*Nonlinear Programming: A Unified Approach*., Prentice-Hall International Series in Management Englewood Cliffs, NJ: Prentice-Hall.MATHGoogle Scholar