Incremental Singular Value Decomposition of Uncertain Data with Missing Values

  • Matthew Brand
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2350)


We introduce an incremental singular value decomposition (svd) of incomplete data. The svd is developed as data arrives, and can handle arbitrary missing/untrusted values, correlated uncertainty across rows or columns of the measurement matrix, and user priors. Since incomplete data does not uniquely specify an svd, the procedure selects one having minimal rank. For a dense p × q matrix of low rank r, the incremental method has time complexity O(pqr) and space complexity O((p + q)r)—better than highly optimized batch algorithms such as matlab’s svd(). In cases of missing data, it produces factorings of lower rank and residual than batch svd algorithms applied to standard missing-data imputations. We show applications in computer vision and audio feature extraction. In computer vision, we use the incremental svd to develop an efficient and unusually robust subspace-estimating flow-based tracker, and to handle occlusions/missing points in structure-from-motion factorizations.


Singular Vector Uncertain Data Minimal Rank Lanczos Method User Prior 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Michael W. Berry. Large scale singular value computations. International Journal of Supercomputer Applications, 6:13–49, 1992.Google Scholar
  2. 2.
    Stan Birchfeld. Derivation of Kanade-Lucas-Tomasi tracking equation. Web-published manuscript at, 1996.
  3. 3.
    Matthew Brand. 3D morphable models from video. In Proc. CVPR01, 2001.Google Scholar
  4. 4.
    J. R. Bunch and C. P. Nielsen. Updating the singular value decomposition. Numer. Math., 31:111–129, 1978.zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    P. Businger. Updating a singular value decomposition. BIT, 10:376–385, 1970.CrossRefGoogle Scholar
  6. 6.
    Michael A. Casey. MPEG-7 sound-recognition tools. IEEE Transactions on Circuits and Systems for Video Technology, 11(6), June 2001.Google Scholar
  7. 7.
    S. Chandrasekaran, B. S. Manjunath, Y. F. Wang, J. Winkeler, and H. Zhang. An eigenspace update algorithm for image analysis. Graphical models and image processing: GMIP, 59(5):321–332, 1997.CrossRefGoogle Scholar
  8. 8.
    Gene Golub and Arthur van Loan. Matrix Computations. Johns Hopkins U. Press, 1996.Google Scholar
  9. 9.
    Benedikt Großer and Bruno Lang. An O(n 2) algorithm for the bidiagonal SVD. Technical Report BUGHW-SC 2000/4, University of Wuppertal Department of Mathematics, 2000.Google Scholar
  10. 10.
    M. Gu and S. C. Eisenstat. A stable and fast algorithm for updating the singular value decomposition. Tech. Report YALEU/DCS/RR-966, Department of Computer Science, Yale University, New Haven, CT, 1993.Google Scholar
  11. 11.
    Michal Irani. Multi-frame optical flow estimation using subspace constraints. In Proc. ICCV, 1999.Google Scholar
  12. 12.
    Michal Irani and P. Anandan. Factorization with uncertainty. In Proc. European Conf. Computer Vision, 2000.Google Scholar
  13. 13.
    J.E. Jackson. A user’s guide to principal components. Wiley, 1991.Google Scholar
  14. 14.
    A. Levy and M. Lindenbaum. Sequential karhunenloeve basis extraction and its application to images. Technical Report CIS9809, Technion, 1998.Google Scholar
  15. 15.
    Jan R. Magnus and Heinz Neudecker. Matrix differential calculus with applications in statistics and econometrics. Wiley, 1999.Google Scholar
  16. 16.
    D.D. Morris and T. Kanade. A unified factorization algorithm for points, line segments and planes with uncertainty models. In Proc. Sixth ICCV, pages 696–702, Bombay, 1998.Google Scholar
  17. 17.
    D.P. O’Leary and G.W. Stewart. Computing the eigenvalues and eigenvectors of arrowhead matrices. Technical Report CS-TR-2203/UMIACS-TR-89-22, University of Maryland Department of Computer Science, February 1989.Google Scholar
  18. 18.
    Conrad Poelman and Takeo Kanade. A paraperspective factorization method for shape and motion recovery. Technical Report CMU-CS-93-219, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, December 1993.Google Scholar
  19. 19.
    B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl. Application of dimensionality reduction in recommender system—a case study. In ACM WebKDD 2000 Web Mining for E-Commerce Workshop. ACM Press, 2000.Google Scholar
  20. 20.
    J. B. Tenenbaum and W. T. Freeman. Separating style and content with bilinear models. Neural Computation, 12:1247–1283, 2000.CrossRefGoogle Scholar
  21. 21.
    Olga Troyanskaya, Michael Cantor, Gavin Sherlock, Pat Brown, Trevor Hastie, Robert Tib-shirani, David Botstein, and Russ B. Altman. Missing value estimation methods for DNA microarrays. BioInformatics, 17:1–6, 2001.CrossRefGoogle Scholar
  22. 22.
    S. van Huffel and H. Park. Efficient reduction algorithms for bordered band matrices. Numerical Linear Algebra with Applications, 2(2), 1995.Google Scholar
  23. 23.
    Hongyuan Zha and Horst D. Simon. On updating problems in latent semantic indexing. SIAM Journal on Scientific Computing, 21(2):782–791), 1999.zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Matthew Brand
    • 1
  1. 1.Mitsubishi Electric Research LabsCambridgeUSA

Personalised recommendations