Skip to main content

Introduction

  • Chapter
  • First Online:
Generalized Principal Component Analysis

Part of the book series: Interdisciplinary Applied Mathematics ((IAM,volume 40))

Abstract

The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work.

The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work.

—John von Neumann

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 89.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Under fairly broad conditions that we will elaborate in this book.

  2. 2.

    In the statistical learning literature, the most commonly used term is “mixture model.” In systems theory, the typical term is “hybrid model.” In algebraic geometry, for the case of subspaces, the typical term is a “subspace arrangement.”

  3. 3.

    In this book, we will use interchangeably “mixture,” “collection,” “union,” and “arrangement” of subspaces or models. But be aware that in the case of subspaces, the formal terminology in algebraic geometry is a “subspace arrangement.”

  4. 4.

    Or equivalently, we may impose a nonuniform prior distribution over all models.

  5. 5.

    Occam’s (or Ockham’s) razor is a principle attributed to the fourteenth-century logician and Franciscan friar William of Occam: “Pluralitas non est ponenda sine neccesitate,” which translates literally as “entities should not be multiplied unnecessarily.” In science, this principle is often interpreted thus: “when you have two competing theories that make exactly the same predictions, the simpler one is better.

  6. 6.

    Roughly speaking, a smooth manifold is a special topological space that is locally homeomorphic to a Euclidean space and has the same dimension everywhere. A general topological space may have singularities and consist of components of different dimensions.

  7. 7.

    If the true distribution from which the data are drawn is \(q(\boldsymbol{x})\), then the maximum likelihood estimate \(p(\boldsymbol{x}\mid \theta ^{{\ast}})\) minimizes the Kullback–Leibler (KL) divergence \(KL(q\|p) =\int q(\boldsymbol{x})\log \frac{q(\boldsymbol{x})} {p(\boldsymbol{x})}\ d\boldsymbol{x}\) among the given class of distributions (see Appendix B.)

  8. 8.

    Singular distributions are probability distributions concentrated on a set of Lebesgue measure zero. Such distributions are not absolutely continuous with respect to the Lebesgue measure. The Cantor distribution is one example of a singular distribution.

  9. 9.

    As the cluster centers.

  10. 10.

    An object is called Lambertian if its apparent brightness is the same from any viewpoint.

  11. 11.

    Depending on the illumination model, the illumination space can be approximately three- or nine-dimensional.

  12. 12.

    The legitimacy of the projection process will be addressed in Chapter 5

  13. 13.

    In contrast to the previous face example, there is no rigorous mathematical justification for why local profiles from a region of similar texture must span a low-dimensional linear subspace. However, there is strong empirical evidence that a linear subspace normally gives a very good approximation.

  14. 14.

    That is, the number d i of basis elements needed to represent the ith region is typically much smaller than the number d of basis elements needed to represent the whole image.

  15. 15.

    An algebraic variety is an irreducible algebraic set. An algebraic set is called irreducible if it cannot be written as the union of two proper algebraic subsets. A subspace is one such example.

  16. 16.

    A prime ideal is an ideal that cannot be decomposed further as the intersection of two other ideals (see Appendix C). The zero-level set of a prime ideal is an irreducible algebraic set, i.e., an algebraic variety.

  17. 17.

    Notice the correspondence between a “union” of algebraic varieties and the “multiplication” of the polynomials associated with the varieties.

  18. 18.

    According to Hilbert’s Nullstellensatz (see Appendix C), there is a one-to-one correspondence between algebraic sets and radical ideals (Eisenbud 1996).

  19. 19.

    For the special case in which the ideal is generated by a single polynomial, the decomposition is equivalent to factoring the polynomial into factors.

  20. 20.

    For example, every function can be approximated arbitrarily well by a piecewise linear function with a sufficient number of pieces.

  21. 21.

    For instance, the complexity of a model can be measured as the minimum number of bits needed to fully describe the model, and the data fidelity can be measured by the distance from the sample points to the model.

References

  • Agarwal, P., & Mustafa, N. (2004). k-means projective clustering. In ACM Symposium on Principles of Database Systems.

    Google Scholar 

  • Agarwal, S., Lim, J., Zelnik-Manor, L., Perona, P., Kriegman, D., & Belongie, S. (2005). Beyond pairwise clustering. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 838–845).

    Google Scholar 

  • Aggarwal, G., Roy-Chowdhury, A., & Chellappa, R. (2004). A system identification approach for video-based face recognition. In Proceedings of International Conference on Pattern Recognition (pp. 23–26).

    Google Scholar 

  • Akaike, H. (1977). A new look at the statistical model selection. IEEE Transactions on Automatic Control, 16(6), 716–723.

    MathSciNet  MATH  Google Scholar 

  • Aldroubi, A., Cabrelli, C., & Molter, U. (2008). Optimal non-linear models for sparsity and sampling. Journal of Fourier Analysis and Applications, 14(5–6), 793–812.

    Article  MathSciNet  MATH  Google Scholar 

  • Aldroubi, A., & Zaringhalam, K. (2009). Nonlinear least squares in N. Acta Applicandae Mathematicae, 107(1–3), 325–337.

    Article  MathSciNet  MATH  Google Scholar 

  • Alessandri, A., & Coletta, P. (2001). Design of Luenberger observers for a class of hybrid linear systems. In Proceedings of Hybrid Systems: Computation and Control (pp. 7–18). New York: Springer.

    Chapter  Google Scholar 

  • Ali, S., Basharat, A., & Shah, M. (2007). Chaotic invariants for human action recognition. In Proceedings of International Conference on Computer Vision.

    Google Scholar 

  • Amaldi, E., & Kann, V. (1998). On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science, 209, 237–260.

    Article  MathSciNet  MATH  Google Scholar 

  • Anderson, B., & Johnson, R. (1982). Exponential convergence of adaptive identification and control algorithms. Automatica, 18(1), 1–13.

    Article  MathSciNet  MATH  Google Scholar 

  • Arbelaez, P. (2006). Boundary extraction in natural images using ultrametric contour maps. In Workshop on Perceptual Organization in Computer Vision.

    Google Scholar 

  • Arbelaez, P., Maire, M., Fowlkes, C., & Malik, J. (2009). From contours to regions: An empirical evaluation. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Arora, S., Bhaskara, A., Ge, R., & Ma, T. (2014). Provable bounds for learning some deep representations. In Proceedings of International Conference on Machine Learning.

    Google Scholar 

  • Avidan, S., & Shashua, A. (2000). Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(4), 348–357.

    Article  Google Scholar 

  • Ayazoglu, M., Li, B., Dicle, C., Sznaier, M., & Camps, O. (2011). Dynamic subspace-based coordinated multicamera tracking. In IEEE International Conference on Computer Vision (pp. 2462–2469)

    Google Scholar 

  • Babaali, M., & Egerstedt, M. (2004). Observability of switched linear systems. In Proceedings of Hybrid Systems: Computation and Control. New York: Springer.

    Google Scholar 

  • Bach, F. (2013). Convex relaxations of structured matrix factorizations. arXiv:1309.3117v1.

    Google Scholar 

  • Bach, F., Mairal, J., & Ponce, J. (2008). Convex sparse matrix factorizations. http://arxiv.org/abs/0812.1869

    Google Scholar 

  • Balluchi, A., Benvenuti, L., Benedetto, M. D., & Sangiovanni-Vincentelli, A. (2002). Design of observers for hybrid systems. In Proceedings of Hybrid Systems: Computation and Control (Vol. 2289, pp. 76–89). New York: Springer.

    Google Scholar 

  • Baraniuk, R. (2007). Compressive sensing. IEEE Signal Processing Magazine, 24(4), 118–121.

    Article  MathSciNet  Google Scholar 

  • Barbic, J., Safonova, A., Pan, J.-Y., Faloutsos, C., Hodgins, J. K., & Pollar, N. S. (2004). Segmenting motion capture data into distinct behaviors. In Graphics Interface.

    Google Scholar 

  • Barnett, V., & Lewis, T. (1983). Outliers in statistical data (2nd ed.). New York: Wiley.

    MATH  Google Scholar 

  • Basri, R., & Jacobs, D. (2003). Lambertian reflection and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.

    Article  Google Scholar 

  • Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202.

    Article  MathSciNet  MATH  Google Scholar 

  • Béjar, B., Zappella, L., & Vidal, R. (2012). Surgical gesture classification from video data. In Medical Image Computing and Computer Assisted Intervention (pp. 34–41).

    Google Scholar 

  • Belhumeur, P., Hespanda, J., & Kriegeman, D. (1997). Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711–720.

    Article  Google Scholar 

  • Belhumeur, P., & Kriegman, D. (1998). What is the set of images of an object under all possible illumination conditions? International Journal of Computer Vision, 28(3), 1–16.

    Article  Google Scholar 

  • Belkin, M., & Niyogi, P. (2002). Laplacian eigenmaps and spectral techniques for embedding and clustering. In Proceedings of Neural Information Processing Systems (NIPS) (pp. 585–591).

    Google Scholar 

  • Beltrami, E. (1873). Sulle funzioni bilineari. Giornale di Mathematiche di Battaglini, 11, 98–106.

    MATH  Google Scholar 

  • Bemporad, A., Ferrari, G., & Morari, M. (2000). Observability and controllability of piecewise affine and hybrid systems. IEEE Transactions on Automatic Control, 45(10), 1864–1876.

    Article  MathSciNet  MATH  Google Scholar 

  • Bemporad, A., Garulli, A., Paoletti, S., & Vicino, A. (2003). A greedy approach to identification of piecewise affine models. In Hybrid systems: Computation and control. Lecture notes in computer science (pp. 97–112). New York: Springer.

    Google Scholar 

  • Bemporad, A., Roll, J., & Ljung, L. (2001). Identification of hybrid systems via mixed-integer programming. In Proceedings of IEEE Conference on Decision & Control (pp. 786–792).

    Google Scholar 

  • Benson, H. (1994). Concave minimization: Theory, applications and algorithms. In R. Horst & P. M. Pardalos (Eds.), Handbook of global optimization (vol. 2, pp. 43-148), Springer Verlag.

    Google Scholar 

  • Bertsekas, D. P. (1999). Nonlinear programming (2nd ed.). Optimization and computation (Vol. 2) Belmont: Athena Scientific.

    Google Scholar 

  • Bickel, P. J. (1976). Another look at robustness: A review of reviews and some new developments. Scandinavian Journal of Statistics, 3(28), 145–168.

    MathSciNet  MATH  Google Scholar 

  • Bickel, P. J., & Doksum, K. A. (2000). Mathematical statistics: Basic ideas and selected topics (2nd ed.). Upper Saddle River: Prentice Hall.

    MATH  Google Scholar 

  • Billio, M., Monfort, A., & Robert, C. (1999). Bayesian estimation of switching ARMA models. Journal of Econometrics, 93(2), 229–255.

    Article  MathSciNet  MATH  Google Scholar 

  • Bissacco, A., Chiuso, A., Ma, Y., & Soatto, S. (2001). Recognition of human gaits. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 52–58).

    Google Scholar 

  • Björner, A. (1994). Subspace arrangements. In First European Congress of Mathematics, Vol. I (Paris, 1992). Progress in mathematics (Vol. 119, pp. 321–370). Basel: Birkhäuser.

    Google Scholar 

  • Björner, A., Peeva, I., & Sidman, J. (2005). Subspace arrangements defined by products of linear forms. Journal of the London Mathematical Society 71 (2), 273–288.

    Article  MathSciNet  MATH  Google Scholar 

  • Blake, A., North, B., & Isard, M. (1999). Learning multi-class dynamics. Advances in Neural Information Processing Systems, 11, 389–395. Cambridge: MIT Press.

    Google Scholar 

  • Bochnak, J., Coste, M., & Roy, M. F. (1998). Real Algebraic Geometry. New York: Springer.

    Book  MATH  Google Scholar 

  • Bottou, L., & Bengio, J. (1995). Convergence properties of the k-means algorithms. In Neural Information Processing and Systems.

    Google Scholar 

  • Boult, T., & Brown, L. (1991). Factorization-based segmentation of motions. In IEEE Workshop on Motion Understanding (pp. 179–186).

    Google Scholar 

  • Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2010). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.

    Article  MATH  Google Scholar 

  • Boyd, S., & Vandenberghe, L. (2004). Convex Optimization. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Bradley, P. S., & Mangasarian, O. L. (2000). k-plane clustering. Journal of Global Optimization, 16(1), 23–32.

    Article  MathSciNet  MATH  Google Scholar 

  • Brandt, S. (2002). Closed-form solutions for affine reconstruction under missing data. In In Proceedings Statistical Methods for Video Processing (ECCV’02 Workshop).

    Google Scholar 

  • Broomhead, D. S., & Kirby, M. (2000). A new approach to dimensionality reduction theory and algorithms. SIAM Journal of Applied Mathematics, 60(6), 2114–2142.

    Article  MathSciNet  MATH  Google Scholar 

  • Bruckstein, A., Donoho, D., & Elad, M. (2009). From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review, 51(1), 34–81.

    Article  MathSciNet  MATH  Google Scholar 

  • Buchanan, A., & Fitzgibbon, A. (2005). Damped Newton algorithms for matrix factorization with missing data. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 316–322).

    Google Scholar 

  • Burer, S., & Monteiro, R. D. C. (2005). Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, Series A, 103(3), 427–444.

    Article  MathSciNet  MATH  Google Scholar 

  • Burges, C. (2005). Geometric methods for feature extraction and dimensional reduction - a guided tour. In The data mining and knowledge discovery handbook (pp. 59–92). Boston: Kluwer Academic.

    Chapter  Google Scholar 

  • Burges, C. J. C. (2010). Dimension reduction: A guided tour. Foundations and Trends in Machine Learning, 2(4), 275–365.

    Article  MATH  Google Scholar 

  • Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach. New York: Springer.

    MATH  Google Scholar 

  • Burt, P. J., & Adelson, E. H. (1983). The Laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4), 532–540.

    Article  Google Scholar 

  • Cai, J.-F., Candés, E. J., & Shen, Z. (2008). A singular value thresholding algorithm for matrix completion. SIAM Journal of Optimization, 20(4), 1956–1982.

    Article  MathSciNet  MATH  Google Scholar 

  • Campbell, N. (1978). The influence function as an aid in outlier detection in discriminant analysis. Applied Statistics, 27(3), 251–258.

    Article  MATH  Google Scholar 

  • Campbell, R. J. (1980). Robust procedures in multivariate analysis I: Robust covariance analysis. Applied Statistics, 29, 231–237.

    Article  MATH  Google Scholar 

  • Candès, E. (2006). Compressive sampling. In Proceedings of the International Congress of Mathematics.

    Google Scholar 

  • Candès, E. (2008). The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique, 346(9–10), 589–592.

    Article  MathSciNet  MATH  Google Scholar 

  • Candès, E., & Donoho, D. (2002). New tight frames of curvelets and optimal representations of objects with smooth singularities. Technical Report. Stanford University.

    MATH  Google Scholar 

  • Candès, E., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis? Journal of the ACM, 58(3).

    Google Scholar 

  • Candès, E., & Plan, Y. (2010). Matrix completion with noise. Proceedings of the IEEE, 98(6), 925–936.

    Article  Google Scholar 

  • Candès, E., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9, 717–772.

    Article  MathSciNet  MATH  Google Scholar 

  • Candès, E., & Recht, B. (2011). Simple bounds for low-complexity model reconstruction. Mathematical Programming Series A, 141(1–2), 577–589.

    MATH  Google Scholar 

  • Candès, E., & Tao, T. (2005). Decoding by linear programming. IEEE Transactions on Information Theory, 51(12), 4203–4215.

    Article  MathSciNet  MATH  Google Scholar 

  • Candès, E., & Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5), 2053–2080.

    Article  MathSciNet  Google Scholar 

  • Candès, E., & Wakin, M. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30.

    Article  Google Scholar 

  • Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.

    Article  Google Scholar 

  • Cetingül, H. E., Wright, M., Thompson, P., & Vidal, R. (2014). Segmentation of high angular resolution diffusion MRI using sparse riemannian manifold clustering. IEEE Transactions on Medical Imaging, 33(2), 301–317.

    Article  Google Scholar 

  • Chan, A., & Vasconcelos, N. (2005a). Classification and retrieval of traffic video using auto-regressive stochastic processes. In Proceedings of 2005 IEEE Intelligent Vehicles Symposium (pp. 771–776).

    Google Scholar 

  • Chan, A., & Vasconcelos, N. (2005b). Mixtures of dynamic textures. In IEEE International Conference on Computer Vision (Vol. 1, pp. 641–647).

    Google Scholar 

  • Chandrasekaran, V., Sanghavi, S., Parrilo, P., & Willsky, A. (2009). Sparse and low-rank matrix decompositions. In IFAC Symposium on System Identification.

    Google Scholar 

  • Chaudhry, R., Ravichandran, A., Hager, G., & Vidal, R. (2009). Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Chen, G., Atev, S., & Lerman, G. (2009). Kernel spectral curvature clustering (KSCC). In Workshop on Dynamical Vision.

    Google Scholar 

  • Chen, G., & Lerman, G. (2009a). Foundations of a multi-way spectral clustering framework for hybrid linear modeling. Foundations of Computational Mathematics, 9(5), 517–558.

    Google Scholar 

  • Chen, G., & Lerman, G. (2009b). Spectral curvature clustering (SCC). International Journal of Computer Vision, 81(3), 317–330.

    Google Scholar 

  • Chen, J.-Q., Pappas, T. N., Mojsilovic, A., & Rogowitz, B. E. (2003). Image segmentation by spatially adaptive color and texture features. In IEEE International Conference on Image Processing.

    Google Scholar 

  • Chen, S., Donoho, D., & Saunders, M. (1998). Atomic decomposition by basis pursuit. SIAM Journal of Scientific Computing, 20(1), 33–61.

    Article  MathSciNet  MATH  Google Scholar 

  • Chung, F. (1997). Spectral graph theory. Washington: Conference Board of the Mathematical Sciences.

    MATH  Google Scholar 

  • Cilibrasi, R., & Vitányi, P. M. (2005). Clustering by compression. IEEE Transactions on Information Theory, 51(4), 1523–1545.

    Article  MathSciNet  MATH  Google Scholar 

  • CMU (2003). MOCAP database. http://mocap.cs.cmu.edu.

  • Coifman, R., & Wickerhauser, M. (1992). Entropy-based algorithms for best bases selection. IEEE Transactions on Information Theory, 38(2), 713–718.

    Article  MATH  Google Scholar 

  • Collins, M., Dasgupta, S., & Schapire, R. (2001). A generalization of principal component analysis to the exponential family. In Neural Information Processing Systems (Vol. 14)

    Google Scholar 

  • Collins, P., & Schuppen, J. V. (2004). Observability of piecewise-affine hybrid systems. In Proceedings of Hybrid Systems: Computation and Control. New York: Springer.

    Google Scholar 

  • Comanicu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 24, 603–619.

    Article  Google Scholar 

  • Costeira, J., & Kanade, T. (1998). A multibody factorization method for independently moving objects. International Journal of Computer Vision, 29(3), 159–179.

    Article  Google Scholar 

  • Cour, T., Benezit, F., & Shi, J. (2005). Spectral segmentation with multiscale graph decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    Google Scholar 

  • Cover, T., & Thomas, J. (1991). Elements of information theory. Wiley.

    Book  MATH  Google Scholar 

  • Cox, T. F., & Cox, M. A. A. (1994). Multidimensional scaling. London: Chapman and Hall.

    MATH  Google Scholar 

  • Critchley, F. (1985). Influence in principal components analysis. Biometrika, 72(3), 627–636.

    Article  MathSciNet  MATH  Google Scholar 

  • Davis, C., & Cahan, W. (1970). The rotation of eigenvectors by a pertubation. SIAM Journal on Numerical Analysis, 7(1), 1–46.

    Article  MathSciNet  Google Scholar 

  • Davison, M. (1983). Multidimensional Scaling. New York: Wiley.

    MATH  Google Scholar 

  • De la Torre, F., & Black, M. J. (2004). A framework for robust subspace learning. International Journal of Computer Vision, 54(1), 117–142.

    MATH  Google Scholar 

  • Delsarte, P., Macq, B., & Slock, D. (1992). Signal-adapted multiresolution transform for image coding. IEEE Transactions on Information Theory, 38, 897–903.

    Article  Google Scholar 

  • Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39(1), 1–38.

    MathSciNet  MATH  Google Scholar 

  • Deng, W., Lai, M.-J., Peng, Z., & Yin, W. (2013). Parallel multi-block admm with o(1/k) convergence. UCLA CAM.

    Google Scholar 

  • Deng, Y., & Manjunath, B. (2001). Unsupervised segmentation of color-texture regions in images and video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8), 800–810.

    Article  Google Scholar 

  • Derksen, H. (2007). Hilbert series of subspace arrangements. Journal of Pure and Applied Algebra, 209(1), 91–98.

    Article  MathSciNet  MATH  Google Scholar 

  • DeVore, R. (1998). Nonlinear approximation. Acta Numerica, 7, 51–150.

    Article  MathSciNet  MATH  Google Scholar 

  • DeVore, R., Jawerth, B., & Lucier, B. (1992). Image compression through wavelet transform coding. IEEE Transactions on Information Theory, 38(2), 719–746.

    Article  MathSciNet  MATH  Google Scholar 

  • Ding, C., Zha, H., He, X., Husbands, P., & Simon, H. D. (2004). Link analysis: Hubs and authoraties on the world wide web. SIAM Review, 46(2), 256–268.

    Article  MathSciNet  MATH  Google Scholar 

  • Do, M. N., & Vetterli, M. (2002). Contourlets: A directional multiresolution image representation. In IEEE International Conference on Image Processing.

    Google Scholar 

  • Donoho, D. (1995). Cart and best-ortho-basis: A connection. Manuscript.

    MATH  Google Scholar 

  • Donoho, D. (1998). Sparse components analysis and optimal atomic decomposition. Technical Report, Department of Statistics, Stanford University.

    Google Scholar 

  • Donoho, D., & Gavish, M. (2014). The optimal hard threshold for singular values is \(4/\sqrt{3}\). IEEE Transactions on Information Theory, 60(8), 5040–5053.

    Article  MathSciNet  Google Scholar 

  • Donoho, D., & Grimes, C. (2003). Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. National Academy of Sciences, 100(10), 5591–5596.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L. (1999). Wedgelets: Nearly-minimax estimation of edges. Annals of Statistics, 27, 859–897.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L. (2005). Neighborly polytopes and sparse solution of underdetermined linear equations. Technical Report. Stanford University.

    Google Scholar 

  • Donoho, D. L. (2006). For most large underdetermined systems of linear equations the minimal 1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 59(6), 797–829.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L., & Elad, M. (2003). Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization. Proceedings of National Academy of Sciences, 100(5), 2197–2202.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L., Vetterli, M., DeVore, R., & Daubechies, I. (1998). Data compression and harmonic analysis. IEEE Transactions on Information Theory, 44(6), 2435–2476.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoser, M., Urschler, M., Hirzer, M., & Bischof, H. (2009). Saliency driven total variation segmentation. In Proceedings of the International Conference on Computer Vision (ICCV).

    Google Scholar 

  • Doretto, G., Chiuso, A., Wu, Y., & Soatto, S. (2003). Dynamic textures. International Journal of Computer Vision, 51(2), 91–109.

    Article  MATH  Google Scholar 

  • Doretto, G., & Soatto, S. (2003). Editable dynamic textures. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 137–142).

    Google Scholar 

  • Doretto, G., & Soatto, S. (2006). Dynamic shape and appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2006–2019.

    Article  Google Scholar 

  • Doucet, A., Logothetis, A., & Krishnamurthy, V. (2000). Stochastic sampling algorithms for state estimation of jump Markov linear systems. IEEE Transactions on Automatic Control, 45(1), 188–202.

    Article  MathSciNet  MATH  Google Scholar 

  • Duda, R., Hart, P., & Stork, D. (2000). Pattern Classification (2nd ed.). Wiley, New York.

    MATH  Google Scholar 

  • Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1, 211–218.

    Article  MATH  Google Scholar 

  • Effros, M., & Chou, P. (1995). Weighted universal transform coding: Universal image compression with the Karhunen-Loéve transform. In IEEE International Conference on Image Processing (Vol. 2, pp. 61–64).

    Google Scholar 

  • Efros, A. A., & Leung, T. K. (1999). Texture synthesis by non-parametric sampling. In IEEE International Conference on Computer Vision (pp. 1033–1038). Corfu, Greece.

    Book  Google Scholar 

  • Eisenbud, D. (1996). Commutative algebra: With a view towards algebraic geometry. Graduate texts in mathematics. New York: Springer.

    Google Scholar 

  • Elad, M., & Bruckstein, A. (2001). On sparse signal representations. In IEEE International Conference on Image Processing.

    Google Scholar 

  • Elad, M., & Bruckstein, A. (2002). A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Transactions on Information Theory, 48(9), 2558–2567.

    Article  MathSciNet  MATH  Google Scholar 

  • Elad, M., Figueiredo, M. A. T., & Ma, Y. (2010). On the role of sparse and redundant representations in image processing. Proceedings of the IEEE, 98(6), 972–982.

    Article  Google Scholar 

  • Elder, J., & Zucker, S. (1996). Computing contour closures. In Proceedings of the European Conference on Computer Vision (ECCV).

    Google Scholar 

  • Elhamifar, E., Sapiro, G., & Vidal, R. (2012a). Finding exemplars from pairwise dissimilarities via simultaneous sparse recovery. In Neural Information Processing and Systems.

    Google Scholar 

  • Elhamifar, E., Sapiro, G., & Vidal, R. (2012b). See all by looking at a few: Sparse modeling for finding representative objects. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Elhamifar, E., & Vidal, R. (2009). Sparse subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Elhamifar, E., & Vidal, R. (2010). Clustering disjoint subspaces via sparse representation. In IEEE International Conference on Acoustics, Speech, and Signal Processing.

    Google Scholar 

  • Elhamifar, E., & Vidal, R. (2011). Sparse manifold clustering and embedding. In Neural Information Processing and Systems.

    Google Scholar 

  • Elhamifar, E., & Vidal, R. (2013). Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(11), 2765–2781.

    Article  Google Scholar 

  • Ezzine, J., & Haddad, A. H. (1989). Controllability and observability of hybrid systems. International Journal of Control, 49(6), 2045–2055.

    Article  MathSciNet  MATH  Google Scholar 

  • Favaro, P., Vidal, R., & Ravichandran, A. (2011). A closed form solution to robust subspace estimation and clustering. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Fazel, M., Hindi, H., & Boyd, S. (2003). Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices. In Proceedings of the American Control Conference (pp. 2156–2162).

    Google Scholar 

  • Fei-Fei, L., Fergus, R., & Perona, P. (2004). Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In Workshop on Generative Model Based Vision.

    Google Scholar 

  • Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision (IJCV), 59(2), 167–181.

    Article  Google Scholar 

  • Feng, J., Xu, H., Mannor, S., & Yang, S. (2013). Online PCA for contaminated data. In NIPS.

    Google Scholar 

  • Feng, X., & Perona, P. (1998). Scene segmentation from 3D motion. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 225–231).

    Google Scholar 

  • Ferguson, T. (1961). On the rejection of outliers. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability.

    Google Scholar 

  • Ferrari-Trecate, G., Mignone, D., & Morari, M. (2002). Moving horizon estimation for hybrid systems. IEEE Transactions on Automatic Control, 47(10), 1663–1676.

    Article  MathSciNet  Google Scholar 

  • Ferrari-Trecate, G., Muselli, M., Liberati, D., & Morari, M. (2003). A clustering technique for the identification of piecewise affine systems. Automatica, 39(2), 205–217.

    Article  MathSciNet  MATH  Google Scholar 

  • Feuer, A., Nemirovski, A. (2003). On sparse representation in pairs of bases. IEEE Transactions on Information Theory, 49(6), 1579–1581.

    Article  MathSciNet  MATH  Google Scholar 

  • Figueiredo, M. A. T., & Jain, A. K. (2002). Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3), 381–396.

    Article  Google Scholar 

  • Fischler, M. A., & Bolles, R. C. (1981). RANSAC random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 26, 381–395.

    Article  MathSciNet  Google Scholar 

  • Fisher, Y. (1995). Fractal Image Compression: Theory and Application. Springer-Verlag Telos.

    Book  Google Scholar 

  • Fitzgibbon, A., & Zisserman, A. (2000). Multibody structure and motion: 3D reconstruction of independently moving objects. In European Conference on Computer Vision (pp. 891–906).

    Google Scholar 

  • Forgy, E. (1965). Cluster analysis of multivariate data: Efficiency vs. interpretability of classifications (abstract). Biometrics, 21, 768–769.

    Google Scholar 

  • Freixenet, J., Munoz, X., Raba, D., Marti, J., & Cuff, X. (2002). Yet another survey on image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV).

    Google Scholar 

  • Frey, B., Colmenarez, A., & Huang, T. (1998). Mixtures of local linear subspaces for face recognition. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Gabriel, K. R. (1978). Least squares approximation of matrices by additive and multiplicative models. Journal of the Royal Statistical Society B, 40, 186–196.

    MathSciNet  MATH  Google Scholar 

  • Ganesh, A., Wright, J., Li, X., Candès, E., & Ma, Y. (2010). Dense error correction for low-rank matrices via principal component pursuit. In International Symposium on Information Theory.

    Google Scholar 

  • Geman, S., & McClure, D. (1987). Statistical methods for tomographic image reconstruction. In Proceedings of the 46th Session of the ISI, Bulletin of the ISI (Vol. 52, pp. 5–21).

    Google Scholar 

  • Georghiades, A., Belhumeur, P., & Kriegman, D. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.

    Article  Google Scholar 

  • Gersho, A., & Gray, R. M. (1992). Vector Quantization and Signal Compression. Boston: Kluwer Academic.

    Book  MATH  Google Scholar 

  • Gevers, T., & Smeulders, A. (1997). Combining region splitting and edge detection through guided Delaunay image subdivision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    Google Scholar 

  • Ghahramani, Z., & Beal, M. (2000). Variational inference for Bayesian mixtures of factor analysers. Advances in Neural Information Processing Systems, 12, 449–455.

    Google Scholar 

  • Ghahramani, Z., & Hinton, G. (1996). The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96-1, University of Toronto, Canada.

    Google Scholar 

  • Ghahramani, Z., & Hinton, G. E. (1998). Variational learning for switching state-space models. Neural Computation, 12(4), 963–996.

    Google Scholar 

  • Ghoreyshi, A., & Vidal, R. (2007). Epicardial segmentation in dynamic cardiac MR sequences using priors on shape, intensity, and dynamics, in a level set framework. In IEEE International Symposium on Biomedical Imaging (pp. 860–863).

    Google Scholar 

  • Gnanadesikan, R., & Kettenring, J. (1972). Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics, 28(1), 81–124.

    Article  Google Scholar 

  • Goh, A., & Vidal, R. (2007). Segmenting motions of different types by unsupervised manifold clustering. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Goh, A., & Vidal, R. (2008). Unsupervised Riemannian clustering of probability density functions. In European Conference on Machine Learning.

    Google Scholar 

  • Goldfarb, D., & Ma, S. (2009). Convergence of fixed point continuation algorithms for matrix rank minimization. Preprint.

    Google Scholar 

  • Golub, H., & Loan, C. V. (1996). Matrix Computations (2nd ed.). Baltimore: Johns Hopkins University Press.

    MATH  Google Scholar 

  • Govindu, V. (2005). A tensor decomposition for geometric grouping and segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1150–1157).

    Google Scholar 

  • Gower, J. (1966). Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika, 53, 325–338.

    Article  MathSciNet  MATH  Google Scholar 

  • Gross, D. (2011). Recovering low-rank matrices from few coefficients in any basis. IEEE Trans on Information Theory, 57(3), 1548–1566.

    Article  MathSciNet  Google Scholar 

  • Gruber, A., & Weiss, Y. (2004). Multibody factorization with uncertainty and missing data using the EM algorithm. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. I, pp. 707–714).

    Google Scholar 

  • H.Aanaes, Fisker, R., Astrom, K., & Carstensen, J. M. (2002). Robust factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9), 1215–1225.

    Article  Google Scholar 

  • Haeffele, B., & Vidal, R. (2015). Global optimality in tensor factorization, deep learning, and beyond. Preprint, http://arxiv.org/abs/1506.07540.

  • Haeffele, B., Young, E., & Vidal, R. (2014). Structured low-rank matrix factorization: Optimality, algorithm, and applications to image processing. In International Conference on Machine Learning.

    Google Scholar 

  • Hamkins, J., & Zeger, K. (2002). Gaussian source coding with spherical codes. IEEE Transactions on Information Theory, 48(11), 2980–2989.

    Article  MathSciNet  MATH  Google Scholar 

  • Hampel, F., Ronchetti, E., Rousseeuw, P., & Stahel, W. (1986). Robust statistics: The approach based on influence functions. New York: Wiley.

    MATH  Google Scholar 

  • Hampel, F. R. (1974). The influence curve and its role in robust estiamtion. Journal of the American Statistical Association, 69, 383–393.

    Article  MathSciNet  MATH  Google Scholar 

  • Han, M., & Kanade, T. (2000). Reconstruction of a scene with multiple linearly moving objects. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 542–549).

    Google Scholar 

  • Han, M., & Kanade, T. (2001). Multiple motion scene reconstruction from uncalibrated views. In Proceedings of IEEE International Conference on Computer Vision (Vol. 1, pp. 163–170).

    Google Scholar 

  • Hansen, M., & Yu, B. (2001). Model selection and the principle of minimum description length. Journal of American Statistical Association, 96, 746–774.

    Article  MathSciNet  MATH  Google Scholar 

  • Haralick, R., & Shapiro, L. (1985). Image segmentation techniques. Computer Vision, Graphics, and Image Processing, 29(1), 100–132.

    Article  Google Scholar 

  • Hardt, M. (2014). Understanding alternating minimization for matrix completion. In Symposium on Foundations of Computer Science.

    Google Scholar 

  • Haro, G., Randall, G., & Sapiro, G. (2006). Stratification learning: Detecting mixed density and dimensionality in high dimensional point clouds. In Neural Information Processing and Systems.

    Google Scholar 

  • Haro, G., Randall, G., & Sapiro, G. (2008). Translated poisson mixture model for stratification learning. International Journal of Computer Vision, 80(3), 358–374.

    Article  Google Scholar 

  • Harris, J. (1992). Algebraic Geometry: A First Course. New York: Springer.

    Book  MATH  Google Scholar 

  • Hartley, R., & Schaffalitzky, F. (2003). Powerfactorization: An approach to affine reconstruction with missing and uncertain data. In Proceedings of Australia-Japan Advanced Workshop on Computer Vision.

    Google Scholar 

  • Hartley, R., & Vidal, R. (2004). The multibody trifocal tensor: Motion segmentation from 3 perspective views. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. I, pp. 769–775).

    Google Scholar 

  • Hartley, R., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Hastie, T. (1984). Principal curves and surfaces. Technical Report, Stanford University.

    Google Scholar 

  • Hastie, T., & Stuetzle, W. (1989). Principal curves. Journal of the American Statistical Association, 84(406), 502–516.

    Article  MathSciNet  MATH  Google Scholar 

  • Hastie, T., Tibshirani, R., & Friedman, J. (2001). The Elements of Statistical Learning. New York: Springer.

    Book  MATH  Google Scholar 

  • He, H., & Garcia, E. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.

    Article  Google Scholar 

  • He, H., & Ma, Y. (2013). Imbalanced Learning: Foundations, Algorithms, and Applications. New York: Wiley.

    Book  MATH  Google Scholar 

  • Hinton, G., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554.

    Article  MathSciNet  MATH  Google Scholar 

  • Hirsch, M. (1976). Differential Topology. New York: Springer.

    Book  MATH  Google Scholar 

  • Ho, J., Yang, M., Lim, J., Lee, K., & Kriegman, D. (2003). Clustering appearances of objects under varying illumination conditions. In Proceedings of International Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Hong, W., Wright, J., Huang, K., & Ma, Y. (2006). Multi-scale hybrid linear models for lossy image representation. IEEE Transactions on Image Processing, 15(12), 3655–3671.

    Article  MathSciNet  Google Scholar 

  • Horn, R. A., & Johnson, C. R. (1985). Matrix Analysis. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 417–441.

    Article  MATH  Google Scholar 

  • Householder, A. S., & Young, G. (1938). Matrix approximation and latent roots. American Mathematical Monthly, 45, 165–171.

    Article  MathSciNet  MATH  Google Scholar 

  • Huang, K., Ma, Y., & Vidal, R. (2004). Minimum effective dimension for mixtures of subspaces: A robust GPCA algorithm and its applications. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 631–638).

    Google Scholar 

  • Huber, P. (1981). Robust Statistics. New York: Wiley.

    Book  MATH  Google Scholar 

  • Hubert, L., Meulman, J., & Heiser, W. (2000). Two purposes for matrix factorization: A historical appraisal. SIAM Review, 42(1), 68–82.

    Article  MathSciNet  MATH  Google Scholar 

  • Hwang, I., Balakrishnan, H., & Tomlin, C. (2003). Observability criteria and estimator design for stochastic linear hybrid systems. In Proceedings of European Control Conference.

    Google Scholar 

  • Hyndman, M., Jepson, A., & Fleet, D. J. (2007). Higher-order autoregressive models for dynamic textures. In British Machine Vision Conference (pp. 76.1–76.10). doi:10.5244/C.21.76.

  • Jacobs, D. (2001). Linear fitting with missing data: Applications to structure-from-motion. Computer Vision and Image Understanding, 82, 57–81.

    Article  MATH  Google Scholar 

  • Jain, A. (1989). Fundamentals of Digital Image Processing. Upper Saddle River: Prentice Hall.

    MATH  Google Scholar 

  • Jain, P., Meka, R., & Dhillon, I. (2010). Guaranteed rank minimization via singular value projection. In Neural Information Processing Systems (pp. 937–945).

    Google Scholar 

  • Jain, P., & Netrapalli, P. (2014). Fast exact matrix completion with finite samples. In http://arxiv.org/pdf/1411.1087v1.pdf.

  • Jain, P., Netrapalli, P., & Sanghavi, S. (2012). Low-rank matrix completion using alternating minimization. In http://arxiv.org/pdf/1411.1087v1.pdf.

  • Jancey, R. (1966). Multidimensional group analysis. Australian Journal of Botany, 14, 127–130.

    Article  Google Scholar 

  • Jarret, K., Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2009). What is the best multi-stage architecture for object recognition. In International Conference on Computer Vision.

    Google Scholar 

  • Jhuo, I.-H., Liu, D., Lee, D., & Chang, S.-F. (2012). Robust visual domain adaptation with low-rank reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 2168–2175).

    Google Scholar 

  • Johnson, C. (1990). Matrix completion problems: A survey. In Proceedings of Symposia in Applied Mathematics.

    Google Scholar 

  • Jolliffe, I. (1986). Principal Component Analysis. New York: Springer.

    Book  MATH  Google Scholar 

  • Jolliffe, I. (2002). Principal Component Analysis (2nd ed.). New York: Springer.

    MATH  Google Scholar 

  • Jordan, M. (1874). Mémoire sur les formes bilinéaires. Journal de Mathématiques Pures et Appliqués, 19, 35–54.

    MATH  Google Scholar 

  • Juloski, A., Heemels, W., & Ferrari-Trecate, G. (2004). Data-based hybrid modelling of the component placement process in pick-and-place machines. In Control Engineering Practice. Amsterdam: Elsevier.

    Google Scholar 

  • Kamvar, S., Klein, D., & Manning, C. (2002). Interpreting and extending classical agglomerative clustering methods using a model-based approach. Technical Report 2002-11, Stanford University Department of Computer Science.

    Google Scholar 

  • Kanatani, K. (1998). Geometric information criterion for model selection. International Journal of Computer Vision (pp. 171–189).

    Google Scholar 

  • Kanatani, K. (2001). Motion segmentation by subspace separation and model selection. In IEEE International Conference on Computer Vision (Vol. 2, pp. 586–591).

    Google Scholar 

  • Kanatani, K. (2002). Evaluation and selection of models for motion segmentation. In Asian Conference on Computer Vision (pp. 7–12).

    Google Scholar 

  • Kanatani, K. (2003). How are statistical methods for geometric inference justified? In Workshop on Statistical and Computational Theories of Vision, IEEE International Conference on Computer Vision.

    Google Scholar 

  • Kanatani, K., & Matsunaga, C. (2002). Estimating the number of independent motions for multibody motion segmentation. In European Conference on Computer Vision (pp. 25–31).

    Google Scholar 

  • Kanatani, K., & Sugaya, Y. (2003). Multi-stage optimization for multi-body motion segmentation. In Australia-Japan Advanced Workshop on Computer Vision (pp. 335–349).

    Google Scholar 

  • Ke, Q., & Kanade, T. (2005). Robust 1-norm factorization in the presence of outliers and missing data. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Keshavan, R., Montanari, A., & Oh, S. (2010a). Matrix completion from a few entries. IEEE Transactions on Information Theory.

    Google Scholar 

  • Keshavan, R., Montanari, A., & Oh, S. (2010b). Matrix completion from noisy entries. Journal of Machine Learning Research, 11, 2057–2078.

    Google Scholar 

  • Keshavan, R. H. (2012). Efficient algorithms for collaborative filtering. Ph.D. Thesis. Stanford University.

    Google Scholar 

  • Kim, J., Fisher, J., Yezzi, A., Cetin, M., & Willsky, A. (2005). A nonparametric statistical method for image segmentation using information theory and curve evolution. PAMI, 14(10), 1486–1502.

    MathSciNet  Google Scholar 

  • Kim, S. J., Doretto, G., Rittscher, J., Tu, P., Krahnstoever, N., & Pollefeys, M. (2009). A model change detection approach to dynamic scene modeling. In Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009 (AVSS ’09) (pp. 490–495).

    Google Scholar 

  • Kim, S. J., Koh, K., Lustig, M., Boyd, S., & Gorinevsky, D. (2007). An interior-point method for large-scale l1-regularized least squares. IEEE Journal on Selected Topics in Signal Processing, 1(4), 606–617.

    Article  Google Scholar 

  • Kim, T., Lee, K., & Lee, S. (2010). Learning full pairwise affinities for spectral segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    Google Scholar 

  • Kleinberg, J. M. (1999). Authorative sources in a hyberlinked environment. Journal of the ACM, 48, 604–632.

    Article  MathSciNet  Google Scholar 

  • Kontogiorgis, S., & Meyer, R. (1989). A variable-penalty alternating direction method for convex optimization. Mathematical Programming, 83, 29–53.

    MathSciNet  MATH  Google Scholar 

  • Kruskal, J. (1964). Nonmetric multidimensional scaling: A numerical method. Psychometrika.

    Google Scholar 

  • Kurita, T. (1995). An efficient clustering algorithm for region merging. IEICE Transactions of Information and Systems, E78-D(12), 1546–1551.

    Google Scholar 

  • Lanczos, C. (1950). An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards, 45, 255–282.

    Article  MathSciNet  Google Scholar 

  • Lang, S. (1993). Algebra (3rd ed.). Reading: Addison-Wesley.

    MATH  Google Scholar 

  • Lee, J. A., & Verleysen, M. (2007). Nonlinear Dimensionality Reduction (1st ed.). New York: Springer.

    Book  MATH  Google Scholar 

  • Lee, K.-C., Ho, J., & Kriegman, D. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 684–698.

    Article  Google Scholar 

  • Leonardis, A., Bischof, H., & Maver, J. (2002). Multiple eigenspaces. Pattern Recognition, 35(11), 2613–2627.

    Article  MATH  Google Scholar 

  • LePennec, E., & Mallat, S. (2005). Sparse geometric image representation with bandelets. IEEE Transactions on Image Processing, 14(4), 423–438.

    Article  MathSciNet  Google Scholar 

  • Levina, E., & Bickel, P. J. (2006). Texture synthesis and non-parametric resampling of random fields. Annals of Statistics, 34(4), 1751–1773.

    Article  MathSciNet  MATH  Google Scholar 

  • Li, B., Ayazoglu, M., Mao, T., Camps, O. I., & Sznaier, M. (2011). Activity recognition using dynamic subspace angles. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (pp. 3193–3200). New York: IEEE.

    Google Scholar 

  • Lin, Z., Chen, M., Wu, L., & Ma, Y. (2011). The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055v2.

    Google Scholar 

  • Lions, P., & Mercier, B. (1979). Splitting algorithms for the sum of two nonlinear operators. SIAM Journal on Numerical Analysis, 16(6), 964–979.

    Article  MathSciNet  MATH  Google Scholar 

  • Liu, G., Lin, Z., Yan, S., Sun, J., & Ma, Y. (2013). Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Analysis and Machine Intelligence, 35(1), 171–184.

    Article  Google Scholar 

  • Liu, G., Lin, Z., & Yu, Y. (2010). Robust subspace segmentation by low-rank representation. In International Conference on Machine Learning.

    Google Scholar 

  • Liu, Y. K., & Zalik, B. (2005). Efficient chain code with Huffman coding. Pattern Recognition, 38(4), 553–557.

    Article  Google Scholar 

  • Lloyd, S. (1957). Least squares quantization in PCM. Technical Report. Bell Laboratories. Published in 1982 in IEEE Transactions on Information Theory, 28, 128–137.

    Google Scholar 

  • Luenberger, D. G. (1973). Linear and Nonlinear Programming. Reading: Addison-Wesley.

    MATH  Google Scholar 

  • Luo, Z. Q., & Tseng, P. (1993). One the convergence rate of dual ascent methods for strictly convex minimization. Mathematics of Operations Research, 18, 846–867.

    Article  MathSciNet  MATH  Google Scholar 

  • Ma, S. (2012). Alternating proximal gradient method for convex minimization. Technical Report.

    Google Scholar 

  • Ma, Y., Derksen, H., Hong, W., & Wright, J. (2007). Segmentation of multivariate mixed data via lossy coding and compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(9), 1546–1562.

    Article  Google Scholar 

  • Ma, Y., Soatto, S., Kosecka, J., & Sastry, S. (2003). An Invitation to 3D Vision: From Images to Geometric Models. New York: Springer.

    MATH  Google Scholar 

  • Ma, Y., & Vidal, R. (2005). Identification of deterministic switched ARX systems via identification of algebraic varieties. In Hybrid Systems: Computation and Control (pp. 449–465). New York: Springer.

    Chapter  Google Scholar 

  • Ma, Y., Yang, A. Y., Derksen, H., & Fossum, R. (2008). Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Review, 50(3), 413–458.

    Article  MathSciNet  MATH  Google Scholar 

  • MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (pp. 281–297).

    Google Scholar 

  • Madiman, M., Harrison, M., & Kontoyiannis, I. (2004). Minimum description length vs. maximum likelihood in lossy data compression. In Proceedings of the 2004 IEEE International Symposium on Information Theory.

    Google Scholar 

  • Malik, J., Belongie, S., Leung, T., & Shi, J. (2001). Contour and texture analysis for image segmentation. International Journal of Computer Vision, 43(1), 7–27.

    Article  MATH  Google Scholar 

  • Mallat, S. (1999). A Wavelet Tour of Signal Processing (2nd ed.). London: Academic.

    MATH  Google Scholar 

  • Mallows, C. (1973). Some comments on C p . Technometrics, 15, 661–675.

    MATH  Google Scholar 

  • Maronna, R. A. (1976). Robust M-estimators of multivariate location and scatter. Annals of Statistics, 4, 51–67.

    Article  MathSciNet  MATH  Google Scholar 

  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In IEEE International Conference on Computer Vision.

    Google Scholar 

  • McLanchlan, G. J., & Krishnan, T. (1997). The EM Algorithms and Extentions. Wiley Series in Probability and Statistics. John Wiley & Sons, Inc.

    Google Scholar 

  • Meila, M. (2005). Comparing clusterings: An axiomatic view. In Proceedings of the International Conference on Machine Learning.

    Google Scholar 

  • Mercer, J. (1909). Functions of positive and negative types and their connection with the theory of integral equations. Philosophical Transactions, Royal Society London, A, 209(1909), 415–446.

    Article  MATH  Google Scholar 

  • Meyer, F. (2000). Fast adaptive wavelet packet image compression. IEEE Transactions on Image Processing, 9(5), 792–800.

    Article  Google Scholar 

  • Meyer, F. (2002). Image compression with adaptive local cosines. IEEE Transactions on Image Processing, 11(6), 616–629.

    Article  Google Scholar 

  • Minka, T. (2000). Automatic choice of dimensionality for PCA. In Neural Information Processing Systems (Vol. 13, pp. 598–604).

    Google Scholar 

  • Mirsky, L. (1975). A trace inequality of John von Neumann. Monatshefte für Mathematic, 79, 303–306.

    Article  MathSciNet  MATH  Google Scholar 

  • Mobahi, H., Rao, S., Yang, A., & Sastry, S. (2011). Segmentation of natural images by texture and boundary compression. International Journal of Computer Vision, 95(1), 86–98.

    Article  Google Scholar 

  • Mori, G., Ren, X., Efros, A., & Malik, J. (2004). Recovering human body configurations: Combining segmentation and recognition. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Muresan, D., & Parks, T. (2003). Adaptive principal components and image denoising. In IEEE International Conference on Image Processing.

    Google Scholar 

  • Murphy, K. (1998). Switching Kalman filters. Technical Report. U.C. Berkeley.

    Google Scholar 

  • Nascimento, J. C., Figueiredo, M. A. T., & Marques, J. S. (2005). Recognition of human activities using space dependent switched dynamical models. In IEEE International Conference on Image Processing (pp. 852–855).

    Google Scholar 

  • Neal, R., & Hinton, G. (1998). A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. Jordan (Ed.), Learning in graphical models (pp. 355–368). Boston: Kluwer Academic.

    Chapter  Google Scholar 

  • Negahban, S., Ravikumar, P., Wainwright, M., & Yu, B. (2010). A unified framework for analyzing m-estimators with decomposible regularizers. Available at http://arxiv.org/abs/1010.2731v1.

    MATH  Google Scholar 

  • Nemirovskii, A. S., & Yudin, D. B. (1979). Complexity of problems and efficiency of optimization methods (in Russian). Moscow: Nauka.

    Google Scholar 

  • Nesterov, Y. (1983). A method of solving a convex programming problem with convergence rate O(1∕k 2). Soviet Mathematics Doklady, 27(2), 372–376.

    MATH  Google Scholar 

  • Ng, A., Weiss, Y., & Jordan, M. (2001). On spectral clustering: Analysis and an algorithm. In Proceedings of Neural Information Processing Systems (NIPS) (pp. 849–856).

    Google Scholar 

  • Niessen, H., & A.Juloski (2004). Comparison of three procedures for identification of hybrid systems. In Conference on Control Applications.

    Google Scholar 

  • Nunez, F., & Cipriano, A. (2009). Visual information model based predictor for froth speed control in flotation process. Minerals Engineering, 22(4), 366–371.

    Article  Google Scholar 

  • Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013). Berkeley MHAD: A comprehensive multimodal human action database. In IEEE Workshop on Applications of Computer Vision.

    Google Scholar 

  • Olshausen, B., & D.J.Field (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583), 607–609.

    Article  Google Scholar 

  • Orlik, P. (1989). Introduction to Arrangements. Conference Board of the Mathematical Sciences Regional Conference Series in Mathematics (Vol. 72). Providence: American Mathematics Society.

    Google Scholar 

  • Overschee, P. V., & Moor, B. D. (1993). Subspace algorithms for the stochastic identification problem. Automatica, 29(3), 649–660.

    Article  MathSciNet  MATH  Google Scholar 

  • Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2014). Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Magazine, 32(3), 53–69.

    Article  Google Scholar 

  • Pavlovic, V., Moulin, P., & Ramchandran, K. (1998). An integrated framework for adaptive subband image coding. IEEE Transactions on Signal Processing, 47(4), 1024–1038.

    Article  Google Scholar 

  • Pavlovic, V., Rehg, J. M., Cham, T. J., & Murphy, K. P. (1999). A dynamic Bayesian network approach to figure tracking using learned dynamic models. In Proceedings of the International Conference on Computer Vision (pp. 94–101).

    Google Scholar 

  • Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh and Dublin Philosphical Magazine and Journal of Science, 2, 559–572.

    Article  MATH  Google Scholar 

  • Peng, Z., Yan, M., & Yin, W. (2013). Parallel and distributed sparse optimization. In Asilomar.

    Google Scholar 

  • Polito, M., & Perona, P. (2002). Grouping and dimensionality reduction by locally linear embedding. In Proceedings of Neural Information Processing Systems (NIPS).

    Google Scholar 

  • Powell, M. J. D. (1973). On search directions for minimization algorithms. Mathematical Programming, 4, 193–201.

    Article  MathSciNet  MATH  Google Scholar 

  • Qiu, Q., Patel, V. M., Turaga, P., & Chellappa, R. (2012). Domain adaptive dictionary learning. In European Conference on Computer Vision (Vol. 7575, pp. 631–645).

    Google Scholar 

  • Rabiee, H., Kashyap, R., & Safavian, S. (1996). Adaptive multiresolution image coding with matching and basis pursuits. In IEEE International Conference on Image Processing.

    Google Scholar 

  • Rahimi, A., Darrell, T., & Recht, B. (2005). Learning appearance manifolds from video. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 1, pp. 868–875).

    Google Scholar 

  • Ramchandran, K., & Vetterli, M. (1993). Best wavelet packets bases in a rate-distortion sense. IEEE Transactions on Image Processing, 2, 160–175.

    Article  Google Scholar 

  • Ramchandran, K., Vetterli, M., & Herley, C. (1996). Wavelets, subband coding, and best basis. Proceedings of the IEEE, 84(4), 541–560.

    Article  Google Scholar 

  • Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336), 846–850.

    Article  Google Scholar 

  • Rao, S., Mobahi, H., Yang, A., & Sastry, S. (2009). Natural image segmentation with adaptive texture and boundary encoding. In Asian Conference on Computer Vision, 1 (pp. 135–146).

    Google Scholar 

  • Rao, S., Tron, R., Ma, Y., & Vidal, R. (2008). Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Rao, S., Tron, R., Vidal, R., & Ma, Y. (2010). Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(10), 1832–1845.

    Article  Google Scholar 

  • Rao, S., Yang, A. Y., Wagner, A., & Ma, Y. (2005). Segmentation of hybrid motions via hybrid quadratic surface analysis. In IEEE International Conference on Computer Vision (pp. 2–9).

    Google Scholar 

  • Ravichandran, A., Chaudhry, R., & Vidal, R. (2009). View-invariant dynamic texture recognition using a bag of dynamical systems. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Ravichandran, A., Chaudhry, R., & Vidal, R. (2013). Categorizing dynamic textures using a bag of dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2), 342–353.

    Article  Google Scholar 

  • Ravichandran, A., & Vidal, R. (2008). Video registration using dynamic textures. In European Conference on Computer Vision.

    Google Scholar 

  • Ravichandran, A., & Vidal, R. (2011). Video registration using dynamic textures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 158–171.

    Article  Google Scholar 

  • Ravichandran, A., Vidal, R., & Halperin, H. (2006). Segmenting a beating heart using polysegment and spatial GPCA. In IEEE International Symposium on Biomedical Imaging (pp. 634–637).

    Google Scholar 

  • Recht, B., Fazel, M., & Parrilo, P. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3), 471–501.

    Article  MathSciNet  MATH  Google Scholar 

  • Ren, X., Fowlkes, C., & Malik, J. (2005). Scale-invariant contour completion using condition random fields. In IEEE International Conference on Computer Vision.

    Google Scholar 

  • Ren, X., Fowlkes, C., & Malik, J. (2008). Learning probabilistic models for contour completion in natural images. International Journal of Computer Vision, 77, 47–63.

    Article  Google Scholar 

  • Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14, 465–471.

    Article  MATH  Google Scholar 

  • Rose, K. (1998). Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proceedings of the IEEE, 86(11), 2210–2239.

    Article  Google Scholar 

  • Rousseeuw, P. (1984). Least median of squares regression. Journal of American Statistics Association, 79, 871–880.

    Article  MathSciNet  MATH  Google Scholar 

  • Roweis, S., & Saul, L. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2323–2326.

    Article  Google Scholar 

  • Roweis, S., & Saul, L. (2003). Think globally, fit locally: Unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research, 4, 119–155.

    MathSciNet  MATH  Google Scholar 

  • Saisan, P., Bissacco, A., Chiuso, A., & Soatto, S. (2004). Modeling and synthesis of facial motion driven by speech. In European Conference on Computer Vision (Vol. 3, pp. 456–467).

    Google Scholar 

  • Santis, E., Benedetto, M. D., & Giordano, P. (2003). On observability and detectability of continuous-time linear switching systems. In Proceedings of IEEE Conference on Decision & Control (pp. 5777–5782).

    Google Scholar 

  • Schindler, K., & Suter, D. (2005). Two-view multibody structure-and-motion with outliers. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Schölkopf, B., & Smola, A. (2002). Learning with kernels. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Schölkopf, B., Smola, A., & Muller, K. R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10, 1299–1319.

    Article  Google Scholar 

  • Selim, S., & Ismail, M. A. (1984). K-means-type algorithms: A generalized convergence theorem and characterization of local optimality. IEEE Transaction on Pattern Analysis and Machine Intelligence, 6(1), 81–87.

    Article  MATH  Google Scholar 

  • Sha, F., & Saul, L. (2005). Analysis and extension of spectral methods for nonlinear dimensionality reduction. In Proceedings of International Conference on Machine Learning (pp. 784–791).

    Google Scholar 

  • Shabalin, A., & Nobel, A. (2010). Reconstruction of a low-rank matrix in the presence of gaussian noise (pp. 1–34). arXiv preprint 1007.4148

    Google Scholar 

  • Shakernia, O., Vidal, R., & Sastry, S. (2003). Multi-body motion estimation and segmentation from multiple central panoramic views. In IEEE International Conference on Robotics and Automation (Vol. 1, pp. 571–576).

    Google Scholar 

  • Shapiro, J. M. (1993). Embedded image coding using zerotrees of wavelet coefficients. IEEE Transactions on Signal Processing, 41(12), 3445–3463.

    Article  MATH  Google Scholar 

  • Shashua, A., & Levin, A. (2001). Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects. In Proceedings of IEEE International Conference on Computer Vision (Vol. 2, pp. 592–599).

    Google Scholar 

  • Shekhar, S., Patel, V. M., Nguyen, H. V., & Chellappa, R. (2013). Generalized domain-adaptive dictionaries. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Shi, J., & Malik, J. (1998). Motion segmentation and tracking using normalized cuts. In IEEE International Conference on Computer Vision (pp. 1154–1160).

    Google Scholar 

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.

    Article  Google Scholar 

  • Shi, T., Belkin, M., & Yin, B. (2008). Data spectroscopy: Eigenspace of convolution operators and clustering. arXiv:0807.3719v1.

    Google Scholar 

  • Shizawa, M., & Mase, K. (1991). A unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 289–295).

    Google Scholar 

  • Shum, H.-Y., Ikeuchi, K., & Reddy, R. (1995). Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(9), 854–867.

    Article  Google Scholar 

  • Sikora, T., & Makai, B. (1995). Shape-adaptive DCT for generic coding of video. IEEE Transactions on Circuits and Systems For Video Technology, 5, 59–62.

    Article  Google Scholar 

  • Soltanolkotabi, M., & Candès, E. J. (2013). A geometric analysis of subspace clustering with outliers. Annals of Statistics, 40(4), 2195–2238.

    Article  MathSciNet  MATH  Google Scholar 

  • Soltanolkotabi, M., Elhamifar, E., & Candès, E. J. (2014). Robust subspace clustering. Annals of Statistics, 42(2), 669–699.

    Article  MathSciNet  MATH  Google Scholar 

  • Souvenir, R., & Pless, R. (2005). Manifold clustering. In Proceedings of International Conference on Computer Vision (Vol. I, pp. 648–653).

    Google Scholar 

  • Spielman, D., Wang, H., & Wright, J. (2012). Exact recovery of sparsity-used dictionaries. Conference on Learning Theory (COLT).

    Google Scholar 

  • Starck, J.-L., Elad, M., & Donoho, D. (2003). Image decomposition: Separation of texture from piecewise smooth content. In Proceedings of the SPIE (Vol. 5207, pp. 571–582).

    Google Scholar 

  • Steward, C. V. (1999). Robust parameter estimation in computer vision. SIAM Review, 41(3), 513–537.

    Article  MathSciNet  Google Scholar 

  • Sturm, P. (2002). Structure and motion for dynamic scenes - the case of points moving in planes. In Proceedings of European Conference on Computer Vision (pp. 867–882).

    Google Scholar 

  • Sun, A., Ge, S. S., & Lee, T. H. (2002). Controllability and reachability criteria for switched linear systems. Automatica, 38, 775–786.

    Article  MathSciNet  MATH  Google Scholar 

  • Sun, J., Qu, Q., & Wright, J. (2015). Complete dictionary recovery over the sphere. Preprint. http://arxiv.org/abs/1504.06785

    Book  Google Scholar 

  • Szigeti, F. (1992). A differential algebraic condition for controllability and observability of time varying linear systems. In Proceedings of IEEE Conference on Decision and Control (pp. 3088–3090).

    Google Scholar 

  • Szummer, M., & Picard, R. W. (1996). Temporal texture modeling. In IEEE International Conference on Image Processing (Vol. 3, pp. 823–826).

    Google Scholar 

  • Taubin, G. (1991). Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(11), 1115–1138.

    Article  Google Scholar 

  • Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319–2323.

    Article  Google Scholar 

  • Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, 58(1), 267–288.

    MathSciNet  MATH  Google Scholar 

  • Tipping, M., & Bishop, C. (1999a). Mixtures of probabilistic principal component analyzers. Neural Computation, 11(2), 443–482.

    Google Scholar 

  • Tipping, M., & Bishop, C. (1999b). Probabilistic principal component analysis. Journal of the Royal Statistical Society, 61(3), 611–622.

    Google Scholar 

  • Torgerson, W. (1958). Theory and Methods of Scaling. New York: Wiley.

    Google Scholar 

  • Torr, P., & Davidson, C. (2003). IMPSAC: Synthesis of importance sampling and random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(3), 354–364.

    Article  Google Scholar 

  • Torr, P., Szeliski, R., & Anandan, P. (2001). An integrated Bayesian approach to layer extraction from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(3), 297–303.

    Article  Google Scholar 

  • Torr, P. H. S. (1998). Geometric motion segmentation and model selection. Philosophical Transactions of the Royal Society of London, 356(1740), 1321–1340.

    Article  MathSciNet  MATH  Google Scholar 

  • Tremeau, A., & Borel, N. (1997). A region growing and merging algorithm to color segmentation. Pattern Recognition, 30(7), 1191–1204.

    Article  Google Scholar 

  • Tron, R., & Vidal, R. (2007). A benchmark for the comparison of 3-D motion segmentation algorithms. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Tse, D., & Viswanath, P. (2005). Fundamentals of Wireless Communications. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Tseng, P. (2000). Nearest q-flat to m points. Journal of Optimization Theory and Applications, 105(1), 249–252.

    Article  MathSciNet  MATH  Google Scholar 

  • Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 109(3), 475–494.

    Article  MathSciNet  MATH  Google Scholar 

  • Tu, Z., & Zhu, S. (2002). Image segmentation by data-driven Markov Chain Monte Carlo. PAMI, 24(5), 657–673.

    Article  Google Scholar 

  • Tugnait, J. K. (1982). Detection and estimation for abruptly changing systems. Automatica, 18(5), 607–615.

    Article  MathSciNet  MATH  Google Scholar 

  • Turaga, P., Veeraraghavan, A., Srivastava, A., & Chellappa, R. (2011). Statistical computations on special manifolds for image and video-based recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11), 2273–2286.

    Article  Google Scholar 

  • Turk, M., & Pentland, A. (1991). Face recognition using eigenfaces. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (pp. 586–591).

    Google Scholar 

  • Udell, M., Horn, C., Zadeh, R., & Boyd, S. (2015). Generalized low rank models. Working manuscript.

    Google Scholar 

  • Ueda, N., Nakan, R., & Ghahramani, Z. (2000). SMEM algorithm for mixture models. Neural Computation, 12, 2109–2128.

    Article  Google Scholar 

  • Vapnik, V. (1995). The nature of statistical learning theory. New York: Springer.

    Book  MATH  Google Scholar 

  • Varma, M., & Zisserman, A. (2003). Texture classification: Are filter banks necessary? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Vasilescu, M., & Terzopoulos, D. (2002). Multilinear analysis of image ensembles: Tensorfaces. In Proceedings of European Conference on Computer Vision (pp. 447–460).

    Google Scholar 

  • Vecchio, D. D., & Murray, R. (2004). Observers for a class of hybrid systems on a lattice. In Proceedings of Hybrid Systems: Computation and Control. New York: Springer.

    Google Scholar 

  • Vetterli, M., & Kovacevic, J. (1995). Wavelets and subband coding. Upper Saddle River: Prentice-Hall.

    MATH  Google Scholar 

  • Vidal, R. (2004). Identification of PWARX hybrid models with unknown and possibly different orders. In American Control Conference (pp. 547–552).

    Google Scholar 

  • Vidal, R. (2005). Multi-subspace methods for motion segmentation from affine, perspective and central panoramic cameras. In IEEE Conference on Robotics and Automation (pp. 1753–1758).

    Google Scholar 

  • Vidal, R. (2008). Recursive identification of switched ARX systems. Automatica, 44(9), 2274–2287.

    Article  MathSciNet  MATH  Google Scholar 

  • Vidal, R., Chiuso, A., & Soatto, S. (2002a). Observability and identifiability of jump linear systems. In IEEE Conference on Decision and Control (pp. 3614–3619).

    Google Scholar 

  • Vidal, R., Chiuso, A., Soatto, S., & Sastry, S. (2003a). Observability of linear hybrid systems. In Hybrid Systems: Computation and Control (pp. 526–539). New York: Springer.

    Google Scholar 

  • Vidal, R., & Favaro, P. (2014). Low rank subspace clustering (LRSC). Pattern Recognition Letters, 43, 47–61.

    Article  Google Scholar 

  • Vidal, R., & Hartley, R. (2004). Motion segmentation with missing data by PowerFactorization and Generalized PCA. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 310–316).

    Google Scholar 

  • Vidal, R., & Ma, Y. (2004). A unified algebraic approach to 2-D and 3-D motion segmentation. In European Conference on Computer Vision (pp. 1–15).

    Google Scholar 

  • Vidal, R., Ma, Y., & Piazzi, J. (2004). A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. I, pp. 510–517).

    Google Scholar 

  • Vidal, R., Ma, Y., & Sastry, S. (2003b). Generalized Principal Component Analysis (GPCA). In IEEE Conference on Computer Vision and Pattern Recognition (Vol. I, pp. 621–628).

    Google Scholar 

  • Vidal, R., Ma, Y., Soatto, S., & Sastry, S. (2006). Two-view multibody structure from motion. International Journal of Computer Vision, 68(1), 7–25.

    Article  Google Scholar 

  • Vidal, R., & Ravichandran, A. (2005). Optical flow estimation and segmentation of multiple moving dynamic textures. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 516–521).

    Google Scholar 

  • Vidal, R., & Sastry, S. (2003). Optimal segmentation of dynamic scenes from two perspective views. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 281–286).

    Google Scholar 

  • Vidal, R., Soatto, S., Ma, Y., & Sastry, S. (2002b). Segmentation of dynamic scenes from the multibody fundamental matrix. In ECCV Workshop on Visual Modeling of Dynamic Scenes.

    Google Scholar 

  • Vidal, R., Soatto, S., Ma, Y., & Sastry, S. (2003c). An algebraic geometric approach to the identification of a class of linear hybrid systems. In IEEE Conference on Decision and Control (pp. 167–172).

    Google Scholar 

  • Vidal, R., Tron, R., & Hartley, R. (2008). Multiframe motion segmentation with missing data using PowerFactorization and GPCA. International Journal of Computer Vision, 79(1), 85–105.

    Article  Google Scholar 

  • von Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and Computing, 17(4), 395–416.

    Article  MathSciNet  Google Scholar 

  • Wallace, C., & Boulton, D. (1968). An information measure for classification. The Computer Journal, 11, 185–194.

    Article  MATH  Google Scholar 

  • Wallace, C., & Dowe, D. (1999). Minimum message length and Kolmogrov complexity. The Computer Journal, 42(4), 270–283.

    Article  MATH  Google Scholar 

  • Wallace, G. K. (1991). The JPEG still picture compression standard. Communications of the ACM. Special issue on digital multimedia systems, 34(4), 30–44.

    Google Scholar 

  • Wang, J., Jia, Y., Hua, X., Zhang, C., & Quan, L. (2008a). Normalized tree partitioning for image segmentation. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Wang, J. M., Fleet, D. J., & Hertzmann, A. (2008b). Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2), 283–298.

    Google Scholar 

  • Wang, Y.-X., & Xu, H. (2013). Noisy sparse subspace clustering. In International Conference on Machine learning.

    Google Scholar 

  • Ward, J. (1963). Hierarchical grouping to optimize and objective function. Journal of the American Statistical Association, 58, 236–244.

    Article  MathSciNet  Google Scholar 

  • Warga, J. (1963). Minimizing certain convex functions. SIAM Journal on Applied Mathematics, 11, 588–593.

    Article  MathSciNet  MATH  Google Scholar 

  • Wei, S., & Lin, Z. (2010). Analysis and improvement of low rank representation for subspace segmentation. Technical Report MSR-TR-2010-177, Microsoft Research Asia.

    Google Scholar 

  • Weinberger, K. Q., & Saul, L. (2004). Unsupervised learning of image manifolds by semidefinite programming. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (pp. 988–955).

    Google Scholar 

  • Wiberg, T. (1976). Computation of principal components when data are missing. In Symposium on Computational Statistics (pp. 229–326).

    Google Scholar 

  • Wilks, S. S. (1962). Mathematical Staistics. New York: Wiley.

    Google Scholar 

  • Williams, C. (2002). On a connection between kernel PCA and metric multidimensional scaling. Machine Learning, 46, 11–19.

    Article  MATH  Google Scholar 

  • Wolf, L., & Shashua, A. (2001a). Affine 3-D reconstruction from two projective images of independently translating planes. In Proceedings of IEEE International Conference on Computer Vision (pp. 238–244).

    Google Scholar 

  • Wolf, L., & Shashua, A. (2001b). Two-body segmentation from two perspective views. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 263–270).

    Google Scholar 

  • Wolf, L., & Shashua, A. (2003). Learning over sets using kernel principal angles. Journal of Machine Learning Research, 4(10), 913–931.

    MathSciNet  MATH  Google Scholar 

  • Woolfe, F., & Fitzgibbon, A. (2006). Shift-invariant dynamic texture recognition. In Proceedings of European Conference on Computer Vision, pages II: 549–562.

    Google Scholar 

  • Wright, J., Ganesh, A., Kerui, M., & Ma, Y. (2013). Compressive principal component analysis. IMA Journal on Information and Inference, 2(1), 32–68.

    Article  MATH  Google Scholar 

  • Wright, J., Ganesh, A., Rao, S., Peng, Y., & Ma, Y. (2009a). Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In NIPS.

    Google Scholar 

  • Wright, J., Ma, Y., Tao, Y., Lin, Z., & Shum, H.-Y. (2009b). Classification via minimum incremental coding length (MICL). SIAM Journal on Imahing Sciences, 2(2), 367–395.

    Google Scholar 

  • Wu, J. (1983). On the convergence properties of the EM algorithm. Annals of Statistics, 11(1), 95–103.

    Article  MathSciNet  MATH  Google Scholar 

  • Wu, Y., Zhang, Z., Huang, T., & Lin, J. (2001). Multibody grouping via orthogonal subspace decomposition. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 252–257).

    Google Scholar 

  • Xiong, F., Camps, O., & Sznaier, M. (2011). Low order dynamics embedding for high dimensional time series. In IEEE International Conference on Computer Vision (pp. 2368–2374).

    Google Scholar 

  • Xiong, F., Camps, O., & Sznaier, M. (2012). Dynamic context for tracking behind occlusions. In European Conference on Computer Vision. Lecture notes in computer science (Vol. 7576, pp. 580–593). Berlin/Heidelberg: Springer.

    Google Scholar 

  • Xu, H., Caramanis, C., & Sanghavi, S. (2010). Robust pca via outlier pursuit. In Neural Information Processing Systems (NIPS).

    Google Scholar 

  • Yan, J., & Pollefeys, M. (2006). A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In European Conference on Computer Vision (pp. 94–106).

    Google Scholar 

  • Yang, A., Wright, J., Ma, Y., & Sastry, S. (2008). Unsupervised segmentation of natural images via lossy data compression. Computer Vision and Image Understanding, 110(2), 212–225.

    Article  Google Scholar 

  • Yang, A. Y., Rao, S. R., & Ma, Y. (2006). Robust statistical estimation and segmentation of multiple subspaces. In CVPR workshop on 25 years of RANSAC.

    Google Scholar 

  • Yang, J., Wright, J., Huang, T., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 19(11), 2861–2873.

    Article  MathSciNet  Google Scholar 

  • Yang, M. H., Ahuja, N., & Kriegman, D. (2000). Face detection using mixtures of linear subspaces. In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Yu, G., Sapiro, G., & Mallat, S. (2010). Image modeling and enhancement via structured sparse model selection. In International Conference on Image Processing.

    Google Scholar 

  • Yu, G., Sapiro, G., & Mallat, S. (2012). Solving inverse problems with piecewise linear estimators: From gaussian mixture models to structured sparsity. IEEE Transactions on Image Processing, 21(5), 2481–2499.

    Article  MathSciNet  Google Scholar 

  • Yu, S. (2005). Segmentation induced by scale invariance. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Yuan, L., Wen, F., Liu, C., & Shum, H. (2004). Synthesizing dynamic texture with closed-loop linear dynamic system. In European Conference on Computer Vision (pp. 603–616).

    Google Scholar 

  • Yuan, X., & Yang, J. (2009). Sparse and low-rank matrix decomposition via alternating direction methods. Preprint.

    Google Scholar 

  • Zadeh, N. (1970). A note on the cyclic coordinate ascent method. Management Science, 16, 642–644.

    Article  MathSciNet  MATH  Google Scholar 

  • Zelnik-Manor, L., & Irani, M. (2003). Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 287–293).

    Google Scholar 

  • Zhang, K., Zhang, L., & Yang, M. (2014). Fast compressive tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10).

    Google Scholar 

  • Zhang, T., Szlam, A., & Lerman, G. (2009). Median k-flats for hybrid linear modeling with many outliers. In Workshop on Subspace Methods.

    Google Scholar 

  • Zhang, T., Szlam, A., Wang, Y., & Lerman, G. (2010). Randomized hybrid linear modeling via local best-fit flats. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1927–1934).

    Google Scholar 

  • Zhang, Z., & Zha, H. (2005). Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM Journal on Scientific Computing, 26(1), 313–338.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhou, F., la Torre, F. D., & Hodgins, J. K. (2008). Aligned cluster analysis for temporal segmentation of human motion. In International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Zhou, M., Wang, C., Chen, M., Paisley, J., Dunson, D., & Carin, L. (2010a). Nonparametric bayesian matrix completion. In Sensor Array and Multichannel Signal Processing Workshop.

    Google Scholar 

  • Zhou, Z., Wright, J., Li, X., Candès, E., & Ma, Y. (2010b). Stable principal component pursuit. In International Symposium on Information Theory.

    Google Scholar 

  • Zhu, Q., Song, G., & Shi, J. (2007). Untangling cycles for contour grouping. In Proceedings of the International Conference on Computer Vision (ICCV).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag New York

About this chapter

Cite this chapter

Vidal, R., Ma, Y., Sastry, S.S. (2016). Introduction. In: Generalized Principal Component Analysis. Interdisciplinary Applied Mathematics, vol 40. Springer, New York, NY. https://doi.org/10.1007/978-0-387-87811-9_1

Download citation

Publish with us

Policies and ethics