German Conference on Pattern Recognition

Pattern Recognition pp 67-78 | Cite as

A Convex Relaxation Approach to the Affine Subspace Clustering Problem

  • Francesco Silvestri
  • Gerhard Reinelt
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)


Prototypical data clustering is known to suffer from poor initializations. Recently, a semidefinite relaxation has been proposed to overcome this issue and to enable the use of convex programming instead of ad-hoc procedures. Unfortunately, this relaxation does not extend to the more involved case where clusters are defined by parametric models, and where the computation of means has to be replaced by parametric regression. In this paper, we provide a novel convex relaxation approach to this more involved problem class that is relevant to many scenarios of unsupervised data analysis. Our approach applies, in particular, to data sets where assumptions of model recovery through sparse regularization, like the independent subspace model, do not hold. Our mathematical analysis enables to distinguish scenarios where the relaxation is tight enough and scenarios where the approach breaks down.


  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)MATHCrossRefGoogle Scholar
  2. 2.
    Berman, A., Shaked-Monderer, N.: Completely Positive Matrices. World Scientific Publishing, New York (2003)MATHCrossRefGoogle Scholar
  3. 3.
    Carin, L., Baraniuk, R., Cevher, V., Dunson, V., Jordan, M., Sapiro, G., Wakin, M.: Learning low-dimensional signal models. IEEE Signal Proc. Mag. 28(2), 39–51 (2011)CrossRefGoogle Scholar
  4. 4.
    Chen, G., Lerman, G.: Foundations of a multi-way spectral clustering framework for hybrid linear modeling. Found. Comp. Math. 9, 517–558 (2009)MATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    Dickinson, P., Gijben, L.: On the computational complexity of membership problems for the completely positive cone and its dual. Comput. Optim. Appl. 57(2), 403–415 (2014)MATHMathSciNetCrossRefGoogle Scholar
  6. 6.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Patt. Anal. Mach. Intell. 35(11), 2765–2781 (2013)CrossRefGoogle Scholar
  7. 7.
    Grossmann, I.E., Lee, S.: Generalized convex disjunctive programming: nonlinear convex hull relaxation. Comput. Optim. Appl. 26(1), 83–100 (2003)MATHMathSciNetCrossRefGoogle Scholar
  8. 8.
    Hofman, T., Buhmann, J.: Pairwise data clustering by deterministic annealing. IEEE Trans. Patt. Anal. Mach. Intell. 19(1), 1–14 (1997)CrossRefGoogle Scholar
  9. 9.
    Ma, Y., Yang, A., Derksen, H., Fossum, R.: Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Rev. 50(3), 413–458 (2008)MATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    du Merle, O., Hansen, P., Jaumard, B., Mladenović, N.: An interior points algorithm for minimum sum-of-squares clustering. SIAM J. Sci. Comput. 21(4), 1485–1505 (2000)MATHCrossRefGoogle Scholar
  11. 11.
    Peng, J., Wei, Y.: Approximating \({K}\)-means-type clustering via semidefinite programming. SIAM J. Optim. 18(1), 186–205 (2007)MATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    Rockafellar, R., Wets, R.J.B.: Variational Analysis, 2nd edn. Springer, New York (2009)MATHGoogle Scholar
  13. 13.
    Rose, K.: Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proc. IEEE 86(11), 2210–2239 (1998)CrossRefGoogle Scholar
  14. 14.
    Singh, V., Mukherjee, L., Peng, J., Xu, J.: Ensemble clustering using semidefinite programming with applications. Mach. Learn. 79(1–2), 177–200 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3 – a MATLAB software package for semidefinite programming, December 1996Google Scholar
  16. 16.
    Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95(2), 189–217 (2003)MATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    Xing, E., Jordan, M.: On Semidefinite relaxation for normalized k-cut and connections to spectral clustering. Technical report UCB/CSD-03-1265, EECS Department, University of California, Berkeley, June 2003Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Francesco Silvestri
    • 1
    • 2
  • Gerhard Reinelt
    • 1
  • Christoph Schnörr
    • 2
  1. 1.Discrete and Combinatorial Optimization GroupHeidelberg UniversityHeidelbergGermany
  2. 2.IPA and HCIHeidelberg UniversityHeidelbergGermany

Personalised recommendations