Advertisement

A Convex Relaxation Approach to the Affine Subspace Clustering Problem

  • Francesco Silvestri
  • Gerhard Reinelt
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)

Abstract

Prototypical data clustering is known to suffer from poor initializations. Recently, a semidefinite relaxation has been proposed to overcome this issue and to enable the use of convex programming instead of ad-hoc procedures. Unfortunately, this relaxation does not extend to the more involved case where clusters are defined by parametric models, and where the computation of means has to be replaced by parametric regression. In this paper, we provide a novel convex relaxation approach to this more involved problem class that is relevant to many scenarios of unsupervised data analysis. Our approach applies, in particular, to data sets where assumptions of model recovery through sparse regularization, like the independent subspace model, do not hold. Our mathematical analysis enables to distinguish scenarios where the relaxation is tight enough and scenarios where the approach breaks down.

Notes

Acknowledgement

Authors gratefully acknowledge support by the DFG, grant GRK 1653.

References

  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)CrossRefGoogle Scholar
  2. 2.
    Berman, A., Shaked-Monderer, N.: Completely Positive Matrices. World Scientific Publishing, New York (2003)CrossRefGoogle Scholar
  3. 3.
    Carin, L., Baraniuk, R., Cevher, V., Dunson, V., Jordan, M., Sapiro, G., Wakin, M.: Learning low-dimensional signal models. IEEE Signal Proc. Mag. 28(2), 39–51 (2011)CrossRefGoogle Scholar
  4. 4.
    Chen, G., Lerman, G.: Foundations of a multi-way spectral clustering framework for hybrid linear modeling. Found. Comp. Math. 9, 517–558 (2009)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Dickinson, P., Gijben, L.: On the computational complexity of membership problems for the completely positive cone and its dual. Comput. Optim. Appl. 57(2), 403–415 (2014)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Patt. Anal. Mach. Intell. 35(11), 2765–2781 (2013)CrossRefGoogle Scholar
  7. 7.
    Grossmann, I.E., Lee, S.: Generalized convex disjunctive programming: nonlinear convex hull relaxation. Comput. Optim. Appl. 26(1), 83–100 (2003)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Hofman, T., Buhmann, J.: Pairwise data clustering by deterministic annealing. IEEE Trans. Patt. Anal. Mach. Intell. 19(1), 1–14 (1997)CrossRefGoogle Scholar
  9. 9.
    Ma, Y., Yang, A., Derksen, H., Fossum, R.: Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Rev. 50(3), 413–458 (2008)MathSciNetCrossRefGoogle Scholar
  10. 10.
    du Merle, O., Hansen, P., Jaumard, B., Mladenović, N.: An interior points algorithm for minimum sum-of-squares clustering. SIAM J. Sci. Comput. 21(4), 1485–1505 (2000)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Peng, J., Wei, Y.: Approximating \({K}\)-means-type clustering via semidefinite programming. SIAM J. Optim. 18(1), 186–205 (2007)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Rockafellar, R., Wets, R.J.B.: Variational Analysis, 2nd edn. Springer, New York (2009)zbMATHGoogle Scholar
  13. 13.
    Rose, K.: Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proc. IEEE 86(11), 2210–2239 (1998)CrossRefGoogle Scholar
  14. 14.
    Singh, V., Mukherjee, L., Peng, J., Xu, J.: Ensemble clustering using semidefinite programming with applications. Mach. Learn. 79(1–2), 177–200 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3 – a MATLAB software package for semidefinite programming, December 1996Google Scholar
  16. 16.
    Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95(2), 189–217 (2003)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Xing, E., Jordan, M.: On Semidefinite relaxation for normalized k-cut and connections to spectral clustering. Technical report UCB/CSD-03-1265, EECS Department, University of California, Berkeley, June 2003Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (http://creativecommons.org/licenses/by-nc/2.5/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Francesco Silvestri
    • 1
    • 2
  • Gerhard Reinelt
    • 1
  • Christoph Schnörr
    • 2
  1. 1.Discrete and Combinatorial Optimization GroupHeidelberg UniversityHeidelbergGermany
  2. 2.IPA and HCIHeidelberg UniversityHeidelbergGermany

Personalised recommendations