Abstract
We present an online algorithm for the efficient clustering of data drawn from a union of arbitrary dimensional, non-static subspaces. Our algorithm is based on an online min-Mahalanobis distance classifier, which simultaneously clusters and is updated from subspace data. In contrast to most existing methods, our algorithm can cope with large amounts of batch or sequential data and is temporally consistent when dealing with time varying data (i.e. time-series). Starting from an initial condition, the classifier provides a first estimate of the subspace clusters in the current time-window. From this estimate, we update the classifier using stochastic gradient descent. The updated classifier is applied back onto the data to refine the subspace clusters, while at the same time we recover the explicit rotations that align the subspaces between time- windows. The whole procedure is repeated until convergence, resulting in a fast, efficient and accurate algorithm. We have tested our algorithm on synthetic and three real datasets and compared with competing methods from literature. Our results show that our algorithm outperforms the competition with superior clustering accuracy and computation speed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Vidal, R.: Subspace clustering. IEEE Signal Process. Mag. 28, 52–68 (2011)
Jolliffe, I.: Principal Component Analysis. Springer, New York (1986)
Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data. Data Min. Knowl. Disc. 11, 5–33 (2005)
Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: CVPR (2009)
Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML (2010)
Zografos, V., Ellis, L., Mester, R.: Discriminative subspace clustering. In: CVPR (2013)
Vidal, R., Favaro, P.: Low rank subspace clustering (LRSC). Pattern Recogn. Lett. 43, 47–61 (2014)
Patel, V.M., Nguyen, H.V., Vidal, R.: Latent space sparse and low-rank subspace clustering. IEEE J. Sel. Top. Sign. Proces. 9, 691–701 (2015)
Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: NIPS (2001)
Wang, S., Fan, Y., Zhang, C., Xu, H., Hao, X., Hu, Y.: Subspace clustering of high dimensional data streams. In: 2008 Seventh IEEE/ACIS International Conference on Computer and Information Science, ICIS 2008, pp. 165–170 (2008)
Park, N.H., Lee, W.S.: Memory efficient subspace clustering for online data streams. In: Proceedings of the 2008 International Symposium on Database Engineering and Applications, IDEAS 2008, pp. 199–208 (2008)
Peng, X., Zhang, L., Yi, Z.: Scalable sparse subspace clustering. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 430–437 (2013)
Peng, X., Zhang, L., Yi, Z.: Inductive sparse subspace clustering. Electron. Lett. 49, 1222–1224 (2013)
Vidal, R.: Online clustering of moving hyperplanes. In: Advances in Neural Information Processing Systems 19, pp. 1433–1440. MIT Press (2006)
Wang, J., Fu, Z.: Online motion segmentation based on sparse subspace clustering. J. Inf. Comput. Sci. 12, 1293–1300 (2015)
Arandjelovic, O., Cipolla, R.: Incremental learning of temporally-coherent gaussian mixture models. In: BMVC (2005)
Engel, P.M., Heinen, M.R.: Incremental learning of multivariate gaussian mixture models. In: Rocha Costa, A.C., Vicari, R.M., Tonidandel, F. (eds.) SBIA 2010. LNCS (LNAI), vol. 6404, pp. 82–91. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16138-4_9
Declercq, A., Piater, J.: Online learning of gaussian mixture models - a two-level approach. In: VISAPP (2008)
Bottou, L., Bengio, Y.: Convergence properties of the K-Means algorithm. In: Advances in Neural Information Processing Systems (1995)
Bradley, S., Mangasarian, O.: K-plane clustering. J. Global Optim. 16(1), 23–32 (2000)
Tseng, P.: Nearest q-flat to M points. J. Optim. Theory Appl. 105, 249–252 (2000)
Wang, D., Ding, C., Li, T.: K-subspace clustering. Mach. Learn. Knowl. Disc. Databases 5782, 506–521 (2009)
Bottou, L.: Online algorithms and stochastic approximations. In: Online Learning and Neural Networks. Cambridge University Press (1998)
Sherman, J., Morrison, W.J.: Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Stat. 21, 124–127 (1950)
Chen, G., Lerman, G.: Spectral curvature clustering (SCC). IJCV 81, 317–330 (2009)
Tron, P., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: CVPR (2007)
Vidal, R., Ma, Y., Sastry, S.: Generalized principal component analysis (GPCA). IEEE PAMI 27, 1945–1959 (2005)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, vol. 86 (1998)
Péteri, R., Fazekas, S., Huiskes, M.J.: Dyntex: a comprehensive database of dynamic textures. Pattern Recogn. Lett. 31, 1627–1632 (2010)
Acknowledgements
This research was supported by the TU Munchen - IAS (funded by the German Excellence Initiative and the EU 7th Framework Programme under grant agreement no. 291763, the Marie Curie COFUND program of the EU).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Zografos, V., Krajsek, K., Menze, B. (2017). An Online Algorithm for Efficient and Temporally Consistent Subspace Clustering. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10111. Springer, Cham. https://doi.org/10.1007/978-3-319-54181-5_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-54181-5_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54180-8
Online ISBN: 978-3-319-54181-5
eBook Packages: Computer ScienceComputer Science (R0)