Motion Segmentation by Model-Based Clustering of Incomplete Trajectories

  • Vasileios Karavasilis
  • Konstantinos Blekas
  • Christophoros Nikou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6912)


In this paper, we present a framework for visual object tracking based on clustering trajectories of image key points extracted from a video. The main contribution of our method is that the trajectories are automatically extracted from the video sequence and they are provided directly to a model-based clustering approach. In most other methodologies, the latter constitutes a difficult part since the resulting feature trajectories have a short duration, as the key points disappear and reappear due to occlusion, illumination, viewpoint changes and noise. We present here a sparse, translation invariant regression mixture model for clustering trajectories of variable length. The overall scheme is converted into a Maximum A Posteriori approach, where the Expectation-Maximization (EM) algorithm is used for estimating the model parameters. The proposed method detects the different objects in the input image sequence by assigning each trajectory to a cluster, and simultaneously provides the motion of all objects. Numerical results demonstrate the ability of the proposed method to offer more accurate and robust solution in comparison with the mean shift tracker, especially in cases of occlusions.


Motion segmentation visual feature tracking trajectory clustering sparse regression 


  1. 1.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(5), 564–577 (2003)CrossRefGoogle Scholar
  2. 2.
    Cuevas, E., Zaldivar, D., Rojas, R.: Kalman filter for vision tracking. Technical Report B 05-12, Freier Universitat Berlin, Institut fur Informatik (2005)Google Scholar
  3. 3.
    Yang, M., Wu, Y., Hua, G.: Context-aware visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(7), 1195–1209 (2009)CrossRefGoogle Scholar
  4. 4.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. International Journal of Computer Vision 29, 5–28 (1998)CrossRefGoogle Scholar
  5. 5.
    Shi, J., Tomasi, C.: Good features to track. In: 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 1994), pp. 593–600 (1994)Google Scholar
  6. 6.
    Wang, Z., Yang, X., Xu, Y., Yu, S.: Camshift guided particle filter for visual tracking. Pattern Recognition Letters 30(4), 407–413 (2009)CrossRefGoogle Scholar
  7. 7.
    Zhoo, H., Yuan, Y., Shi, C.: Object tracking using SIFT features and mean shift. Computer Vision and Image Understanding 113(3), 345–352 (2009)CrossRefGoogle Scholar
  8. 8.
    Zhao, Q., Tao, H.: Differential Earth Mover’s Distance with its application to visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 274–287 (2010)CrossRefGoogle Scholar
  9. 9.
    Karavasilis, V., Nikou, C., Likas, A.: Visual tracking using the earth mover’s distance between Gaussian mixtures and Kalman filtering. Image and Vision Computing 29(5), 295–305 (2011)CrossRefGoogle Scholar
  10. 10.
    Black, M., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  11. 11.
    Wong, K.Y., Spetsakis, M.E.: Motion segmentation by EM clustering of good features. In: Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 2004), vol. 11, pp. 1–8. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  12. 12.
    Pundlik, S.J., Birchfield, S.T.: Real-time motion segmentation of sparse feature points at any speed. IEEE Transactions on Systems, Man, and Cybernetics 38, 731–742 (2008)CrossRefGoogle Scholar
  13. 13.
    Buzan, D., Sclaroff, S., Kollios, G.: Extraction and clustering of motion trajectories in video. In: International Conference on Pattern Recognition, pp. 521–524 (2004)Google Scholar
  14. 14.
    Alon, J., Sclaroff, S., Kollios, G., Pavlovic, V.: Discovering clusters in motion time-series data. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 375–381 (2003)Google Scholar
  15. 15.
    Zappella, L., Llado, X., Provenzi, E., Salvi, J.: Enhanced local subspace affinity for feature-based motion segmentation. Pattern Recognition 44, 454–470 (2011)CrossRefGoogle Scholar
  16. 16.
    Zeppelzauer, M., Zaharieva, M., Mitrovic, D., Breiteneder, C.: A novel trajectory clustering approach for motion segmentation. In: Boll, S., Tian, Q., Zhang, L., Zhang, Z., Chen, Y.-P.P. (eds.) MMM 2010. LNCS, vol. 5916, pp. 433–443. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Lauer, F., Schnorr, C.: Spectral clustering of linear subspaces for motion segmentation. In: Proc. of the 12th IEEE Int. Conf. on Computer Vision, ICCV 2009 (2009)Google Scholar
  18. 18.
    Julià, C., Sappa, A., Lumbreras, F., Serrat, J., López, A.: Motion segmentation from feature trajectories with missing data. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4477, pp. 483–490. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  19. 19.
    Yan, J., Pollefeys, M.: A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  20. 20.
    Kim, J.H., Agapito, L.: Motion segmentation using the Hadamard product and spectral clustering. In: Proceedings of the 2009 International Conference on Motion and Video Computing, WMVC 2009, pp. 126–133. IEEE Computer Society, Los Alamitos (2009)Google Scholar
  21. 21.
    Masoud, O., Papanikolopoulos, N.P.: A novel method for tracking and counting pedestrians in real-time using a single camera. IEEE Transactions on Vehicular Technology 50(5), 1267–1278 (2001)CrossRefGoogle Scholar
  22. 22.
    Harris, C., Stephens, M.: A combined corner and edge detection. In: Proceedings of The Fourth Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  23. 23.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI 1981), pp. 674–679 (1981)Google Scholar
  24. 24.
    Lowe, D.G.: Distinctive image features from Scale-Invariant keypoint. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  25. 25.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A comparison of affine region detectors. International Journal of Computer Vision 65(1/2), 43–72 (2005)CrossRefGoogle Scholar
  26. 26.
    Gaffney, S.: Probabilistic curve-aligned clustering and prediction with regression mixture models. PhD thesis, Department of Computer Science, University of California, Irvine (2004)Google Scholar
  27. 27.
    Tipping, M.: Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research 1, 211–244 (2001)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. J. Roy. Statist. Soc. B 39, 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Blekas, K., Nikou, C., Galatsanos, N., Tsekos, N.: A regression mixture model with spatial constraints for clustering spatiotemporal data. Intern. Journal on Artificial Intelligence Tools 17(5), 1023–1041 (2008)CrossRefGoogle Scholar
  30. 30.
    Oikonomopoulos, A., Patras, I., Pantic, M., Paragios, N.: Trajectory-based representation of human actions. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) ICMI/IJCAI Workshops 2007. LNCS (LNAI), vol. 4451, pp. 133–154. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Vasileios Karavasilis
    • 1
  • Konstantinos Blekas
    • 1
  • Christophoros Nikou
    • 1
  1. 1.Department of Computer ScienceUniversity of IoanninaIoanninaGreece

Personalised recommendations