Advertisement

Exploring the Similarities of Neighboring Spatiotemporal Points for Action Pair Matching

  • Irene Kotsia
  • Ioannis Patras
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7726)

Abstract

In this paper we present a novel similarity measure between two image sequences, that is a)robust to different viewpoints and recording conditions (illumination variations and clothing) b)robust to geometric transformations (translation, scale and rotation transformations) and c)invariant to the number of frames of the image sequence as well as of its time scaling. More precisely, we create a similarity measure that exploits the underlying relationships among neighborhoods of detected spatiotemporal points in a frame of an image sequence. We find the space in which the similarities of neighboring spatiotemporal points lie in, and map it to another space of smaller dimensionality. In the new space the projected similarities are of fixed dimensionality, depending on the number of neighbors we have considered. We use the information about that newly extracted space to define a novel similarity measure between two image sequences and create in that way a similarity vector that can be used as an input to a classifier. We apply the proposed similarity measure to the ‘action pair matching’ problem, in which we try to decide whether two action image sequences contain the same action or not. Experiments conducted using the Action Similarity Labeling (ASLAN) dataset verify the superiority of the proposed method over state of the art techniques in terms of accuracy rate.

Keywords

Image Sequence Action Recognition Area Under Curve Geometric Transformation Linear Support Vector Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belongie, S., Malik, J., Puzicha, J.: Shape context: A new descriptor for shape matching and object recognition. In: Proceedings of Advances in Neural Information Processing Systems (2001)Google Scholar
  2. 2.
    Zhang, H., Berg, A.C., Maire, M., Malik, J.: Svm-knn: Discriminative nearest neighbor classification for visual category recognition. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR (2006)Google Scholar
  3. 3.
    Yang, L., Jin, R.: Distance metric learning: A comprehensive survey. Michigan State Universiy (2006)Google Scholar
  4. 4.
    Bar-Hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning distance functions using equivalence relations. In: International Conference on Machine Learning, ICML (2003)Google Scholar
  5. 5.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighborhood components analysis. In: Advances in Neural Information Processing Systems, NIPS (2005)Google Scholar
  6. 6.
    Xing, E., Ng, A., Jordan, M., Russell, S.: Distance metric learning with application to clustering with side-information. In: Advances in Neural Information Processing Systems, NIPS (2002)Google Scholar
  7. 7.
    Weinberger, K., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbor classification. In: Advances in Neural Information Processing Systems, NIPS (2006)Google Scholar
  8. 8.
    Davis, J., Kulis, B., Jain, P., Sra, S., Dhillon, I.: Information-theoretic metric learning. In: International Conference on Machine Learning, ICML (2007)Google Scholar
  9. 9.
    Chechik, G., Sharma, V., Shalit, U., Bengio, S.: Large scale online learning of image similarity through ranking. The Journal of Machine Learning Research, JMLR (2010)Google Scholar
  10. 10.
    Nguyen, H.V., Bai, L.: Cosine Similarity Metric Learning for Face Verification. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part II. LNCS, vol. 6493, pp. 709–720. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  11. 11.
    Gilbert, A., Illingworth, J., Bowden, R.: Action Recognition Using Mined Hierarchical Compound Features. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 883–897 (2011)CrossRefGoogle Scholar
  12. 12.
    Yuan, J., Wu, Y.: Mining Visual Collocation Patterns via Self-Supervised Subspace Learning. IEEE Transactions on Systems, Man and Cybernetics Part B: Cybernetics 42, 334–346 (2012)CrossRefGoogle Scholar
  13. 13.
    Kliper-Gross, O., Hassner, T., Wolf, L.: One Shot Similarity Metric Learning for Action Recognition. In: Pelillo, M., Hancock, E.R. (eds.) SIMBAD 2011. LNCS, vol. 7005, pp. 31–45. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Wang, H., Klaser, A., Schmid, C., Liu, C.L.: Action recognition by dense trajectories. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  15. 15.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision Pattern Recognition (2008)Google Scholar
  16. 16.
    Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  17. 17.
    Kliper-Gross, O., Hassner, T., Wolf, L.: The action similarity labeling challenge. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 615–621 (2012)CrossRefGoogle Scholar
  18. 18.
    Wolf, L., Hassner, T., Taigman, Y.: Effective unconstrained face recognition by combining multiple descriptors and learned background statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 1978–1990 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Irene Kotsia
    • 1
  • Ioannis Patras
    • 1
  1. 1.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonUK

Personalised recommendations