Skip to main content

Graph-Based Multimodal Clustering for Social Event Detection in Large Collections of Images

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNISA,volume 8325)

Abstract

A common approach to the problem of SED in collections of multimedia relies on the use of clustering methods. Due to the heterogeneity of features associated with multimedia items in such collections, such a clustering task is very challenging and special multimodal clustering approaches need to be deployed. In this paper, we present a scalable graph-based multimodal clustering approach for SED in large collections of multimedia. The proposed approach utilizes example relevant clusterings to learn a model of the “same event” relationship between two items in the multimodal domain and subsequently to organize the items in a graph. Two variants of the approach are presented: the first based on a batch and the second on an incremental community detection algorithm. Experimental results indicate that both variants provide excellent clustering performance.

Keywords

  • Social media
  • Social event detection
  • Multimodal clustering

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-04114-8_13
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-04114-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   129.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-Up Robust Features (SURF). Comp. Vis. Image Underst. 110(3), 346–359 (2008)

    CrossRef  Google Scholar 

  2. Becker, H., Naaman, M., Gravano, L.: Learning similarity metrics for event identification in social media. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, WSDM 2010, pp. 291–300. ACM, New York (2010)

    Google Scholar 

  3. Bekkerman, R., Jeon, J.: Multi-modal clustering for multimedia collections. In: CVPR (2007)

    Google Scholar 

  4. Brenner, M., Izquierdo, E.: Mediaeval benchmark: Social Event Detection in collaborative photo collections. In: MediaEval. CEUR Workshop Proceedings (2011)

    Google Scholar 

  5. Cai, X., Nie, F., Huang, H., Kamangar, F.: Heterogeneous image feature integration via multi-modal spectral clustering. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1977–1984 (June 2011)

    Google Scholar 

  6. Fortunato, S.: Community detection in graphs. Physics Reports 486(3-5), 75–174 (2010)

    CrossRef  MathSciNet  Google Scholar 

  7. Goder, A., Filkov, V.: Consensus clustering algorithms: Comparison and refinement. In: Ian Munro, J. (ed.) Proceedings of the Workshop on Algorithm Engineering and Experiments, ALENEX 2008, San Francisco, California, USA, pp. 109–117. SIAM (January 19, 2008)

    Google Scholar 

  8. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explorations Newsletter 11(1), 10–18 (2009)

    CrossRef  Google Scholar 

  9. Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(1), 117–128 (2011)

    CrossRef  Google Scholar 

  10. Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: 23rd IEEE Conference on Computer Vision & Pattern Recognition, CVPR 2010, pp. 3304–3311. IEEE Computer Society, San Francisco (2010)

    CrossRef  Google Scholar 

  11. Khalidov, V., Forbes, F., Horaud, R.P.: Conjugate mixture models for clustering multimodal data. Neural Computation 23(2), 517–557 (2011)

    CrossRef  MATH  MathSciNet  Google Scholar 

  12. Li, Y., Crandall, D.J., Huttenlocher, D.P.: Landmark classification in large-scale image collections. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, September 27 - October 4, pp. 1957–1964. IEEE (2009)

    Google Scholar 

  13. Liu, X., Troncy, R., Huet, B.: Using social media to identify events. In: ACM Multimedia 3rd Workshop on Social Media, WSM 2011, Scottsdale, Arizona, USA, November 18-December 1, p. 11 (2011)

    Google Scholar 

  14. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)

    CrossRef  MATH  Google Scholar 

  15. Nguyen, N.P., Dinh, T.N., Xuan, Y., Thai, M.T.: Adaptive algorithms for detecting community structure in dynamic social networks. In: 30th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2011, Shanghai, China, April 10-15, pp. 2282–2290. IEEE (2011)

    Google Scholar 

  16. Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vision 42(3), 145–175 (2001)

    CrossRef  MATH  Google Scholar 

  17. Papadopoulos, S., Kompatsiaris, Y., Vakali, A., Spyridonos, P.: Community detection in social media. Data Mining and Knowledge Discovery 24(3), 515–554 (2012)

    CrossRef  Google Scholar 

  18. Papadopoulos, S., Schinas, E., Mezaris, V., Troncy, R., Kompatsiaris, I.: The 2012 Social Event Detection dataset. In: 4th ACM Multimedia Systems, Dataset Session, MMSys 2013, Oslo, Norway, February 27-March 1 (2013)

    Google Scholar 

  19. Papadopoulos, S., Schinas, E., Mezaris, V., Troncy, R., Kompatsiaris, Y.: Social Event Detection at MediaEval 2012: Challenges, Dataset and Evaluation. In: MediaEval 2012 Workshop, Pisa, Italy, October 4-5 (2012)

    Google Scholar 

  20. Papadopoulos, S., Troncy, R., Mezaris, V., Huet, B., Kompatsiaris, I.: Social Event Detection at Mediaeval 2011: Challenges, dataset and evaluation. In: MediaEval. CEUR Workshop Proceedings (2011)

    Google Scholar 

  21. Papadopoulos, S., Zigkolis, C., Kompatsiaris, Y., Vakali, A.: CERTH@Mediaeval 2011 social event detection task. In: MediaEval. CEUR Workshop Proceedings (2011)

    Google Scholar 

  22. Papadopoulos, S., Zigkolis, C., Kompatsiaris, Y., Vakali, A.: Cluster-based landmark and event detection for tagged photo collections. IEEE Multimedia 18(1), 52–63 (2011)

    CrossRef  Google Scholar 

  23. Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR 2012, pp. 23:1–23:8. ACM, New York (2012)

    Google Scholar 

  24. Phuvipadawat, S., Murata, T.: Breaking news detection and tracking in twitter. In: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, vol. 3, pp. 120–123 (2010)

    Google Scholar 

  25. Rendle, S., Schmidt-Thieme, L.: Scaling record linkage to non-uniform distributed class sizes. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 308–319. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  26. Reuter, T., Cimiano, P.: Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR 2012, pp. 22:1–22:8. ACM, New York (2012)

    Google Scholar 

  27. Schinas, E., Mantziou, E., Papadopoulos, S., Petkos, G., Kompatsiaris, Y.: CERTH @ Mediaeval 2013 Social Event Detection Task. In: MediaEval. CEUR Workshop Proceedings (2013)

    Google Scholar 

  28. Schinas, E., Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: CERTH @ Mediaeval 2012 Social Event Detection Task. In: MediaEval. CEUR Workshop Proceedings, vol. 927 (2012)

    Google Scholar 

  29. Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, MULTIMEDIA 2005, pp. 399–402. ACM, New York (2005)

    Google Scholar 

  30. Xu, X., Yuruk, N., Feng, Z., Schweiger, T.A.J.: Scan: a structural clustering algorithm for networks. In: Proceedings of the 13th ACM SIGKDD, KDD 2007, pp. 824–833. ACM, NY (2007)

    Google Scholar 

  31. Ye, Z., Hu, S., Yu, J.: Adaptive clustering algorithm for community detection in complex networks. Physical Review E 78(4) (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Petkos, G., Papadopoulos, S., Schinas, E., Kompatsiaris, Y. (2014). Graph-Based Multimodal Clustering for Social Event Detection in Large Collections of Images. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds) MultiMedia Modeling. MMM 2014. Lecture Notes in Computer Science, vol 8325. Springer, Cham. https://doi.org/10.1007/978-3-319-04114-8_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-04114-8_13

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-04113-1

  • Online ISBN: 978-3-319-04114-8

  • eBook Packages: Computer ScienceComputer Science (R0)