Skip to main content

Dual graph regularized NMF model for social event detection from Flickr data

World Wide Web volume 20, pages 995–1015 (2017)Cite this article

Abstract

In this work, we aim to discover real-world events from Flickr data by devising a three-stage event detection framework. In the first stage, a multimodal fusion (MF) model is designed to deal with the heterogeneous feature modalities possessed by the user-shared data, which is advantageous in computation complexity. In the second stage, a dual graph regularized non-negative matrix factorization (DGNMF) model is proposed to learn compact feature representations. DGNMF incorporates Laplacian regularization terms for the data graph and base graph into the objective, keeping the geometry structures underlying the data samples and dictionary bases simultaneously. In the third stage, hybrid clustering algorithms are applied seamlessly to discover event clusters. Extensive experiments conducted on the real-world dataset reveal the MF-DGNMF-based approaches outperform the baselines.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

References

  1. Ahsan, U., Essa, I.: Clustering social event images using kernel canonical correlation analysis. In: Computer Vision and Pattern Recognition Workshops, 2014 IEEE Conference on, pp. 814–819 (2014)

  2. Ah-Pine, J., Csurka, G., Clinchant, S.: Semi-supervised visual and textual information fusion in CBMIR using graph-based methods. ACM Trans. Inf. Syst. (TOIS) 33(2), 9 (2015)

    Article  Google Scholar 

  3. Cai, X., Nie, F., Huang, H., Kamangar, F.: Heterogeneous image feature integration via multi-modal spectral clustering. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 1977–1984 (2011)

  4. Cai, D., He, X., Han, J., Huang, T.S.: Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1548–1560 (2011)

    Article  Google Scholar 

  5. Cands, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 11 (2011)

    MathSciNet  MATH  Google Scholar 

  6. Chen, L., Roy, A.: Event detection from flickr data through wavelet-based spatial analysis. In: Proceedings of the 18th ACM conference on Information and knowledge management, pp. 523–532 (2009)

  7. Chen, J., Cui, Y., Ye, G., Liu, D., Chang, S.F.: Event-driven semantic concept discovery by exploiting weakly tagged internet images. In: Proceedings of International Conference on Multimedia Retrieval, p. 1 (2014)

  8. Choi, J., Kim, E., Larson, M., Friedland, G., Hanjalic, A.: Evento 360: Social event discovery from Web-scale multimedia collection (2015)

  9. Duan, K., Crandall, D.J., Batra, D.: Multimodal learning in loosely-organized Web images. In: Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 2465–2472 (2014)

  10. Elhamifar, E., Vidal, R.: Sparse subspace clustering: Algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  11. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  12. Hinton, G.: A practical guide to training restricted Boltzmann machines. Momentum 9(1), 926 (2010)

    Google Scholar 

  13. Hyvrinen, A., Karhunen, J., Oja, E.: Independent component analysis (Vol. 46) John Wiley & Sons (2004)

  14. Jiang, X., Lai, J.: Sparse and dense hybrid representation via dictionary decomposition for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(5), 1067–1079 (2015)

    Article  Google Scholar 

  15. Jolliffe, I.: Principal component analysis. John Wiley & Sons Ltd (2002)

  16. Kim, G., Sigal, S.M.L.: Joint photo stream and blog post summarization and exploration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3081–3089 (2015)

  17. Lee, D.D., Sebastian Seung, H.: Algorithms for non-negative matrix factorization. Advances in neural information processing systems, 556–562 (2001)

  18. Li, R., Lei, K.H., Khadiwala, R., Chang, K.: Tedas: A twitter-based event detection and analysis system. In: Data engineering (ICDE), 2012 IEEE 28th international conference on, pp. 1273–1276 (2001)

  19. Li, Z., Liu, J., Tang, J., Lu, H.: Robust structured subspace learning for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2085–2098 (2015)

    Article  Google Scholar 

  20. Liu, X., Huet, B.: Heterogeneous features and model selection for event-based media classification. In: 3rd ACM International Conference on Multimedia Retrieval, pp. 151–158 (2013)

  21. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)

    Article  Google Scholar 

  22. Liu, G., Xu, H., Tang, J., Liu, Q., Yan, S.: A deterministic analysis for LRR. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 417–430 (2015)

    Article  Google Scholar 

  23. Nitta, N., Kumihashi, Y., Kato, T., Babaguchi, N.: Real-World Event detection using flickr images. In: MultiMedia Modeling, pp. 307–314 (2014)

  24. Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, p. 23 (2012)

  25. Petkos, G., Papadopoulos, S., Mezaris, V., Kompatsiaris, Y.: Social event detection at MediaEval 2014: Challenges, datasets, and evaluation. In: MediaEval 2014 Workshop, Barcelona, Spain (2014)

  26. Qian, S., Zhang, T., Xu, C., Hossain, M.S.: Social event classification via boosted multimodal supervised latent dirichlet allocation. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 11(2), 27 (2014)

    Google Scholar 

  27. Sakaki, T., Okazaki, M., Matsuo, Y.: Earthquake shakes Twitter users: real-time event detection by social sensors. In: Proceedings of the 19th international conference on World wide web, pp. 851–860 (2010)

  28. Sakaki, T., Okazaki, M., Matsuo, Y.: Tweet analysis for real-time event detection and earthquake reporting system development. IEEE Trans. Knowl. Data Eng. 25(4), 919–931 (2013)

    Article  Google Scholar 

  29. Schinas, M., Papadopoulos, S., Petkos, G., Kompatsiaris, Y., Mitkas, P.A.: Multimodal graph-based event detection and summarization in social media streams. In: Proceedings of the 23rd Annual ACM Conference on Multimedia Conference, pp. 189–192 (2015)

  30. Shekhar, S., Patel, V.M., Nasrabadi, N.M., Chellappa, R.: Joint sparse representation for robust multimodal biometrics recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 113–126 (2014)

    Article  Google Scholar 

  31. Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th annual ACM international conference on Multimedia, pp. 399–402 (2005)

  32. Strehl, A., Ghosh, J.: Cluster ensembles-a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2003)

    MathSciNet  MATH  Google Scholar 

  33. Sutanto, T., Nayak, R.: The ranking based constrained document clustering method and its application to social event detection. In: Database Systems for Advanced Applications, pp. 47–60 (2014)

  34. Sutanto, T., Nayak, R.: Ranking based clustering for social event detection. In: Working Notes Proceedings of the MediaEval 2014 Workshop, vol. 1263, pp. 1–2 (2014)

  35. Wang, Y., Sundaram, H., Xie, L.: Social event detection with interaction graph modeling. In: Proceedings of the 20th ACM international conference on Multimedia, pp. 865–868 (2012)

  36. Wu, F., Yu, Z., Yang, Y., Tang, S., Zhang, Y., Zhuang, Y.: Sparse multi-modal hashing. IEEE Trans. Multimedia 16(2), 427–439 (2014)

    Article  Google Scholar 

  37. Yang, Z., Li, Q., Lu, Z., Ma, Y., Gong, Z., Pan, H.: Semi-supervised multimodal clustering algorithm integrating label signals for social event detection. In: Multimedia Big Data (BigMM), 2015 IEEE International Conference on, pp. 32–39 (2015)

  38. Yang, Z., Li, Q., Lu, Z., Ma, Y., Gong, Z., Pan, H., Chen, Y.: Semi-Supervised Multimodal fusion model for social event detection on web image collections. Int. J. Multimedia Data Eng. Manag. (IJMDEM) 6(4), 1–22 (2015)

    Article  Google Scholar 

  39. Yang, X., Zhang, T., Xu, C., Hossain, M.S.: Automatic visual concept learning for social event understanding. IEEE Trans. Multimedia 17(3), 346–358 (2015)

    Article  Google Scholar 

  40. Yang, Z., Li, Q., Liu, W., Ma, Y.: Learning manifold representation from multimodal data for event detection in Flickr-like social media, The 3rd International Workshop on Semantic Computing and Personalization in conjunction with The 21th International Conference on Database Systems for Advanced Applications, 160–167 (2016)

  41. Yin, M., Gao, J., Lin, Z.: Laplacian regularized low-rank representation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 504–517 (2015)

    Article  Google Scholar 

  42. Zhang, Z., Zhao, K.: Low-rank matrix approximation with manifold regularization. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1717–1729 (2015)

    Article  Google Scholar 

  43. Zhang, W., Zeng, S., Wang, D., Xue, X.: Weakly supervised semantic segmentation for social images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2718–2726 (2015)

  44. Zhou, X., Chen, L.: Event detection over twitter social media streams. VLDB J. 23(3), 381–400 (2014)

    MathSciNet  Article  Google Scholar 

  45. Zhuang, L., Gao, S., Tang, J., Wang, J., Lin, Z., Ma, Y.: Constructing a non-Negative low rank and sparse graph with data-adaptive features. IEEE Trans. Image Process. 24(11), 3717–3728 (2015)

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Dr. Zheng Lu and Mr. Yangbin Chen for all the discussions. The research described in this paper has been supported by a National Natural Science Foundation of China (Project no. 61472337).

Author information

Authors and Affiliations

  1. Department of Computer Science, City University of Hong Kong, Hong Kong, China

    Zhenguo Yang, Qing Li, Yun Ma & Min Cheng

  2. Multimedia-software Engineering Research Centre, City University of Hong Kong, Hong Kong, China

    Qing Li

  3. School of Computer Science and Technology, Guangdong University of Technology, Guangdong, China

    Wenyin Liu

Authors
  1. Zhenguo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenguo Yang.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 251 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., Li, Q., Liu, W. et al. Dual graph regularized NMF model for social event detection from Flickr data. World Wide Web 20, 995–1015 (2017). https://doi.org/10.1007/s11280-016-0405-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-016-0405-1

Keywords

  • Social media analytics
  • Multimedia content analysis
  • Multimodal fusion
  • Data representation learning
  • Event detection