Semantic Entity-Relationship Model for Large-Scale Multimedia News Exploration and Recommendation

  • Hangzai Luo
  • Peng Cai
  • Wei Gong
  • Jianping Fan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5916)


Even though current news websites use large amount of multimedia materials including image, video and audio, the multimedia materials are used as supplementary to the traditional text-based framework. As users always prefer multimedia, the traditional text-based news exploration interface receives more and more criticisms from both journalists and general audiences. To resolve this problem, we propose a novel framework for multimedia news exploration and analysis. The proposed framework adopts our semantic entity-relationship model to model the multimedia semantics. The proposed semantic entity-relationship model has three nice properties. First, it is able to model multimedia semantics with visual, audio and text properties in a uniform framework. Second, it can be extracted via existing semantic analysis and machine learning algorithms. Third, it is easy to implement sophisticated information mining and visualization algorithms based on the model. Based on this model, we implemented a novel multimedia news exploration and analysis system by integrating visual analytics and information mining techniques. Our system not only provides higher efficiency on news exploration and retrieval but also reveals extra interesting information that is not available on traditional news exploration systems.


News Report Video Shot Informative Image Semantic Entity Multimedia Material 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wagstaff, J.: On news visualization (2005),
  2. 2.
    Harris, J.: Tenbyten,
  3. 3.
  4. 4.
  5. 5.
    Gastner, M., Shalizi, C., Newman, M.: Maps and cartograms of the 2004 us presidential election results (2004),
  6. 6.
    Yang, J., Hauptmann, A.G.: Annotating news video with locations. In: Sundaram, H., Naphade, M., Smith, J.R., Rui, Y. (eds.) CIVR 2006. LNCS, vol. 4071, pp. 153–162. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Borgne, H.L., Guerin-Dugue, A., O’Connor, N.: Learning midlevel image features for natural scene and texture classification. IEEE Trans. on CSVT 17(3), 286–297 (2007)Google Scholar
  8. 8.
    Chang, S., Manmatha, R., Chua, T.: Combining text and audio-visual features in video indexing. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Philadelphia, pp. 1005–1008 (2005)Google Scholar
  9. 9.
    Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation 28, 11–21 (1972)CrossRefGoogle Scholar
  10. 10.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: Bringing order to the web,
  11. 11.
    Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46, 668–677 (1999)CrossRefGoogle Scholar
  12. 12.
    Hetzler, E.G., Whitney, P., Martucci, L., Thomas, J.: Multi-faceted insight through interoperable visual information analysis paradigms. In: IEEE Symposium on Information Visualization, p. 137 (1998)Google Scholar
  13. 13.
    McCallum, A., Li, W.: Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Natural language learning at HLT-NAACL, pp. 188–191 (2003)Google Scholar
  14. 14.
    Luo, H., Fan, J., Satoh, S., Yang, J., Ribarsky, W.: Integrating multi-modal content analysis and hyperbolic visualization for large-scale news video retrieval and exploration. Signal Processing: Image Communication 23, 538–553 (2008)CrossRefGoogle Scholar
  15. 15.
    Zhai, Y., Liu, B.: Structured data extraction from the web based on partial tree alignment. IEEE Transactions on Knowledge and Data Engineering 18, 1614–1628 (2006)CrossRefGoogle Scholar
  16. 16.
    Orriols, A., Bernad’oMansilla, E.: The class imbalance problem in learning classifier systems: A preliminary study. In: GECCO Workshops, pp. 74–78 (2005)Google Scholar
  17. 17.
    Luo, H., Fan, J., Yang, J., Ribarsky, W., Satoh, S.: Exploring large-scale video news via interactive visualization. In: IEEE Symposium on Visual Analytics Science and Technology, pp. 75–82 (2006)Google Scholar
  18. 18.
    Luo, H., Fan, J., Yang, J., Ribarsky, W., Satoh, S.: Analyzing large-scale news video databases to support knowledge visualization and intuitive retrieval. In: IEEE Symposium on Visual Analytics Science and Technology (2007)Google Scholar
  19. 19.
    Luo, H., Fan, J., Keim, D.A., Satoh, S.: Personalized news video recommendation. In: Multimedia Modeling, pp. 459–471 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Hangzai Luo
    • 1
  • Peng Cai
    • 1
  • Wei Gong
    • 1
  • Jianping Fan
    • 2
  1. 1.Shanghai Key Lab of Trustworthy ComputingEast China Normal University 
  2. 2.Department of Computer ScienceUniversity of North Carolina at Charlotte 

Personalised recommendations