A Visualized Communication System Using Cross-Media Semantic Association

  • Xinming Zhang
  • Yang Liu
  • Chao Liang
  • Changsheng Xu
Conference paper

DOI: 10.1007/978-3-642-17829-0_9

Part of the Lecture Notes in Computer Science book series (LNCS, volume 6524)
Cite this paper as:
Zhang X., Liu Y., Liang C., Xu C. (2011) A Visualized Communication System Using Cross-Media Semantic Association. In: Lee KT., Tsai WH., Liao HY.M., Chen T., Hsieh JW., Tseng CC. (eds) Advances in Multimedia Modeling. MMM 2011. Lecture Notes in Computer Science, vol 6524. Springer, Berlin, Heidelberg

Abstract

Can you imagine that two people who have different native languages and cannot understand other’s language are able to communicate with each other without professional interpreter? In this paper, a visualized communication system is designed to facilitate such people chatting with each other via visual information. Differing from the online instant message tools such as MSN, Google talk and ICQ, which are mostly based on textual information, the visualized communication system resorts to the vivid images which are relevant to the conversation context aside from text to jump the language obstacle. The multi-phase visual concept detection strategy is applied to associate the text with the corresponding web images. Then, a re-ranking algorithm attempts to return the most related and highest quality images at top positions. In addition, sentiment analysis is performed to help people understand the emotion of each other to further reduce the language obstacle. A number of daily conversation scenes are implemented in the experiments and the performance is evaluated by user study. The experimental results show that the visualized communication system is able to effectively help people with language obstacle to better understand each other.

Keywords

Visualized Communication Sentiment Analysis Semantic Concept Detection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Xinming Zhang
    • 1
    • 2
  • Yang Liu
    • 1
    • 2
  • Chao Liang
    • 1
    • 2
  • Changsheng Xu
    • 1
    • 2
  1. 1.National Labratory of Pattern RecognitionInstitute of Automation, Chinese Academy of SciencesBeijngChina
  2. 2.China-Singapore Institute of Digital MediaSingapore

Personalised recommendations