Communicative Interactivity – A Multimodal Communicative Situation Classification Approach

  • Tomasz M. Rutkowski
  • Danilo Mandic
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3697)


The problem of modality detection in so called Communicative Interactivity is addressed. Multiple audio and video recordings of human communication are analyzed within this framework, based on fusion of the extracted features. At the decision level, Support Vector Machines (SVM) are utilized to segregate between the communication modalities. The proposed approach is verified through simulations on real world recordings.


Support Vector Machine Discrete Cosine Transformation Audio Feature Video Feature Multimodal Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, M.: Visualizing the pulse of a classroom. In: Proceedings of the Eleventh ACM International Conference on Multimedia, pp. 555–561. ACM Press, New York (2003)CrossRefGoogle Scholar
  2. 2.
    Gibson, J.J.: The theory of affordances. In: Shaw, R., Bransford, J. (eds.) Perceiving, Acting and Knowing. Erlbaum, Hillsdale (1977)Google Scholar
  3. 3.
    Rutkowski, T.M., Yokoo, M., Mandic, D., Yagi, K., Kameda, Y., Kakusho, K., Minoh, M.: Identification and tracking of active speaker’s position in noisy environments. In: Proceedings of International Workshop on Acoustic Echo and Noise Control (IWAENC 2003), Kyoto, Japan, pp. 283–286 (2003)Google Scholar
  4. 4.
    Rutkowski, T.M., Kakusho, K., Kryssanov, V.V., Minoh, M.: Evaluation of the communication atmosphere. In: Negoita, M., Howlett, R.J., Jain, L.C. (eds.) KES 2004. LNCS, vol. 3215, pp. 364–370. Springer, Heidelberg (2004)Google Scholar
  5. 5.
    Becchetti, C., Ricotti, L.P.: Speech Recognition. John Wiley & Sons, Inc., Great Britain (1999)Google Scholar
  6. 6.
    Sigan, L., Sclaroff, S., Athitsos, V.: Estimtion and prediction of evolving color distributions for skin segmentation under varying illumination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 152–159. IEEE, Los Alamitos (2000)Google Scholar
  7. 7.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  8. 8.
    Rutkowski, T.M., Seki, S., Yamakata, Y., Kakusho, K., Minoh, M.: Toward the human communication efficiency monitoring from captured audio and video media in real environments. In: Palade, V., Howlett, R.J., Jain, L.C. (eds.) KES 2003. LNCS, vol. 2774, pp. 1093–1100. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Shannon, C., Weaver, W.: The Mathematical Theory of Communication. University of Illinois Press, Urbana (1949)MATHGoogle Scholar
  10. 10.
    Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley & Sons, Chichester (2001)CrossRefGoogle Scholar
  11. 11.
    Hsu, C.W., Lin, C.J.: A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)CrossRefGoogle Scholar
  12. 12.
    Cherkassky, V., Mulier, F.: Learning from Data. In: Adaptive and Learning Systems for Signal Processing, Communication, and Control. John Wiley & Sons, Inc., Chichester (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Tomasz M. Rutkowski
    • 1
  • Danilo Mandic
    • 2
  1. 1.Academic Center for Computing and Media StudiesKyoto UniversityKyotoJapan
  2. 2.Department of Electrical and Electronic EngineeringImperial College LondonUnited Kingdom

Personalised recommendations