A Multimodal Approach to Communicative Interactivity Classification
The problem of modality detection in so called communicative interactivity is addressed. Multiple audio and video recordings of human communication are analyzed within this framework, based on fusion of the extracted features. At the decision level, support vector machines (SVMs) are utilized to segregate between the communication modalities. The proposed approach is verified through simulations on real world recordings.
Keywordshuman communication analysis data fusion multimedia information processing audiovisual data fusion
Unable to display preview. Download preview PDF.
- 1.M. Chen, “Visualizing the Pulse of a Classroom,” in Proceedings of the Eleventh ACM International Conference on Multimedia, ACM Press, 2003, pp. 555–561.Google Scholar
- 2.J.J. Gibson, “The Theory of Affordances,” in Perceiving, Acting and Knowing, R. Shaw and J. Bransford (Eds.), Erlbaum, Hillsdale, NJ, 1977.Google Scholar
- 3.T.M. Rutkowski, M. Yokoo, D. Mandic, K. Yagi, Y. Kameda, K. Kakusho and M. Minoh, “Identification and Tracking of Active Speaker’s Position in Noisy Environments,” in Proceedings of International Workshop on Acoustic Echo and Noise Control (IWAENC2003), Kyoto, Japan, 2003, pp. 283–286.Google Scholar
- 5.T.M. Rutkowski, Y. Yamakata, K. Kakusho and M. Minoh, “Smart sensor mesh—intelligent sensor clusters configuration based on communicative affordances principle,” Lecture Notes in Artificial Intelligence, vol. 3490, 2005, pp. 147–157.Google Scholar
- 6.T.M. Rutkowski and D. Mandic, “Communicative interactivity—a multimodal communicative situation classification approach,” Lect. Notes Comput. Sci., vol. 3697, 2005, pp. 741–746.Google Scholar
- 7.S. Furui, “Digital Speech Processing, Synthesis, and Recognition—Second Edition, Revised and Expanded. 2nd edn. Signal Processing and Communications Series,” Marcell Dekker, Inc., New York, Basel, 2001.Google Scholar
- 9.A. Hyvarinen, J. Karhunen, E. Oja, “Independent Component Analysis,” Wiley, 2001.Google Scholar
- 12.V. Kryssanov and K. Kakusho, “From Semiotics of Hypermedia to Physics of Semiosis: A view from System Theory,” Semiotica, vol. 154, no. 1/4, 2005, pp. 11–38.Google Scholar
- 14.V. Cherkassky and F. Mulier, “Learning from Data. Adaptive and Learning Systems for Signal Processing, Communication, and Control,” Wiley, USA (1998).Google Scholar