Cognitive Computation

, Volume 4, Issue 2, pp 157–171 | Cite as

CO-WORKER: Toward Real-Time and Context-Aware Systems for Human Collaborative Knowledge Building

  • Stefano Squartini
  • Anna EspositoEmail author


The information exchange occurring during human interactions, conveyed through verbal and nonverbal communication modes, builds up a new-shared knowledge among the interacting people. A current automatic meeting assistance system is just able to store such an exchange (for successive offline processing), while it would be valuable developing automatic tools that provide appropriate support as it takes place. Currently, the international scientific community is strongly committed toward the implementation of intelligent instruments able to recognize and process in real-time relevant interactional signals in order to provide timely support to the happening interaction. This work will argue on an even more comprehensive paradigm for collaborative computer support to human interaction, not adequately addressed in the literature so far, concerning the implementation of human–computer interaction (HCI) systems able to process in real-time multimodal signals, to infer contextual information, and support in a collaborative way human interaction in-group activities, such as learning, discussion, work cooperation, decision-making, and problem solving. Such systems should act as co-workers, actively cooperating and contributing to the group’s knowledge building and pretending to share with the group, significances and individual potentialities rather than act as passive data storing devices. In carrying out their functions, these HCI systems will be placed on a group cognitive level, where individual purposes, actions, and emotions are mediated by the group interaction, and meanings are mainly built through the group shared knowledge and experience.


Collaborative knowledge building (CKB) Human–computer interaction (HCI) Real-time signal processing Affective computing Group cognition Machine learning 


  1. 1.
    Zastrow C, Kirst-Ashman KK. Understanding human behavior and the social environment. Belmont: Thomson Brooks-Cole; 2007.Google Scholar
  2. 2.
    Hogg M, Abrams D. Social identifications: a social psychology of intergroup relations and group processes. New York: Thomson Publishing; 1988.Google Scholar
  3. 3.
    Stahl G. Group cognition: computer support for building collaborative knowledge (acting with technology). MIT Press. 2006.
  4. 4.
    Tur G, et al. The CALO meeting assistant system. IEEE Trans Audio Speech Lang Process. 2010;18(6):1601–11.CrossRefGoogle Scholar
  5. 5.
    Bourlard H, Renals S. Recognition and understanding of meetings overview of the European AMI and AMIDA projects. 2008 (Online).
  6. 6.
    AMI-AMIDA projects: augmented multi-party interaction, 2009 (Online).
  7. 7.
    NCCR-IM2 project: the swiss national center of competence in research on interactive multimodal information management, 2011 (Online).
  8. 8.
    Waibel A, Steusloff H, Stiefelhagen R, Watson K. Computers in the human interaction loop. Springer, London; 2009. p. 3–6.Google Scholar
  9. 9.
    Stahl G. Studying virtual math teams. 2010 (Online).
  10. 10.
    Picard RW. Affective computing. Cambridge, MA: MIT Press; 1997.Google Scholar
  11. 11.
    Minsky M. The emotion machine. New York: Simon & Schuster; 2006.Google Scholar
  12. 12.
    Krenn B. The NECA project: net environments for embodied emotional conversational agents. In Proceedings of workshop on emotionally rich virtual worlds with emotion synthesis at the 8th international conference on 3D web technology (Web3D). 10 Mar 2003.Google Scholar
  13. 13.
    Balomenos T, Raouzaiou A, Ioannou S, Drosopoulos A, Karpouzis K, Kollias S (2005) Emotion analysis in man-machine interaction systems. In Machine learning for multimodal interaction, pp 318–328.Google Scholar
  14. 14.
    Paiva A, et al. (2001) Safira: supporting affective interactions in real-time applications. CAST 2001: living in mixed realities. Schloss Birlinghoven, Germany, p. 227–230.Google Scholar
  15. 15.
    The HUMAINE portal, 2011 (Online).
  16. 16.
    SSPnet, a European network of excellence in social signal processing, 2011 (Online).
  17. 17.
    Anolli L, Mantovani F, Balestra M, Agliati A, Realdon O, Zurloni V, Mortillaro M, Vescovo A, Confalonieri L. The potential of affective computing in e-learning: MYSELF project experience. International conference on human-computer interaction (interact 2005), workshop on eLearning and human-computer interaction: exploring design synergies for more effective learning experiences, 2005.Google Scholar
  18. 18.
    SIENTO project: affective learning technologies, 2011 (Online).
  19. 19.
    Beaudouin-Lafon M. Computer supported co-operative work. New York: Wiley; 1999.Google Scholar
  20. 20.
    Scardamalia M, Bereiter C. Knowledge building: theory, pedagogy, and technology. In: Sawyer K, editor. Cambridge handbook of the learning sciences. New York: Cambridge University Press; 2006. p. 97–118.Google Scholar
  21. 21.
    Principi E, Cifani S, Rocchi C, Squartini S, Piazza F. Keyword spotting based system for conversation fostering in tabletop scenarios: preliminary evaluation. HSI’09: the 2nd IES international conference on human system interaction, Catania, Italy, 21st–23rd May 2009.Google Scholar
  22. 22.
    Rocchi C, Principi E, Cifani S, Rotili R, Squartini S, Piazza F. A real-time speech-interfaced system for group conversation modeling. In Bruno A, Simone B, Carlo M, editors. Proceeding of the 2009 conference on neural nets WIRN09: proceedings of the 19th Italian workshop on neural nets, vietri sul mare, Salerno, Italy, May 28–30 2009. Amsterdam: IOS Press; 2009. p. 70–80.Google Scholar
  23. 23.
    Hutchins E. Cognition in the wild. Cambridge: MIT Press; 1995.Google Scholar
  24. 24.
    Leontev AN. Activity, consciousness, and personality. Englewood Cliffs: Prentice-Hall; 2000.Google Scholar
  25. 25.
    Garfinkel H. Studies in ethnomethodology. Malden, MA: Blackwell Publishers Inc.; 1984.Google Scholar
  26. 26.
    Watzlawick P, Hemlick Beavin J, Jackson DD. Pragmatics of human communication. New York: W.W. Norton; 1967.Google Scholar
  27. 27.
    McNeill D. Gestures and thoughts. Chicago: University of Chicago Press; 2005.Google Scholar
  28. 28.
    Munhall KG, Jones JA, Callan DE, Kuratate T, Vatikokis-Bateson E. Visual prosody and speech intelligibility. Psych Sci. 2004;15(2):133–7.CrossRefGoogle Scholar
  29. 29.
    Bargh JA, Chen M, Burrows L. Automaticity of social behavior: direct effects of trait construct and stereotype priming on action. J Pers Soc Psych. 1996;71:230–44.CrossRefGoogle Scholar
  30. 30.
    Fodor JA. The modularity of mind. Cambridge: MIT Press; 1983.Google Scholar
  31. 31.
    Barsalou LW, Niedenthal PM, Barbey A, Ruppert J. Social embodiment. In: Ross B, editor. The psychology of learning and motivation, vol. 43. San Diego: Academic Press; 2003. p. 43–92.Google Scholar
  32. 32.
    Hughes HC, et al. Visual auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psych Hum Percept Perf. 1994;20:131–53.CrossRefGoogle Scholar
  33. 33.
    Perrott DR, et al. Aurally aided visual search in the central visual field: effects of visual load and visual enhancement of the target. Hum Factors J Hum Factors Ergon Soc. 1991;33(4):389–400.Google Scholar
  34. 34.
    Callan DE, et al. Neural processes underlying perceptual enhancement by visual speech gestures. NeuroReport. 2003;14:2213–8.PubMedCrossRefGoogle Scholar
  35. 35.
    McGurk H, MacDonald J. Hearing lips and seeing voices. Nature. 1976;264:746–8.PubMedCrossRefGoogle Scholar
  36. 36.
    Esposito A. The perceptual and cognitive role of visual and auditory channels in conveying emotional information. Cognit Comput. 2009;1:268–78.CrossRefGoogle Scholar
  37. 37.
    Bales RF. Interaction process analysis: a method for the study of small groups. University of Chicago Press (Online).
  38. 38.
    Kipp MA. A generic annotation tool for multimodal dialogue. In Proceedings of the 7th European conference on speech communication and technology (Eurospeech); 2001. p. 1367–1370.Google Scholar
  39. 39.
    Sloetjes H, Wittenburg P. Annotation by category: ELAN and ISO DCR. In Proceedings of the 6th international conference on language resources and evaluation, 2008.Google Scholar
  40. 40.
    Neale D, Carroll J, Rosson M. Evaluating computer-supported cooperative work: models and frameworks. In Proceedings of the 2004 ACM conference on computer supported cooperative work; 2004. p. 112–121.Google Scholar
  41. 41.
    Gress CLZ, Fior M, Hadwin AF, Winne PH. Measurement and assessment in computer-supported collaborative learning. Comput Hum Behav. 2010;26(5):806–14.CrossRefGoogle Scholar
  42. 42.
    Bastien C, Scapin DL. Evaluating a user interface with ergonomic criteria. Int J Hum Comput Interact. 1995;7:105–21.CrossRefGoogle Scholar
  43. 43.
    Schmidt K. Cooperative work and coordinative practices. Computer supported co-operative work. London: Springer; 2011 (Online). doi: 10.1007/978-1-84800-068-1-1.
  44. 44.
    Stahl G, Hesse FW. The CSCL field matures. Int J Comput Support Collab Learn. 2010;5(1):1–3.CrossRefGoogle Scholar
  45. 45.
    Stahl G, Koschmann T, Suthers D. Computer-supported collaborative learning: an historical perspective. In Sawyer RK, editor. Cambridge handbook of the learning sciences. Cambridge, UK: Cambridge University Press. p. 409–426 (Online)., 2006.
  46. 46.
    Dillenbourg P, Jarvela S, Fischer F. The evolution of research on computer-supported collaborative learning. In Balacheff N, et al., editors. Technology-enhanced learning. Springer; 2009. p. 3–19.Google Scholar
  47. 47.
    Ellis C, Gibbs S, Rein G. Groupware: some issues and experiences. Commun ACM. 1991;34(1):39–58.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.3MediaLabs, Dipartimento di Ingegneria dell’InformazioneUniversità Politecnica delle MarcheAnconaItaly
  2. 2.Dipartimento di Psicologia and IIASSSeconda Università di NapoliSalernoItaly

Personalised recommendations