The Effects of Multimodal Mobile Communications on Cooperative Team Interactions Executing Distributed Tasks

  • Gregory M. Burnett
  • Andres Calvo
  • Victor Finomore
  • Gregory Funke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8028)


Mobile devices are rapidly becoming an indispensible part of our everyday life. Integrated with various embedded sensors and the ability to support on-the-move processing, mobile devices are being investigated as potential tools to support cooperative team interactions and distributed real-time decision making in both military and civilian applications. A driving interest is how a mobile device equipped with multimodal communication capabilities can contribute to the effectiveness and efficiency of real-time, task outcome and performance. In this paper, we investigate the effects of a prototype multimodal collaborative Android application on distributed collaborating partners jointly working on a physical task. The mobile application’s implementation supports real-time data dissemination of an active workspace’s perspective between distributed operators. The prototype application was demonstrated in a scenario where teammates utilize different features of the software to collaboratively assemble a complex structure. Results indicated significant improvements in completion times when users visually shared their perspectives and were able to utilize image annotation versus relying on verbal descriptors.


Multimodal interfaces mobile computing remote collaboration 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clark, H.H., Brennan, S.E.: Grounding in communication. In: Resnick, L.B., Levine, R.M., Teasley, S.D. (eds.) Perspectives on Socially Shared Cognition, pp. 127–149 (1991)Google Scholar
  2. 2.
    Fussel, S.R., Kraut, R.E., Siegel, J.: Coordination of communication: effects of shared visual context on collaborative work. In: Proceedings of CSCW 2000, pp. 21–30. ACM Press, NY (2000)Google Scholar
  3. 3.
    Gergle, D., Kraut, R.E., Fussell, S.R.: Action as language in a shared visual space. In: Proceedings of CSCW 2004, vol. 6(3), pp. 487–496. ACM Press, NY (2004)Google Scholar
  4. 4.
    Gergle, D.: The value of shared visual space for collaborative physical task. In: Proceedings of CHI 2005, pp. 1116–1117 (2005)Google Scholar
  5. 5.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 139–183. North-Holland, Amsterdam (1988)CrossRefGoogle Scholar
  6. 6.
    Kirk, D., Rodden, T., Fraser, D.S.: Turn it this way: ground collaborative action with remote gestures. In: Proceedings of Computer Human Interaction (CHI): Distributed Interaction, San Jose, CA (2007)Google Scholar
  7. 7.
    Kraut, R.E., Darren, G., Fussell, S.R.: The use of visual information in shared visual co-presence. In: Proceedings of CSCW 2002, pp. 31–40 (2002)Google Scholar
  8. 8.
    Kuzuoka, H.: Spatial workspace collaboration: a sharedview video support system for remote collaboration capability. In: Proceedings of CHI 1992, pp. 533–540 (1992)Google Scholar
  9. 9.
    Ou, J., Fussell, S.R., Chen, X., Setlock, L.D., Yang, J.: Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In: International Conference on Multimodal Interfaces, pp. 242–249. ACM Press, Vancouver (2003)Google Scholar
  10. 10.
    Stevenson, D., Li, J., Smith, J., Hutchins, M.: A collaborative guidance case study. In: AUIC 2008, Wollongong, NSW, Australia, pp. 33–42 (2008)Google Scholar
  11. 11.
    Wickens, C.D., Vidulich, M., Sandry-Garza, D.: Principles of S-C-R compatibility with spatial and verbal tasks: The role of display-control location and voice-interactive display-control interfacing. Human Factors 26, 533–543 (1984)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gregory M. Burnett
    • 1
  • Andres Calvo
    • 2
  • Victor Finomore
    • 1
  • Gregory Funke
    • 1
  1. 1.Air Force Research LaboratoryWPAFBUSA
  2. 2.Ball AersopaceFairbornUSA

Personalised recommendations