Personal and Ubiquitous Computing

, Volume 12, Issue 3, pp 255–267 | Cite as

Using job-shop scheduling tasks for evaluating collocated collaboration

  • Desney S. TanEmail author
  • Darren Gergle
  • Regan Mandryk
  • Kori Inkpen
  • Melanie Kellar
  • Kirstie Hawkey
  • Mary Czerwinski
Original Article


Researchers have begun to explore tools that allow multiple users to collaborate across multiple devices in collocated environments. These tools often allow users to simultaneously place and interact with information on shared displays. Unfortunately, there is a lack of experimental tasks to evaluate the effectiveness of these tools for information coordination in such scenarios. In this article, we introduce job-shop scheduling as a task that could be used to evaluate systems and interactions within computer-supported collaboration environments. We describe properties that make the task useful, as well as evaluation measures that may be used. We also present two experiments as case studies to illustrate the breadth of scenarios in which this task may be applied. The first experiment shows the differences when users interact with different communicative gesturing schemes, while the second demonstrates the benefits of shared visual information on large displays. We close by discussing the general applicability of the tasks.


Job-shop scheduling task Evaluation Collocated environments Computer-supported collaborative work User study 



We thank Patrick Baudisch, Andrew Faulring, James Fogarty, Jana Foster, Susan Fussell, Teresa Janz, Brian Meyers, Jeff Nichols, Judy Olson, Randy Pausch, Daniel Robbins, George Robertson, Greg Smith, and Ryder Ziola for their discussions of the tasks and assistance with the experiments.


  1. 1.
    Mennecke BE, Wheeler BC (1993) An essay and resource guide for dyadic and group task selection and usage. Institute for research on the management of information systems working paper #9309, Indiana University, BloomingtonGoogle Scholar
  2. 2.
    Heath CC, Knoblauch J, Luff P (2000) Technology and social interaction: the emergence of workplace studies. Br J Sociol 51(2):299–320CrossRefGoogle Scholar
  3. 3.
    Dourish P (2006) Implications for design. Proceedings of CHI 2006 conference on human factors in computing systems, pp 541–550Google Scholar
  4. 4.
    Inkpen K, Mandryk R, DiMicco JM, Scott S (2004) Methodologies for evaluating collaboration in co-located environments. Workshop presented at the ACM conference on computer-supported cooperative work 2004Google Scholar
  5. 5.
    Pinelle D, Gutwin C, Greenberg S (2003) Task analysis for groupware usability evaluation: modeling shared-workspace tasks with the mechanics of collaboration. ACM Trans Hum Comput Interact 10(4):281–311CrossRefGoogle Scholar
  6. 6.
    McGrath JE (1984) Groups: interaction and performance. Prentice Hall, New JerseyGoogle Scholar
  7. 7.
    Olson JS, Olson GM, Storrøsten M, Carter M (1993) Groupwork close up: a comparison of the group design process with and without a simple group editor. ACM Trans Hum Comput Interact 11:321–348Google Scholar
  8. 8.
    Scott SD, Carpendale MST, Inkpen, KM (2004) Territoriality in collaborative tabletop workspaces. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 294–303Google Scholar
  9. 9.
    Gutwin C (2002) Traces: visualizing the immediate past to support group interaction. Proceedings of graphics interface 2002, pp 43–50Google Scholar
  10. 10.
    Clark HH, Krych MA (2004) Speaking while monitoring addressees for understanding. J Mem Lang 50:62–81CrossRefGoogle Scholar
  11. 11.
    Kraut RE, Gergle D, Fussell SR (2002) The use of visual information in shared visual spaces: informing the development of virtual co-presence. Proceedings of the ACM conference on computer-supported cooperative work 2002, pp 31–40Google Scholar
  12. 12.
    Rocco E (1998) Trust breaks down in electronic contexts but can be repaired by some initial face-to-face contact. Proceedings of CHI 1998 conference on human factors in computing systems, pp 496–502Google Scholar
  13. 13.
    Setlock LD, Fussell SR, Neuwirth C (2004). Taking it out of context: collaborating within and across cultures in face-to-face settings and via instant messaging. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 604–613Google Scholar
  14. 14.
    Karger D, Stein C, Wein J (1997) Scheduling algorithms. In: Atallah MJ (ed) Handbook of algorithms and theory of computation. CRC Press, Boca RatonGoogle Scholar
  15. 15.
    Gutwin C, Penner R (2002) Improving interpretation of remote gestures with telepointer traces. Proceedings of the ACM conference on computer-supported cooperative work 2002, pp 49–57Google Scholar
  16. 16.
    Khan A, Matejka J, Fitzmaurice G, Kurtenbach G (2005) Spotlight: directing users’ attention on large displays. Proceedings of CHI 2005 conference on human factors in computing systems, pp 791–798Google Scholar
  17. 17.
    Clark HH, Wilkes-Gibbs D (1986) Referring as a collaborative process. Cognition 22:1–39CrossRefGoogle Scholar
  18. 18.
    Fisher H, Thompson G (1963) Probabilistic learning combinations of local job-shop scheduling rules. In: Muth J, Thompson G (eds) Industrial scheduling. Prentice Hall, Englewood Cliffs, pp 225–251Google Scholar
  19. 19.
    Beasley JE (1990) OR-Library: distributing test problems by electronic mail. J Oper Res Soc 41(11):1069–1072.
  20. 20.
    Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Human Factors Special Issue Situation Awareness 37:32–64Google Scholar
  21. 21.
    Kraut RE, Fussell SR, Siegel J (2003) Visual information as a conversational resource in collaborative physical tasks. Hum Comput Interact 18:13–49CrossRefGoogle Scholar
  22. 22.
    Gergle D, Kraut RE, Fussell, SR (2004) Action as language in a shared visual space. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 487–496Google Scholar
  23. 23.
    Tan DS, Meyers B, Czerwinski M (2004) Wincuts: manipulating arbitrary window regions for more effective use of screen space. Proceedings of CHI 2004 conference on human factors in computing systems, pp 1525–1528Google Scholar
  24. 24.
    Kenny DA, Mannetti L, Pierro A, Livi S, Kashy DA (2002) The statistical analysis of data from small groups. J Pers Soc Psychol 83:126–137CrossRefGoogle Scholar
  25. 25.
    Hoenig JM, Heisey DM (2001) The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat 55:19–24CrossRefMathSciNetGoogle Scholar
  26. 26.
    Ryall K, Forlines C, Shen C, Ringel-Morris M (2004) Tabletop design: exploring the effects of group size and table size on interactions with tabletops shared-display groupware. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 284–293Google Scholar
  27. 27.
    Scott SD (2005) Territoriality in collaborative tabletop workspaces. Doctoral Dissertation, Department of Computer Science, University of CalgaryGoogle Scholar
  28. 28.
    Tan DS (2004) Exploiting the cognitive and social benefits of physically large displays. Doctoral Dissertation, School of Computer Science, Carnegie Mellon UniversityGoogle Scholar
  29. 29.
    Johanson B, Fox A, Winograd T (2002) The interactive workspaces project: Experiences with ubiquitous computing rooms. IEEE Pervasive Comput Mag 1(2):67–74CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2007

Authors and Affiliations

  • Desney S. Tan
    • 1
    Email author
  • Darren Gergle
    • 2
  • Regan Mandryk
    • 3
  • Kori Inkpen
    • 3
  • Melanie Kellar
    • 3
  • Kirstie Hawkey
    • 3
  • Mary Czerwinski
    • 1
  1. 1.Microsoft ResearchRedmondUSA
  2. 2.Northwestern UniversityEvanstonUSA
  3. 3.Dalhousie UniversityHalifaxCanada

Personalised recommendations