Advertisement

Personal and Ubiquitous Computing

, Volume 19, Issue 5–6, pp 881–896 | Cite as

Select & Apply: understanding how users act upon objects across devices

  • Adalberto L. Simeone
  • Ming Ki Chong
  • Corina Sas
  • Hans Gellersen
Original Article

Abstract

As our interactions increasingly cut across diverse devices, we often encounter situations where we find information on one device but wish to use it on another device, for instance, a phone number spotted on a public display but wanted on a mobile. We conceptualize this problem as Select & Apply and contribute two user studies where we presented participants with eight different scenarios involving different device combinations, applications and data types. In the first, we used a think-aloud methodology to gain insights on how users currently accomplish such tasks and how they ideally would like to accomplish them. In the second, we conducted a focus group study to investigate which factors influence their actions. Results indicate shortcomings in present support for Select & Apply and contribute a better understanding of which factors affect cross-device interaction.

Keywords

Mobile Device Interaction Technique Direct Manipulation Public Display Apply Task 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

The first author was supported by a Marie Curie Intra European Fellowship within the 7th European Community Framework Programme.

References

  1. 1.
    Ballendat T, Marquardt N, Greenberg S (2010) Proxemic interaction: designing for a proximity and orientation-aware environment. In: ACM international conference on interactive tabletops and surfaces, ITS’10. ACM, New York, NY, USA, pp 121–130Google Scholar
  2. 2.
    Baudisch P, Cutrell E, Robbins D, Czerwinski M, Tandler P, Bederson B, Zierlinger A (2003) Drag-and-pop and drag-and-pick: techniques for accessing remote screen content on touch and pen-operated systems. In: Proceedings of interact’03. IOS Press, pp 57–64Google Scholar
  3. 3.
    Beigl M (1999) Point and click: interaction in smart environments. In: Proceedings of Handheld and Ubiquitous Computing, HUC’99. Springer, London, UK pp 311–313Google Scholar
  4. 4.
    Boring S, Baur D, Butz A, Gustafson S, Baudisch P (2010) Touch projector: mobile interaction through video. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’10. ACM, New York, NY, USA, pp 2287–2296Google Scholar
  5. 5.
    Bragdon A, DeLine R, Hinckley K, Morris MR (2011) Code space: touch + air gesture hybrid interactions for supporting developer meetings. In: Proceedings of interactive tabletops and surfaces, ITS’11. ACM, pp 212–221Google Scholar
  6. 6.
    Chang T-H, Li Y (2011) Deep shot: a framework for migrating tasks across devices using mobile phone cameras. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’11. ACM, pp 2163–2172Google Scholar
  7. 7.
    Dearman D, Pierce JS (2008) It’s on my other computer!: computing with multiple devices. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’08. ACM New York, NY, USA, pp 767–776Google Scholar
  8. 8.
    Fitzmaurice GW, Ishii H, Buxton WA (1995) Bricks: laying the foundations for graspable user interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’95. ACM Press/Addison-Wesley Publishing Co., pp 442–449Google Scholar
  9. 9.
    Hardy R, Rukzio E (2008) Touch and interact: touch-based interaction of mobile phones with displays. In: Proceedings of Mobile HCI’08, pp 245–254Google Scholar
  10. 10.
    Hinckley K (2003) Synchronous gestures for multiple persons and computers. In: Proceedings of the ACM symposium on user interface software and technology, UIST’03. ACM, pp 149–158Google Scholar
  11. 11.
    Hinckley K, Ramos G, Guimbretiere F, Baudisch P, Smith M (2004) Stitching: pen gestures that span multiple displays. In: Proceedings of advanced visual interfaces, AVI’04. ACM, pp 23–31Google Scholar
  12. 12.
    Holman D, Vertegaal R, Altosaar M, Troje N, Johns D (2005) Paper windows: interaction techniques for digital paper. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’05. ACM, New York, NY, USA, pp 591–599Google Scholar
  13. 13.
    Kray C, Nesbitt D, Dawson J, Rohs M (2010) User-defined gestures for connecting mobile phones, public displays, and tabletops. In: Proceedings of mobile HCI’10. ACM, pp 239–248Google Scholar
  14. 14.
    Kujala S (2003) User involvement: a review of the benefits and challenges. Behav Inf Technol 22(1):1–16CrossRefGoogle Scholar
  15. 15.
    Kujala S (2008) Effective user involvement in product development by improving the analysis of user needs. Behav Inf Technol 27(6):457–473CrossRefGoogle Scholar
  16. 16.
    Kurdyukova E, Redlin M, André E Studying user-defined ipad gestures for interaction in multi-display environment. In: Proceedings of intelligent user interfaces, IUI’12. ACM, pp 93–96Google Scholar
  17. 17.
    Lakoff G, Johnson M (1999) Philosophy in the flesh: the embodied mind and interactive tabletops and surfaces challenge to western thought. Basic books , New YorkGoogle Scholar
  18. 18.
    Lakoff G, Turner M (1989) More than cool reason: a field guide to poetic metaphor. University of Chicago Press, ChicagoCrossRefGoogle Scholar
  19. 19.
    Lucero A, Holopainen J, Jokela T (2011) Pass-them-around: collaborative use of mobile phones for photo sharing. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’11. ACM, pp 1787–1796Google Scholar
  20. 20.
    Marquardt N, Hinckley K, Greenberg S (2012) Cross-device interaction via micro-mobility and f-formations. In: Proceedings of the ACM symposium on user interface software and technology, UIST’12. ACM, pp 13–22Google Scholar
  21. 21.
    Miller RC, Myers BA et al (1999) Synchronizing clipboards of multiple computers. In: Proceedings of the ACM symposium on user interface software and technology, vol 7 of UIST’99. pp 65–66Google Scholar
  22. 22.
    Mistry P, Nanayakkara S, Maes P Touch and copy, touch and paste. In: CHI extended Abstract’11. ACM, pp 1095–1098Google Scholar
  23. 23.
    Morris MR, Wobbrock JO, Wilson AD (2010) Understanding users’ preferences for surface gestures. In: Proceedings of graphics interface, GI’10. Canadian Information Processing Society, pp 261–268Google Scholar
  24. 24.
    Rekimoto J (1997) Pick-and-drop: a direct manipulation technique for multiple computer environments. In: Proceedings of the ACM symposium on user interface software and technology, UIST’97. ACM, pp 31–39Google Scholar
  25. 25.
    Rekimoto J, Ayatsuka Y, Kohno M, Oba H (2003) Proximal interactions: a direct manipulation technique for wireless networking. In: Proceedings of INTERACT’03. IOS Press, pp 511–518Google Scholar
  26. 26.
    Rekimoto J, Saitoh M (1999) Augmented surfaces: a spatially continuous work space for hybrid computing environments. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’99. ACM, pp 378–385Google Scholar
  27. 27.
    Scharf F, Wolters C, Cassens J, Herczeg M (2013) Cross-device interaction. In: AMBIENT 2013, the 3rd international conference on ambient computing, applications, services and technologies, pp 35–41Google Scholar
  28. 28.
    Schmidt D, Chehimi F, Rukzio E, Gellersen H (2010) PhoneTouch: a technique for direct phone interaction on surfaces. In: Proceedings of the ACM symposium on user interface software and technology, UIST’10. ACM, pp 13–16Google Scholar
  29. 29.
    Simeone AL, Seifert J, Schmidt D, Holleis P, Rukzio E, Gellersen H (2013) A cross-device drag-and-drop technique. In: Proceedings of the 12th international conference on mobile and ubiquitous multimedia, MUM’13. ACM, New York, NY, USA, pp 10:1–10:4Google Scholar
  30. 30.
    Swindells C, Inkpen KM, Dill JC, Tory M (2002) That one there! pointing to establish device identity. In: Proceedings of the ACM symposium on user interface software and technology, UIST’02. ACM, pp 151–160Google Scholar
  31. 31.
    Turner J, Bulling A, Alexander J, Gellersen H (2014) Cross-device gaze-supported point-to-point content transfer. In: Proceedings of eye tracking research and applications symposium, ETRA’14Google Scholar
  32. 32.
    Wobbrock JO, Morris MR, Wilson AD (2009) User-defined gestures for surface computing. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’09. ACM, pp 1083–1092Google Scholar

Copyright information

© Springer-Verlag London 2015

Authors and Affiliations

  • Adalberto L. Simeone
    • 1
    • 2
  • Ming Ki Chong
    • 1
  • Corina Sas
    • 1
  • Hans Gellersen
    • 1
  1. 1.School of Computing and CommunicationsLancaster UniversityLancasterUK
  2. 2.School of Creative TechnologiesUniversity of PortsmouthPortsmouthUK

Personalised recommendations