Advertisement

Grab ‘n’ Drop: User Configurable Toolglasses

  • James R. EaganEmail author
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10515)

Abstract

We introduce the grab ‘n’ drop toolglass, an extension of the toolglass bi-manual interaction technique. It enables users to create and configure their own toolglasses from existing user interfaces that were not designed for toolglasses. Users compose their own toolglass interactions at runtime from an application’s user interface elements, bringing interaction closer to the objects of interest in a workspace. Through a proof-of-concept implementation for Mac OS X, we show how grab ‘n’ drop capabilities could be added to existing applications at the toolkit level, without modifying application source code or UI design. Finally, we evaluate the power and flexibility of this approach by applying it to a variety of applications. We further identify limitations and risks associated with this approach and propose changes to existing toolkits to foster such user-reconfigurable interaction.

Keywords

User interfaces Toolglasses Instrumental interaction Polymorphism 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Apple Computer Inc.: AppleScript Language Guide. Addison-Wesley Longman Publishing Co., Inc., Boston (1994)Google Scholar
  2. 2.
    Beaudouin-Lafon, M.: Instrumental interaction: an interaction model for designing post-WIMP user interfaces. In: CHI 2000: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 446–453. ACM, New York (2000)Google Scholar
  3. 3.
    Beaudouin-Lafon, M., Mackay, W.E.: Reification, polymorphism and reuse: three principles for designing visual interfaces. In: AVI 2000: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 102–109. ACM Press (2000)Google Scholar
  4. 4.
    Bier, E.A., Stone, M.C., Pier, K., Buxton, W., DeRose, T.D.: Toolglass and magic lenses: the see-through interface. In: SIGGRAPH 1993: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 73–80. ACM, New York (1993)Google Scholar
  5. 5.
    Coutaz, J.: PAC: an object oriented model for implementing user interfaces. SIGCHI Bull. 19(2), 37–41 (1987). http://doi.acm.org/10.1145/36111.1045592 CrossRefGoogle Scholar
  6. 6.
    Cypher, A., Dontcheva, M., Lau, T., Nichols, J.: No Code Required: Giving Users Tools to Transform the Web. Morgan Kaufmann Publishers Inc., San Francisco (2010)Google Scholar
  7. 7.
    Cypher, A., Halbert, D.C., Kurlander, D., Lieberman, H., Maulsby, D., Myers, B.A., Turransky, A. (eds.): Watch What I Do: Programming by Demonstration. MIT Press, Cambridge (1993)Google Scholar
  8. 8.
    Dixon, M., Fogarty, J.: Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure. In: CHI 2010: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 1525–1534. ACM, New York (2010)Google Scholar
  9. 9.
    Dorn, B., Tew, A.E., Guzdial, M.: Introductory computing construct use in an end-user programming community. In: Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, VLHCC 2007, pp. 27–32 (2007). http://dx.doi.org/10.1109/VLHCC.2007.33
  10. 10.
    Eagan, J.R., Beaudouin-Lafon, M., Mackay, W.E.: Cracking the cocoa nut: user interface programming at runtime. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST 2011, pp. 225–234. ACM, New York (2011). http://doi.acm.org/10.1145/2047196.2047226
  11. 11.
    Fishkin, K., Stone, M.C.: Enhanced dynamic queries via movable filters. In: CHI 1995: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 415–420. ACM Press/Addison-Wesley Publishing Co., New York (1995)Google Scholar
  12. 12.
    Gantt, M., Nardi, B.A.: Gardeners and gurus: patterns of cooperation among cad users. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 107–117. ACM, New York (1992)Google Scholar
  13. 13.
    Hudson, S.E., Rodenstein, R., Smith, I.: Debugging lenses: a new class of transparent tools for user interface debugging. In: UIST 1997: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, pp. 179–187. ACM, New York (1997)Google Scholar
  14. 14.
    Hutchings, D.R., Stasko, J.: Quantifying the performance effect of window snipping in multiple-monitor environments. In: Baranauskas, C., Palanque, P., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4663, pp. 461–474. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74800-7_42 CrossRefGoogle Scholar
  15. 15.
    Kato, J., Sakamoto, D., Igarashi, T.: Picode: inline photos representing posture data in source code. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2013, pp. 3097–3100. ACM, New York (2013). http://doi.acm.org/10.1145/2470654.2466422
  16. 16.
    Klokmose, C.N., Zander, P.-O.: Rethinking laboratory notebooks. In: Lewkowicz, M., Hassanaly, P., Wulf, V., Rohde, M. (eds.) Proceedings of COOP 2010, pp. 119–139. Springer, London (2010). doi: 10.1007/978-1-84996-211-7_8 CrossRefGoogle Scholar
  17. 17.
    Mackay, W.E.: Patterns of sharing customizable software. In: Proceedings of the 1990 ACM Conference on Computer-Supported Cooperative Work, pp. 209–221. ACM Press, New York (1990)Google Scholar
  18. 18.
    Mackay, W.E.: Users and Customizable Software: A Co-Adaptive Phenomenon. Ph.D. thesis, Massechusetts Institute of Technology (1990)Google Scholar
  19. 19.
    Mackay, W.E.: Triggers and barriers to customizing software. In: CHI 1991: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 153–160. ACM Press (1991)Google Scholar
  20. 20.
    Mackay, W.E.: Which interaction technique works when?: floating palettes, marking menus and toolglasses support different task strategies. In: AVI 2002: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 203–208. ACM, New York (2002)Google Scholar
  21. 21.
    MacLean, A., Carter, K., Lövstrand, L., Moran, T.: User-tailorable systems: pressing the issues with buttons. In: CHI 1990: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 175–182. ACM Press, New York (1990)Google Scholar
  22. 22.
    Nardi, B.A.: A Small Matter of Programming: Perspectives on End User Computing. MIT Press, Cambridge (1993)Google Scholar
  23. 23.
    Ponsard, A., McGrenere, J.: Anchored customization: anchoring settings to the application interface to afford customization. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI 2016. pp. 4154–4165. ACM, New York (2016). http://doi.acm.org/10.1145/2858036.2858129
  24. 24.
    Reenskaug, T.: Models—views—controllers. Technical report, Xerox PARC, December 1979Google Scholar
  25. 25.
    Robertson, G.G., Henderson Jr., D.A., Card, S.K.: Buttons as first class objects on an X desktop. In: Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology, pp. 35–44. ACM Press, New York (1991)Google Scholar
  26. 26.
    Stuerzlinger, W., Chapuis, O., Phillips, D., Roussel, N.: User interface façades: towards fully adaptable user interfaces. In: UIST 2006: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology, pp. 309–318. ACM, New York (2006)Google Scholar
  27. 27.
    Tan, D.S., Meyers, B., Czerwinski, M.: WinCuts: manipulating arbitrary window regions for more effective use of screen space. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1525–1528. ACM, New York (2004)Google Scholar
  28. 28.
    Yeh, T., Chang, T.H., Miller, R.C.: Sikuli: using GUI screenshots for search and automation. In: UIST 2009: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, pp. 183–192. ACM, New York (2009)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  1. 1.LTCI, Télécom ParisTechUniversité Paris-SaclayParisFrance

Personalised recommendations

Cite paper

Buy options