Dynamic Tangible User Interface Palettes

  • Martin Spindler
  • Victor Cheung
  • Raimund Dachselt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8120)


Graphics editors often suffer from a large number of tool palettes that compete with valuable document space. To address this problem and to bring back physical affordances similar to a painter’s palette, we propose to augment a digital tabletop with spatially tracked handheld displays. These displays are dynamically updated depending on their spatial location. We introduce the concept of spatial Work Zones that take up distinct 3D regions above the table surface and serve as physical containers for digital content that is organized as stacks of horizontal layers. Spatial Work Zones are represented either by physical objects or on-screen on the tabletop. Associated layers can be explored fluently by entering a spatial Work Zone with a handheld display. This provides quick access and seamless changes between tools and parts of the document that are instantly functional, i.e., ready to be used by a digital pen. We discuss several use cases illustrating our techniques and setting them into context with previous systems. Early user feedback indicates that combining dynamic GUI functionality with the physicality of spatially tracked handheld displays is promising and can be generalized beyond graphics editing.


Tangible user interface palettes spatial Work Zones tabletop displays dynamic pen heads spatial management 


  1. 1.
    Anoto Group AB – Website,
  2. 2.
    Bailly, G., Lecolinet, E., Guiard, Y.: Finger-count & radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces. In: Proc. CHI 2010, pp. 591–594. ACM Press (2010)Google Scholar
  3. 3.
    Ballendat, T., Marquardt, N., Greenberg, S.: Proxemic interaction: designing for a proximity and orientation-aware environment. In: Proc. ITS, pp. 121–130. ACM Press (2010)Google Scholar
  4. 4.
    Bencina, R., Kaltenbrunner, M.: The design and evolution of fiducials for the reacTIVision system. In: Proc. Generative Systems in the Electronic Arts 2005 (2005)Google Scholar
  5. 5.
    Bier, E.A., Stone, M.C., Pier, K., Buxton, W., DeRose, T.D.: Toolglass and magic lenses: the see-through interface. In: Proc. SIGGRAPH 1993, pp. 73–80. ACM Press (1993)Google Scholar
  6. 6.
    Block, F., Haller, M., Gellersen, H., Gutwin, C., Billinghurst, M.: VoodooSketch: extending interactive surfaces with adaptable interface palettes. In: Proc. TCI 2008, pp. 55–58. ACM Press (2008)Google Scholar
  7. 7.
    Brandl, P., Leitner, J., Seifried, T., Haller, M., Doray, B., To, P.: Occlusion-aware menu design for digital tabletops. Ext. Abstracts CHI 2009, pp. 3223–3228. ACM Press (2009)Google Scholar
  8. 8.
    Dietz, P., Leigh, D.: DiamondTouch: a multi-user touch technology. In: Proc. UIST 2001, pp. 219–226. ACM Press (2001)Google Scholar
  9. 9.
    Elrod, S., Bruce, R., Gold, R., Goldberg, R., Halasz, F., Janssen, W., Lee, D., McCall, K., Pedersen, E., Pier, K., Tang, J., Welch, B.: Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration. In: Proc. CHI 1992, pp. 599–607. ACM Press (1992)Google Scholar
  10. 10.
    Greenberg, S., Fitchett, C.: Phidgets: easy development of physical interfaces through physical widgets. In: Proc. UIST 2001, pp. 209–218. ACM Press (2001)Google Scholar
  11. 11.
    Guimbretière, F., Nguyen, C.: Bimanual marking menu for near surface interactions. In: Proc. CHI 2012, pp. 825–828. ACM Press (2012)Google Scholar
  12. 12.
    Hall, E.T.: The Hidden Dimension. Anchor Books (1966)Google Scholar
  13. 13.
    Haller, M., Brandl, P., Leithinger, D., Leitner, J., Seifried, T., Billinghurst, M.: Shared design space: Sketching ideas using digital pens and a large augmented tabletop setup. In: Pan, Z., Cheok, D.A.D., Haller, M., Lau, R., Saito, H., Liang, R. (eds.) ICAT 2006. LNCS, vol. 4282, pp. 185–196. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Harrison, C., Dey, A.K.: Lean and zoom: proximity-aware user interface and document magnification. In: Proc. CHI 2008, pp. 507–510. ACM Press (2008)Google Scholar
  15. 15.
    Holman, D., Vertegaal, R., Altosaar, M., Troje, N., Johns, D.: Paper windows: interaction techniques for digital paper. In: Proc. CHI 2005, pp. 591–599. ACM Press (2005)Google Scholar
  16. 16.
    Holmquist, L.E., Mattern, F., Schiele, B., Alahuhta, P., Beigl, M., Gellersen, H.-W.: Smart-its friends: A technique for users to easily establish connections between smart artefacts. In: Abowd, G.D., Brumitt, B., Shafer, S. (eds.) UbiComp 2001. LNCS, vol. 2201, pp. 116–122. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  17. 17.
    Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proc. CHI 1997, pp. 234–241. ACM Press (1997)Google Scholar
  18. 18.
    Izadi, S., Hodges, S., Taylor, S., Rosenfeld, D., Villar, N., Butler, A., Westhues, J.: Going beyond the display: a surface technology with an electronically switchable diffuser. In: Proc. UIST 2008, pp. 269–278. ACM Press (2008)Google Scholar
  19. 19.
    Johanson, B., Fox, A., Winograd, T.: The Interactive Workspaces Project: Experiences with Ubiquitous Computing Rooms. IEEE Pervasive Computing 1(2), 67–74 (2002)CrossRefGoogle Scholar
  20. 20.
    Kakehi, Y., Naemura, T.: UlteriorScape: Interactive optical superimposition on a view-dependent tabletop display. In: Proc. TABLETOP 2008, pp. 189–192 (2008)Google Scholar
  21. 21.
    Kirsh, D.: The intelligent use of space. Artificial Intelligence - Special Volume on Computational Research on Interaction and Agency, Part 2 73(1-2), 31–68 (1995)Google Scholar
  22. 22.
    Kray, C., Rohs, M., Hook, J., Kratz, S.: Group coordination and negotiation through spatial proximity regions around mobile devices on augmented tabletops. In: Proc. TABLETOP 2008, pp. 1–8 (2008)Google Scholar
  23. 23.
    Kurtenbach, G., Buxton, W.: Issues in combining marking and direct manipulation techniques. In: Proc. UIST 1991, pp. 137–144. ACM (1991)Google Scholar
  24. 24.
    Lee, J.C., Hudson, S.E., Summet, J.W., Dietz, P.H.: Moveable interactive projected displays using projector based tracking. In: Proc. UIST 2005, pp. 63–72. ACM Press (2005)Google Scholar
  25. 25.
    Leitner, J., Powell, J., Brandl, P., Seifried, T., Haller, M., Dorray, B., To, P.: Flux: a tilting multi-touch and pen based surface. Ext. Abstracts CHI 2009, pp. 3211–3216. ACM Press (2009)Google Scholar
  26. 26.
    Morris, M.R., Ryall, K., Shen, C., Forlines, C., Vernier, F.: Beyond “social protocols”: multi-user coordination policies for co-located groupware. In: Proc. CSCW 2004, pp. 262–265. ACM Press (2004)Google Scholar
  27. 27.
    Ryokai, K., Marti, S., Ishii, H.: I/O Brush: Drawing with Everyday Objects as Ink. In: Proc. CHI 2004. ACM Press (2004)Google Scholar
  28. 28.
    Song, H., Benko, H., Guimbretiere, F., Izadi, S., Cao, X., Hinckley, K.: Grips and Gestures on a Multi-Touch Pen. In: Proc. CHI, pp. 1323–1332. ACM Press (2011)Google Scholar
  29. 29.
    Spindler, M., Stellmach, S., Dachselt, R.: PaperLens: advanced magic lens interaction above the tabletop. In: Proc. ITS 2009, pp. 69–76. ACM Press (2009)Google Scholar
  30. 30.
    Spindler, M., Tominski, C., Schumann, H., Dachselt, R.: Tangible views for information visualization. In: Proc. ITS 2010, pp. 157–166. ACM Press (2010)Google Scholar
  31. 31.
    Spindler, M., Martsch, M., Dachselt, R.: Going Beyond the Surface: Studying Multi-Layer Interaction Above the Tabletop. In: Proc. CHI 2012. ACM Press (2012)Google Scholar
  32. 32.
    Sellen, A.J., Harper, R.H.R.: The Myth of the Paperless Office. The MIT Press (2001)Google Scholar
  33. 33.
    Ullmer, B., Ishii, H.: The metaDESK: Models and Prototypes for Tangible User Interfaces. In: Proc. UIST. ACM Press (1997)Google Scholar
  34. 34.
    Vandoren, P., Van Laerhoven, T., Claesen, L., Taelman, J., Raymaekers, C., Van Reeth, F.: IntuPaint: Bridging the gap between physical and digital painting. In: Proc. TABLETOP 2008, pp. 65–72 (2008)Google Scholar
  35. 35.
    Weiser, M.: The computer for the 21st century. Scientific American 265(3), 66–75 (1991)CrossRefGoogle Scholar
  36. 36.
    Wilson, A., Benko, H.: Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces. In: Proc. UIST 2010, pp. 273–282. ACM Press (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Martin Spindler
    • 1
  • Victor Cheung
    • 2
  • Raimund Dachselt
    • 3
  1. 1.User Interface & Software Engineering GroupUniversity of MagdeburgGermany
  2. 2.Collaborative Systems LaboratoryUniversity of WaterlooCanada
  3. 3.Interactive Media Lab DresdenTechnische Universität DresdenGermany

Personalised recommendations