Building Interactive Multi-touch Surfaces

  • Johannes SchöningEmail author
  • Jonathan Hook
  • Tom Bartindale
  • Dominik Schmidt
  • Patrick Oliver
  • Florian Echtler
  • Nima Motamedi
  • Peter Brandl
  • Ulrich von Zadow
Part of the Human-Computer Interaction Series book series (HCIS)


Multi-touch interaction with computationally enhanced surfaces has received considerable attention in recent years. Hardware implementations of multi-touch interaction such as Frustrated Total Internal Reflection (FTIR) and Diffuse Illumination (DI) have allowed for the low cost development of surfaces. Although many of these technologies and associated applications have been presented in academic settings, the practicalities of building a high quality multi-touch enabled surface, both in terms of the software and hardware required, are not widely known. We draw upon our extensive experience as developers of multi-touch technology to provide practical advice in relation to building, and deploying applications upon, multi-touch surfaces. This includes technical details of the construction of optical multi-touch surfaces, including: infrared illumination, silicone compliant surfaces, projection screens, cameras, filters, and projectors, and an overview of existing software libraries for tracking.


Fiducial Marker Shadow Image Projection Screen Illumination Source Compliant Surface 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We would like to thank Florian Daiber, Otmar Hilliges, Markus Löchtefeld, Laurence Muller, Tim Roth, Daivd Smith, Urich von Zadow, David Hollmann and Antonio Krüger for their help with the Multi-touch Bootcamp 2008 in Amsterdam in conjunction with IEEE Tabletops and for their comments and help (e.g. graphics and feedback) on this chapter.


  1. 1.
    Johnson RG (1972) Touch actuable data input panel assembly. US Patent
  2. 2.
    Kasday LR (1984) #4,484,179: Touch position sensitive surface. US Patent
  3. 3.
    Mallos JB (1982) #4,346,376: Touch position sensitive surface. US Patent
  4. 4.
    Mueller RE (1974) #3,846,826: Direct television drawing and image manipulation system. US Patent
  5. 5.
    White RM (1987) #4,484,179: Tactile sensor employing a light conducting element and a resiliently deformable sheet. US Patent
  6. 6.
    Han JY (2005) Low-cost multi-touch sensing through frustrated total internal reflection. In: Proceedings of the 18th annual ACM symposium on user interface software and technology (UIST ’05), ACM Press, New York, pp 115–118Google Scholar
  7. 7.
    Dietz P, Leigh D (2001) DiamondTouch: A multi-user touch technology. Proceedings of the 14th annual ACM symposium on user interface software and technology, pp 219–226Google Scholar
  8. 8.
    Kin K, Agrawala M, DeRose T (2009) Determining the benefits of direct-touch, bimanual, and multifinger input on a multitouch workstation. In: Proceedings of graphics interface 2009 (GI ’09), canadian information processing society, Toronto, Ontario, Canada, pp 119–124Google Scholar
  9. 9.
    Moscovich T (2006) Multi-touch interaction. In: CHI ’06 extended abstracts on human factors in computing systems (CHI ’06), ACM Press, New York, pp 1775–1778Google Scholar
  10. 10.
    Peltonen P, Kurvinen E, Salovaara A, Jacucci G, Ilmonen T, Evans J, Oulasvirta A, Saarikko P (2008) It’s mine, don’t touch!: Interactions at a large multi-touch display in a city centre. In: Proceeding of the twenty-sixth annual SIGCHI conference on human factors in computing systems (CHI ’08), ACM Press, New York, pp 1285–1294Google Scholar
  11. 11.
    Rekimoto J (2002) Smartskin: An infrastructure for freehand manipulation on interactive surfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ’02), ACM Press, New York, pp 113–120Google Scholar
  12. 12.
    Schöning J, Hecht B, Raubal M, Krüger A, Marsh M, Rohs M (2008) Improving interaction with virtual globes through spatial thinking: Helping users ask "why?". In: Proceedings of the 13th international conference on intelligent user interfaces (IUI ’08), ACM Press, New York, pp 129–138Google Scholar
  13. 13.
    Valli A, Linari L (2008) Natural interaction sensitivetable. In: CHI ’08 extended abstracts on human factors in computing systems (CHI ’08), ACM Press, New York, pp 2315–2318Google Scholar
  14. 14.
    Schöning J, Krüger A, Olivier P (2009) Multi-touch is dead, long live multi-touch. CHI 2009: Workshop on Multi-touch and Surface Computing, April 4 – March 9, Boston, MA,∼jschoen/website/Publications_files/SchoeningMultiTouchisDead.pdf, accessed 11.03.2010Google Scholar
  15. 15.
    Schöning J, Brandl P, Daiber F, Echtler F, Hilliges O, Hook J, Löchtefeld M, Motamedi N, Muller L, Olivier P, Roth T, von Zadow U (2008) Multi-touch surfaces: A technical guide. Technical report, Technical University of MunichGoogle Scholar
  16. 16.
    Harrison C, Hudson SE (2009) Providing dynamically changeable physical buttons on a visual display. In: Proceedings of the 27th international conference on human factors in computing systems (CHI ’09), ACM Press, New York, pp 299–308Google Scholar
  17. 17.
    Rosenberg ID, Grau A, Hendee C, Awad N, Perlin K (2009) Impad: An inexpensive multi-touchpressure acquisition device. In: CHIProceedings of the 27th international conference extended abstracts on human factors in computing systems (EA ’09), ACM Press, New York, pp 3217–3222Google Scholar
  18. 18.
    Wikipedia (2009) Indium tin oxide — Wikipedia, The Free Encyclopedia, accessed 10.07.2009Google Scholar
  19. 19.
    Downs R (2005) Using resistive touch screens for human/machine interface. Technical Report, Texas Instruments Incorporated, Texas Instruments, Post Office Box 655303, Dallas, Texas 75265Google Scholar
  20. 20.
    Costanza E, Robinson J (2003) A region adjacency tree approach to the detection and design of fiducials. Vision, Video and Graphics (VVG), pp 63–70Google Scholar
  21. 21.
    Schmidt D (2009) Design and realization of an interactive multi-touch table. Technical Report, Lancaster UniversityGoogle Scholar
  22. 22.
    Tuddenham P, Robinson P (2007) Distributed tabletops: Supporting remote and mixed-presence tabletop collaboration. In: Horizontal Interactive Human-Computer Systems, 2007. TABLETOP’07: Proceedings of the 2nd annual IEEE international workshop, pp 19–26Google Scholar
  23. 23.
    Motamedi N (2008) Hd touch: Multi-touch and object sensing on a high definition lcd tv. In: CHI ’08 extended abstracts on human factors in computing systems (CHI ’08), ACM Press, New York, pp 3069–3074Google Scholar
  24. 24.
    Echtler F, Sielhorst T, Huber M, Klinker G (2009) A short guide to modulated light. In: Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI ’09), ACM Press, New York, pp 393–396Google Scholar
  25. 25.
    Han Y, Wagner R (1990) An efficient and fast parallel-connected component algorithm. Journal of the ACM (JACM) 37(3):626–642MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Kaltenbrunner M, Bencina R (2007) reacTIVision: A computer-vision framework for table-based tangible interaction. Proceedings of the 1st international conference on Tangible and embedded interaction pp 69–74Google Scholar
  27. 27.
    Costanza E, Shelley S, Robinson J (2003) Introducing audio d-touch: A tangible user interface for music composition and performance. Proceedings of the 6th IntÕl Conference on Digital Audio Effects (DAFX), pp 63–70Google Scholar
  28. 28.
    Kaltenbrunner M, Bovermann T, Bencina R, Costanza E (2005) TUIO: A protocol for table-top tangible user interfaces. Proceedings of the 6th IntÕl Workshop on Gesture in Human-Computer Interaction and SimulationGoogle Scholar
  29. 29.
    Echtler F (2008) TISCH: Tangible Interactive Surfaces for Collaboration between Humans., accessed 23.09.2008
  30. 30.
    Echtler F, Klinker G (2008) A multitouch software architecture. In: Proceedings of the (NordiCHI ’08), ACM Press, New York, pp 463–466Google Scholar
  31. 31.
    Buxton B (2008) The long nose of innovation., accessed 26.08.2008
  32. 32.
    Daiber F, Schöning J, Krüger A (2009) Whole body interaction with geospatial data. Smart Graphics, pp 81–92Google Scholar

Copyright information

© Springer-Verlag London 2010

Authors and Affiliations

  • Johannes Schöning
      Email author
    • Jonathan Hook
      • 1
    • Tom Bartindale
      • 1
    • Dominik Schmidt
      • 2
    • Patrick Oliver
      • 1
    • Florian Echtler
      • 3
    • Nima Motamedi
      • 4
    • Peter Brandl
      • 5
    • Ulrich von Zadow
      • 6
    1. 1.German Research Center for Artificial Intelligence (DFKI)SaarbrueckenGermany
    2. 2.Culture LabNewcastle UniversityNewcastle upon TyneUK
    3. 3.Lancaster UniversityLancasterUK
    4. 4.Technical University of MunichMunichGermany
    5. 5.Simon Fraser UniversityBurnabyCanada
    6. 6.Media Interaction LabUpper Austria University of Applied SciencesWelsAustria

    Personalised recommendations