When Paper Meets Multi-touch: A Study of Multi-modal Interactions in Air Traffic Control

  • Cheryl Savery
  • Christophe Hurter
  • Rémi Lesbordes
  • Maxime Cordeil
  • T. C. Nicholas Graham
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8119)


For expert interfaces, it is not obvious whether providing multiple modes of interaction, each tuned to different sub-tasks, leads to a better user experience than providing a more limited set. In this paper, we investigate this question in the context of air traffic control. We present and analyze an augmented flight strip board offering several forms of interaction, including touch, digital pen and physical paper objects. We explore the technical challenges of adding finger detection to such a flight strip board and evaluate how expert air traffic controllers interact with the resulting system. We find that users are able to quickly adapt to the wide range of offered modalities. Users were not overburden by the choice of different modalities, and did not find it difficult to determine the appropriate modality to use for each interaction.


Paper computing augmented paper digital pen interactive paper tangible interfaces air traffic control 


  1. 1.
    Aitenbichler, E., Schnelle-Walka, D.: An extensible architecture for multitouch & pen interactive tables. In: Proceedings of EICS (2010)Google Scholar
  2. 2.
    Anoto, digital pen technology,
  3. 3.
  4. 4.
    Brandl, P., Forlines, C., Wigdor, D., Haller, M., Shen, C.: Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. In: Proc. AVI, pp. 154–161. ACM (2008)Google Scholar
  5. 5.
    Brandl, P., Leitner, J., Seifried, T., Haller, M., Doray, B., To, P.: Occlusion aware menu design for digital tabletops. In: Proc. CHI, pp. 3223–3228. ACM (2009)Google Scholar
  6. 6.
    Cohen, P., McGee, D.: Tangible multimodal interfaces for safety-critical applications. Communications of the ACM 47(1), 41–46 (2004)CrossRefGoogle Scholar
  7. 7.
    Do-Lenh, S., Kaplan, F., Sharma, A., Dillenbourg, P.: Multi-finger interactions with papers on augmented tabletops. In: Proc.TEI, pp. 267–274. ACM (2009)Google Scholar
  8. 8.
    Forlines, C., Wigdor, D., Shen, C., Balakrishnan, R.: Direct-touch vs. mouse input for tabletop displays. In: Proc. CHI, pp. 647–656. ACM (2007)Google Scholar
  9. 9.
    Hinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H., Buxton, B.: Pen+ touch= new tools. In: Proc. UIST, pp. 27–36. ACM (2010)Google Scholar
  10. 10.
    Hofer, R., Kunz, A.: Digisketch: taming anoto technology on lcds. In: Proc. 2nd ACM SIGCHI Symposium Engineering Interactive Computing Systems, pp. 103–108. ACM (2010)Google Scholar
  11. 11.
    Holman, D., Vertegaal, R., Altosaar, M., Troje, N., Johns, D.: Paper windows: interaction techniques for digital paper. In: Proc. SIGCHI, pp. 591–599. ACM (2005)Google Scholar
  12. 12.
    Hurter, C., Lesbordes, R., Letondal, C., Vinot, J., Conversy, S.: Strip’TIC: exploring augmented paper strips for air traffic controllers. In: Proc. AVI, pp. 225–232. ACM (2012)Google Scholar
  13. 13.
    Ishii, H.: Tangible bits: beyond pixels. In: Proc. TEI, pp. xv–xxv. ACM (2008)Google Scholar
  14. 14.
    Letondal, C., Hurter, C., Vinot, J., Lesbordes, R., Conversy, S.: Strip’TIC: Designing a paper-based tangible interactive space for air traffic controllers. In: Proc. CHI. ACM (2013)Google Scholar
  15. 15.
    Liwicki, M., Rostanin, O., El-Neklawy, S., Dengel, A.: Touch & write-a multi-touch table with pen-input. In: 9th Int. Workshop on Document Analysis Systems, pp. 479–484 (2010)Google Scholar
  16. 16.
    MacKay, W.: Is paper safer? The role of paper flight strips in air traffic control. ACM Transactions on Computer-Human Interaction (TOCHI) 6(4), 311–340 (1999)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Mackay, W., Pothier, G., Letondal, C., Bøegh, K., Sørensen, H.: The missing link: augmenting biology laboratory notebooks. In: Proc. UIST, pp. 41–50. ACM (2002)Google Scholar
  18. 18.
    Morris, M., Huang, A., Paepcke, A., Winograd, T.: Cooperative gestures: multi-user gestural interactions for co-located groupware. In: Proc. CHI, pp. 1201–1210. ACM (2006)Google Scholar
  19. 19.
    Paelke, V., Sester, M.: Augmented paper maps: Exploring the design space of a mixed reality system. Journal of Photogrammetry and Remote Sensing 65(3), 256–265 (2010)CrossRefGoogle Scholar
  20. 20.
    Ringel, M., Ryall, K., Shen, C., Forlines, C., Vernier, F.: Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables. In: Proc. CHI, pp. 1441–1444. ACM (2004)Google Scholar
  21. 21.
    Song, H., Guimbretiere, F., Grossman, T., Fitzmaurice, G.: Mouselight: bi-manual interactions on digital paper using a pen and a spatially-aware mobile projector. In: Proc. CHI, pp. 2451–2460. ACM (2010)Google Scholar
  22. 22.
    Steimle, J.: Survey of pen-and-paper computing. In: Pen-and-Paper User Interfaces, Human Computer Interaction Series, pp. 19–65. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  23. 23.
    Taub, D.: The BoPen: a tangible pointer tracked in six degrees of freedom. PhD thesis, Massachusetts Institute of Technology (2009)Google Scholar
  24. 24.
    Teiche, A., Rai, A., Yanc, C., Moore, C., Solms, D., Cetin, G., Riggio, J., Ramseyer, N., Dintino, P., Muller, L.: Multi-touch technologies. NUI Group (2009)Google Scholar
  25. 25.
    Tsandilas, T.: Interpreting strokes on paper with a mobile assistant. In: Proc. UIST (2012)Google Scholar
  26. 26.
    Walny, J., Lee, B., Johns, P., Riche, N., Carpendale, S.: Understanding pen and touch interaction for data exploration on interactive whiteboards. IEEE Transactions on Visualization and Computer Graphics (2012)Google Scholar
  27. 27.
    Wigdor, D.: Architecting next-generation user interfaces. In: Proc. AVI, pp. 16–22. ACM (2010)Google Scholar
  28. 28.
    Wu, M., Shen, C., Ryall, K., Forlines, C., Balakrishnan, R.: Gesture registration, relaxation, and reuse for multi-point direct-touch surfaces. In: TableTop, pp. 183–190. IEEE (2006)Google Scholar
  29. 29.
    Yee, K.: Two-handed interaction on a tablet display. In: Proc. CHI, pp. 1493–1496. ACM (2004)Google Scholar
  30. 30.
    Yeh, R., Paepcke, A., Klemmer, S.: Iterative design and evaluation of an event architecture for pen-and-paper interfaces. In: Proc. UIST, pp. 111–120. ACM (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Cheryl Savery
    • 1
    • 3
  • Christophe Hurter
    • 3
    • 4
  • Rémi Lesbordes
    • 2
  • Maxime Cordeil
    • 2
    • 3
    • 4
  • T. C. Nicholas Graham
    • 1
  1. 1.School of ComputingQueen’s UniversityKingstonCanada
  2. 2.DGAC DSNA DTI R&DToulouseFrance
  3. 3.ENACToulouseFrance
  4. 4.IRITUniversité de ToulouseToulouse cedexFrance

Personalised recommendations