Interaction with 3D Environments Using Multi-Touch Screens

  • Francisco Ortego
  • Naphtali Rishe
  • Armando Barreto
  • Melek Adjouadi
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 152)

Abstract

The increase in availability of multi-touch devices has motivated us to consider interaction approaches outside the limitations associated with the use of a mouse. The problem that we try to solve is how to interact in a 3D world using a 2D surface multi-touch display. Before showing our proposed solution, we briefly review previous work in related fields that provided a framework for the development of our approach. Finally, we propose a set of multi-touch gestures and outline an experiment design for the evaluation of these forms of interaction.

Keywords

Multi-touch 3D interaction 3D navigation Gestures 

Notes

Acknowledgments

This work was sponsored by NSF grants HRD-0833093, and CNS-0959985. Mr. Francisco Ortega is the recipient of a GAANN fellowship, from the US Department of Education, at Florida International University.

References

  1. 1.
    Nielson G, Olsen D Jr (1987) Direct manipulation techniques for 3D objects using 2D locator devices. In: Proceedings of the 1986 workshop on Interactive 3D graphics, pp 175–182Google Scholar
  2. 2.
    Chen M, Mountford S, Sellen A (1988) A study in interactive 3-D rotation using 2-D control devices. ACM SIGGRAPH Comput Graph 22(4):129CrossRefGoogle Scholar
  3. 3.
    Jacob R, Girouard A, Hirshfield L, Horn MS, Shaer O, Solovey ET, Zigelbaum J (2008) Reality-based interaction: a framework for post-WIMP interfaces. In: Proceeding of the twenty-sixth annual SIGCHI conference on human factors in computing systems (CHI ’08), pp 201–210Google Scholar
  4. 4.
    Wang F, Cao X, Ren X, Irani P (2009) Detecting and leveraging finger orientation for interaction with direct-touch surfaces. In: Proceedings of the 22nd annual ACM symposium on user interface software and technology, pp 23–32Google Scholar
  5. 5.
    Wang F, Ren X (2009) Empirical evaluation for finger input properties in multi-touch interaction. In: Proceedings of the 27th international conference on human factors. ACM, BostonGoogle Scholar
  6. 6.
    Wilson A, Izadi S, Hilliges O, Garcia-Mendoza A, Kirk D (2008) Bringing physics to the surface. In: Proceedings of the 21st annual ACM symposium on user interface software and technology, pp 67–76Google Scholar
  7. 7.
    Cao X, Wilson A, Balakrishnan R, Hinckley K, Hudson S (2008) ShapeTouch: leveraging contact shape on interactive surfaces. In: TABLETOP 2008. 3rd IEEE international workshop on horizontal interactive human computer system 2008, pp 129–136Google Scholar
  8. 8.
    Benko H, Wilson A, Baudisch P (2006) Precise selection techniques for multi-touch screens. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ’06), pp 1263–1272Google Scholar
  9. 9.
    Nacenta MA, Baudisch P, Benko H, Wilson A (2009) Separability of spatial manipulations in multi-touch interfaces. In: GI ’09 proceedings of graphics interface 2009. Canadian Information Processing Society, Toronto, May 2009Google Scholar
  10. 10.
    Hancock M, Carpendale S, Cockburn A (2007) Shallow-depth 3D interaction: design and evaluation of one-, two-and three-touch techniques. In: Proceedings of the SIGCHI conference on human factors in computing systems, p 1156Google Scholar
  11. 11.
    Kin K, Agrawala M, DeRose T (2009) Determining the benefits of direct-touch, bimanual, and multifinger input on a multitouch workstation. Canadian Information Processing Society, Toronto, May 2009Google Scholar
  12. 12.
    Moscovich T, Hughes J (2008) Indirect mappings of multi-touch input using one and two hands. In: Proceeding of the twenty-sixth annual SIGCHI conference on human factors in computing systems (CHI ’08), pp 1275–1284Google Scholar
  13. 13.
    Glassner AS (1993) Graphics gems. Morgan Kaufmann, San FranciscoGoogle Scholar
  14. 14.
    Arvo J (1994) Graphics gems II. Morgan Kaufmann, San FranciscoGoogle Scholar
  15. 15.
    Heckbert PS (1994) Graphics gems IV. Morgan Kaufmann, San FranciscoGoogle Scholar
  16. 16.
    Bowman DA (2005) 3D user interfaces: theory and practice. Addison-Wesley, Boston, p 478Google Scholar
  17. 17.
    Sezgin T, Davis R (2005) HMM-based efficient sketch recognition. In: Proceedings of the 10th international conference on intelligent user interfaces (IUI ’05)Google Scholar
  18. 18.
    Hong P, Huang T (2000) Constructing finite state machines for fast gesture recognition. In: 15th international conference on pattern recognition (ICPR’00), vol 3, p 3695Google Scholar
  19. 19.
    Hong P, Huang T, Turk M (2000) Gesture modeling and recognition using finite state machines. In: IEEE conference on face and gesture recognition, Mar 2000Google Scholar
  20. 20.
    Pittman J (1991) Recognizing handwritten text. In: Human factors in computing systems: reaching through technology (CHI ’91), New York, pp 271–275Google Scholar
  21. 21.
    Rubine D (1991) Specifying gestures by example. ACM SIGGRAPH Comput Graph 25(4):329–337CrossRefGoogle Scholar
  22. 22.
    MacLean S, Labahn G (2010) Elastic matching in linear time and constant space. In: International workshop on document analysis systems 2010 (DAS ’10)Google Scholar
  23. 23.
    Kara L, Stahovich T (2005) An image-based, trainable symbol recognizer for hand-drawn sketches. Comput Graph 29(4):501–517CrossRefGoogle Scholar
  24. 24.
    Notowidigdo M, Miller R (2004) Off-line sketch interpretation. In: AAAI fall symposium, pp 120–126Google Scholar
  25. 25.
    Wobbrock J, Wilson A (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology (UIST ’07)Google Scholar
  26. 26.
    Anthony L, Wobbrock J (2010) A lightweight multistroke recognizer for user interface prototypes. In: Proceedings of graphics interface 2010 (GI’10), TorontoGoogle Scholar
  27. 27.
    Li Y (2010) Protractor: a fast and accurate gesture recognizer. In: Proceedings of the 28th international conference on human factors in computing systems (CHI ’10), New YorkGoogle Scholar
  28. 28.
    Johnson G, Gross M, Hong J (2009) Computational support for sketching in design: a review. Found Trends Human–Comput Inter 2: 1–93Google Scholar
  29. 29.
    Press WH, Flannery BP, Teukolsky SA, Vetterling WT (2007) Numerical recipes, 3rd edn. The art of scientific computing. Cambridge University Press, Hong KongGoogle Scholar
  30. 30.
    Kruger R, Carpendale S, Scott S, Tang A (2005) Fluid integration of rotation and translation. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 601–610Google Scholar
  31. 31.
    Reisman J, Davidson P, Han J (2009) A screen-space formulation for 2D and 3D direct manipulation. In: Proceedings of the 22nd annual ACM symposium on User interface software and technology, pp 69–78Google Scholar
  32. 32.
    Wright RS, Haemel N, Sellers G, Lipchak B (2010) OpenGL superbible. Comprehensive tutorial and reference. Addison-Wesley, ReadingGoogle Scholar
  33. 33.
    Han J, Kim J (2011) 3D graphics for game programming. Chapman and Hall, LondonGoogle Scholar
  34. 34.
    Kiriaty Y, Moroney L, Goldshtein S, Fliess A (2009) Introducing windows 7 for developers. Microsoft Press, RedmondGoogle Scholar
  35. 35.
    Laufs U, Ruff C, Zibuschka J (2010) MT4j-a cross-platform multi-touch development framework. In: ACM EICS 2010, workshop: engineering patterns for multi-touch interfaces, pp 52–57Google Scholar
  36. 36.
    O’Brien T, Keefe D, Laidlaw D (2008) A case study in using gestures and bimanual interaction to extend a high-DOF input device. In: Proceedings of the 2008 symposium on interactive 3D graphics and games (I3D ’08), New YorkGoogle Scholar
  37. 37.
    Lazar J, Jinjuan Heidi Feng D, Harry Hochheiser D (2010) Research methods in human–computer interaction. Wiley, New YorkGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Francisco Ortego
    • 1
  • Naphtali Rishe
    • 1
  • Armando Barreto
    • 2
  • Melek Adjouadi
    • 2
  1. 1.School of Computing and Information SciencesFlorida International UniversityMiamiUSA
  2. 2.Electrical and Computer Engineering DepartmentFlorida International UniversityMiamiUSA

Personalised recommendations