Advertisement

Computational Visual Media

, Volume 4, Issue 2, pp 139–147 | Cite as

EZ-Manipulator: Designing a mobile, fast, and ambiguity-free 3D manipulation interface using smartphones

  • Po-Huan Tseng
  • Shih-Hsuan HungEmail author
  • Pei-Ying Chiang
  • Chih-Yuan Yao
  • Hung-Kuo Chu
Open Access
Research Article

Abstract

Interacting with digital contents in 3D is an essential task in various applications such as modeling packages, gaming, virtual reality, etc. Traditional interfaces using keyboard and mouse or trackball usually require a non-trivial amount of working space as well as a learning process. We present the design of EZ-Manipulator, a new 3D manipulation interface using smartphones that supports mobile, fast, and ambiguity-free interaction with 3D objects. Our system leverages the built-in multi-touch input and gyroscope sensor of smartphones to achieve 9 degrees-of-freedom axis-constrained manipulation and free-form rotation. Using EZ-Manipulator to manipulate objects in 3D is easy. The user merely has to perform intuitive singleor two-finger gestures and rotate the hand-held device to perform manipulations at fine-grained and coarse levels respectively.We further investigate the ambiguity in manipulation introduced by indirect manipulations using a multi-touch interface, and propose a dynamic virtual camera adjustment to effectively resolve the ambiguity. A preliminary study shows that our system has significant lower task completion time compared to conventional use of a keyboard–mouse interface, and provides a positive user experience to both novices and experts.

Keywords

3D manipulation interface interaction technique multi-touch gyroscope sensor mobile device input device 

Supplementary material

EZ-Manipulator Designing a Mobile, Fast, and Ambiguity-Free 3D Manipulation Interface using Smartphones

References

  1. [1]
    Chen, M.; Mountford, S. J.; Sellen, A. A study in interactive 3-D rotation using 2-D control devices. In: Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques, 121–129, 1988.Google Scholar
  2. [2]
    Henriksen, K.; Sporring, J.; Hornbæk, K. Virtual trackballs revisited. IEEE Transactions on Visualization and Computer Graphics Vol. 10, No. (2), 206–216, 2004.CrossRefGoogle Scholar
  3. [3]
    Bier, E. A. Skitters and jacks: Interactive 3D positioning tools. In: Proceedings of the 1986 Workshop on Interactive 3D graphics, 183–196, 1987.CrossRefGoogle Scholar
  4. [4]
    Conner, B. D.; Snibbe, S. S.; Herndon, K. P.; Robbins, D. C.; Zeleznik, R. C.; van Dam, A. Threedimensional widgets. In: Proceedings of the Symposium on Interactive 3D Graphics, 183–188, 1992.Google Scholar
  5. [5]
    Ware, C.; Jessome, D. R. Using the bat: A six dimensional mouse for object placement. IEEE Computer Graphics and Applications Vol. 8, No. (6), 65–70, 1988.CrossRefGoogle Scholar
  6. [6]
    Zimmerman, T. G.; Lanier, J.; Blanchard, C.; Bryson, S.; Harvill, Y. A hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, 189–192, 1986.Google Scholar
  7. [7]
    Froehlich, B.; Hochstrate, J.; Skuk, V.; Huckauf, A. The GlobeFish and the GlobeMouse: Two new six degree of freedom input devices for graphics applications. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 191–199, 2006.Google Scholar
  8. [8]
    Perelman, G.; Serrano, M.; Raynal, M.; Picard, C.; Derras, M.; Dubois, E. The roly-poly mouse: Designing a rolling input device unifying 2D and 3D interaction. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 327–336, 2015.Google Scholar
  9. [9]
    Wang, R. Y.; Popović, J. Real-time hand-tracking with a color glove. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 63, 2009.Google Scholar
  10. [10]
    Wang, R.; Paris, S.; Popović, J. 6D hands: Markerless hand-tracking for computer aided design. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 549–558, 2011.Google Scholar
  11. [11]
    Song, P.; Goh, W. B.; Hutama, W.; Fu, C.-W.; Liu, X. A handle bar metaphor for virtual object manipulation with mid-air interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1297–1306, 2012.Google Scholar
  12. [12]
    Hancock, M.; Carpendale, S.; Cockburn, A. Shallowdepth 3d interaction: Design and evaluation of one-, two- and three-touch techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1147–1156, 2007.CrossRefGoogle Scholar
  13. [13]
    Hancock, M.; ten Cate, T.; Carpendale, S. Sticky tools: Full 6DOF force-based interaction for multi-touch tables. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, 133–140, 2009.CrossRefGoogle Scholar
  14. [14]
    Reisman, J. L.; Davidson, P. L.; Han, J. Y. A screenspace formulation for 2D and 3D direct manipulation. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, 69–78, 2009.Google Scholar
  15. [15]
    Martinet, A.; Casiez, G.; Grisoni, L. The effect of DOF separation in 3D manipulation tasks with multi-touch displays. In: Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology, 111–118, 2010.CrossRefGoogle Scholar
  16. [16]
    Martinet, A.; Casiez, G.; Grisoni, L. Integrality and separability of multitouch interaction techniques in 3D manipulation tasks. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. (3), 369–380, 2012.CrossRefGoogle Scholar
  17. [17]
    Liu, J.; Au, O.K.-C.; Fu, H.; Tai, C.-L. Two-finger gestures for 6DOF manipulation of 3D objects. Computer Graphics Forum Vol. 31, No. (7), 2047–2055, 2012.CrossRefGoogle Scholar
  18. [18]
    Au, O.K.-C.; Tai, C.-L.; Fu, H. Multitouch gestures for constrained transformation of 3D objects. Computer Graphics Forum Vol. 31, No. 2pt3, 651–660, 2012.Google Scholar
  19. [19]
    Hachet, M.; Bossavit, B.; Cohé, A.; de la Rivière, J.-B. Toucheo: Multitouch and stereo combined in a seamless workspace. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 587–592, 2011.Google Scholar
  20. [20]
    Schmidt, R.; Singh, K.; Balakrishnan, R. Sketching and composing widgets for 3D manipulation. Computer Graphics Forum Vol. 27, No. (2), 301–310, 2008.CrossRefGoogle Scholar
  21. [21]
    Cohé, A.; Décle, F.; Hachet, M. tBox: A 3d transformation widget designed for touch-screens. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 3005–3008, 2011.Google Scholar
  22. [22]
    Kin, K.; Miller, T.; Bollensdorff, B.; DeRose, T.; Hartmann, B.; Agrawala, M. Eden: A professional multitouch tool for constructing virtual organic environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1343–1352, 2011.Google Scholar
  23. [23]
    Sun, Q.; Lin, J.; Fu, C.-W.; Kaijima, S.; He, Y. A multi-touch interface for fast architectural sketching and massing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 247–256, 2013.CrossRefGoogle Scholar
  24. [24]
    Paczkowski, P.; Dorsey, J.; Rushmeier, H.; Kim, M. H. Paper3D: Bringing casual 3D modeling to a multi-touch interface. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, 23–32, 2014.Google Scholar
  25. [25]
    Chen, C.-Y.; Yao, C.-Y.; Lin, C.-Y.; Hung, S.-H. Realtime remote control of artificial cilia actuation using fingertip drawing for efficient micromixing. Journal of Laboratory Automation Vol. 19, No. (5), 492–497, 2014.CrossRefGoogle Scholar
  26. [26]
    Hilliges, O.; Kim, D.; Izadi, S.; Weiss, M.; Wilson, A. HoloDesk: Direct 3d interactions with a situated see-through display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2421–2430, 2012.Google Scholar
  27. [27]
    Nancel, M.; Wagner, J.; Pietriga, E.; Chapuis, O.; Mackay, W. Mid-air pan-and-zoom on wall-sized displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 177–186, 2011.Google Scholar
  28. [28]
    Microsoft. Kinect for Xbox360. 2010. Available at https://www.xbox.com/enUS/kinect.Google Scholar
  29. [29]
    Chapuis, O.; Bezerianos, A.; Frantzeskakis, S. Smarties: An input system for wall display development. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2763–2772, 2014.Google Scholar
  30. [30]
    Song, J.; Sörös, G.; Pece, F.; Fanello, S. R.; Izadi, S.; Keskin, C.; Hilliges, O. In-air gestures around unmodified mobile devices. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, 319–329, 2014.Google Scholar
  31. [31]
    Boring, S.; Gehring, S.; Wiethoff, A.; Blöckner, A. M.; Schöning, J.; Butz, A. Multi-user interaction on media facades through live video on mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2721–2724, 2011.Google Scholar
  32. [32]
    Boring, S.; Baur, D.; Butz, A.; Gustafson, S.; Baudisch, P. Touch projector: Mobile interaction through video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2287–2296, 2010.Google Scholar
  33. [33]
    Chang, Y.-C.; Wang, H.-C.; Chu, H.-K.; Lin, S.-T.; Wang, S.-P. AlphaRead: Support unambiguous referencing in remote collaboration with readable object annotation. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing, 2246–2259, 2017.Google Scholar
  34. [34]
    Kashiwakuma, J.; Kitahara, I.; Kameda, Y.; Ohta, Y. A virtual camera controlling method using multitouch gestures for capturing free-viewpoint video. In: Proceedings of the 11th European Conference on Interactive TV and Video, 67–74, 2013.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Authors and Affiliations

  • Po-Huan Tseng
    • 1
  • Shih-Hsuan Hung
    • 2
    Email author
  • Pei-Ying Chiang
    • 3
  • Chih-Yuan Yao
    • 2
  • Hung-Kuo Chu
    • 1
  1. 1.“National Tsing Hua University”TaiwanChina
  2. 2.“National Taiwan University of Science and Technology”TaiwanChina
  3. 3.“National Taipei University of Technology”TaiwanChina

Personalised recommendations