Skip to main content

VRSketch: Investigating 2D Sketching in Virtual Reality with Different Levels of Hand and Pen Transparency

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2021 (INTERACT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12936))

Included in the following conference series:

Abstract

Sketching is a vital step in design processes. While analog sketching on pen and paper is the defacto standard, Virtual Reality (VR) seems promising for improving the sketching experience. It provides myriads of new opportunities to express creative ideas. In contrast to reality, possible drawbacks of pen and paper drawing can be tackled by altering the virtual environment. In this work, we investigate how hand and pen transparency impacts users’ 2D sketching abilities. We conducted a lab study (\(N=20\)) investigating different combinations of hand and pen transparency. Our results show that a more transparent pen helps one sketch more quickly, while a transparent hand slows down. Further, we found that transparency improves sketching accuracy while drawing in the direction that is occupied by the user’s hand.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Google Tilt Brush. https://www.tiltbrush.com, last retrieved August 12, 2021.

  2. 2.

    Gravity Sketch. https://www.gravitysketch.com, last retrieved August 12, 2021.

  3. 3.

    https://github.com/rarora7777/VRSketchingStudyCHI17.

  4. 4.

    Job Simulator. https://store.steampowered.com/app/448280/Job_Simulator, last retrieved August 12, 2021.

References

  1. Annett, M., Bischof, W.F.: Hands, hover, and nibs: understanding stylus accuracy on tablets. In: Proceedings of the 41st Graphics Interface Conference, GI 2015, Toronto, ON, Canada, pp. 203–210. Canadian Information Processing Society (2015). http://dl.acm.org/citation.cfm?id=2788890.2788926. Event-place: Halifax, Nova Scotia, Canada

  2. Arora, R., Kazi, R.H., Anderson, F., Grossman, T., Singh, K., Fitzmaurice, G.: Experimental evaluation of sketching on surfaces in VR. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI 2017, New York, NY, USA, pp. 5643–5654. ACM (2017). https://doi.org/10.1145/3025453.3025474. Event-place: Denver, Colorado, USA

  3. Bowman, D.A., Rhoton, C.J., Pinho, M.S.: Text input techniques for immersive virtual environments: an empirical comparison. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, vol. 46, pp. 2154–2158. SAGE Publications (2002). https://doi.org/10.3390/technologies7020031

  4. Drey, T., Gugenheimer, J., Karlbauer, J., Milo, M., Rukzio, E.: VRSketchin: exploring the design space of pen and tablet interaction for 3D sketching in virtual reality. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI 2020, New York, NY, USA, pp. 1–14. Association for Computing Machinery (2020). https://doi.org/10.1145/3313831.3376628

  5. Elsayed, H., et al.: VRSketchpen: unconstrained haptic assistance for sketching in virtual 3d environments. In: 26th ACM Symposium on Virtual Reality Software and Technology, VRST 2020, New York, NY, USA. Association for Computing Machinery (2020). https://doi.org/10.1145/3385956.3418953

  6. Gerry, L.J.: Paint with me: stimulating creativity and empathy while painting with a painter in virtual reality. IEEE Trans. Vis. Comput. Graph. 23(4), 1418–1426 (2017). https://doi.org/10.1109/TVCG.2017.2657239

    Article  Google Scholar 

  7. Gesslein, T., et al.: Pen-based interaction with spreadsheets in mobile virtual reality. arXiv:2008.04543 [cs], August 2020

  8. González, G., Molina, J.P., García, A.S., Martínez, D., González, P.: Evaluation of text input techniques in immersive virtual environments. In: Macías, J., Granollers Saltiveri, A., Latorre, P. (eds.) New Trends on Human-Computer Interaction, pp. 109–118. Springer, London (2009). https://doi.org/10.1007/978-1-84882-352-5_11

    Chapter  Google Scholar 

  9. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Advances in Psychology, vol. 52, pp. 139–183. Elsevier (1988). https://doi.org/10.1016/S0166-4115(08)62386-9. https://linkinghub.elsevier.com/retrieve/pii/S0166411508623869

  10. Holman, D., Vertegaal, R., Altosaar, M., Troje, N., Johns, D.: Paper windows: interaction techniques for digital paper. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 591–599. ACM (2005). https://doi.org/10.1145/1054972.1055054

  11. Jackson, B., Keefe, D.F.: Lift-off: using reference imagery and freehand sketching to create 3D models in VR. IEEE Trans. Vis. Comput. Graph. 22(4), 1442–1451 (2016). https://doi.org/10.1109/TVCG.2016.2518099

    Article  Google Scholar 

  12. Jetter, H.C., Rädle, R., Feuchtner, T., Anthes, C., Friedl, J., Klokmose, C.N.: “In VR, everything is possible!”: sketching and simulating spatially-aware interactive spaces in virtual reality. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI 2020, New York, NY, USA, pp. 1–16. Association for Computing Machinery (2020). https://doi.org/10.1145/3313831.3376652

  13. Knierim, P., Schwind, V., Feit, A.M., Nieuwenhuizen, F., Henze, N.: Physical keyboards in virtual reality: analysis of typing performance and effects of avatar hands. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, p. 345. ACM (2018). https://doi.org/10.1145/3173574.3173919

  14. Lee, D., Son, K., Lee, J.H., Bae, S.H.: PhantomPen: virtualization of pen head for digital drawing free from pen occlusion & #38; visual parallax. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST 2012, New York, NY, USA, pp. 331–340. ACM (2012). https://doi.org/10.1145/2380116.2380159. Event-place: Cambridge, Massachusetts, USA

  15. Lee, H., Ko, I., Tan, Y.R., Zhang, D., Li, C.: Usability impact of occlusion-free techniques on commonly-used multitouch actions. In: Proceedings of the 5th International ACM In-Cooperation HCI and UX Conference, CHIuXiD 2019, New York, NY, USA, pp. 96–105. ACM (2019). https://doi.org/10.1145/3328243.3328256. Event-place: Jakarta, Surabaya, Bali, Indonesia

  16. Li, Z., Annett, M., Hinckley, K., Singh, K., Wigdor, D.: HoloDoc: enabling mixed reality workspaces that harness physical and digital content. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, New York, NY, USA, pp. 687:1–687:14. ACM (2019). https://doi.org/10.1145/3290605.3300917. Event-place: Glasgow, Scotland, UK

  17. Pham, D.M., Stuerzlinger, W.: Is the pen mightier than the controller? A comparison of input devices for selection in virtual and augmented reality. In: 25th ACM Symposium on Virtual Reality Software and Technology, pp. 1–11. ACM, Parramatta NSW Australia, November 2019. https://doi.org/10.1145/3359996.3364264

  18. Poupyrev, I., Tomokazu, N., Weghorst, S.: Virtual notepad: handwriting in immersive VR. In: Proceedings of the IEEE 1998 Virtual Reality Annual International Symposium (Cat. No. 98CB36180), pp. 126–132. IEEE (1998). https://doi.org/10.1109/VRAIS.1998.658467

  19. Sandnes, F.E.: Panoramagrid: a graph paper tracing framework for sketching 360-degree immersed experiences. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, AVI 2016, New York, NY, USA, pp. 342–343. Association for Computing Machinery (2016). https://doi.org/10.1145/2909132.2926058

  20. Schrepp, M., Hinderks, A., Thomaschewski, J.: Design and evaluation of a short version of the user experience questionnaire (UEQ-S). Int. J. Interact. Multimed. Artif. Intell. 4(6), 103 (2017)

    Google Scholar 

  21. Schubert, T., Friedmann, F., Regenbrecht, H.: The experience of presence: factor analytic insights. Presence Teleoper. Virtual Environ. 10(3), 266–281 (2001). https://doi.org/10.1162/105474601300343603

    Article  Google Scholar 

  22. Song, H., Guimbretiere, F., Grossman, T., Fitzmaurice, G.: MouseLight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2010, New York, NY, USA, pp. 2451–2460. ACM (2010). https://doi.org/10.1145/1753326.1753697. Event-place: Atlanta, Georgia, USA

  23. Vogel, D., Baudisch, P.: Shift: a technique for operating pen-based interfaces using touch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2007, New York, NY, USA, pp. 657–666. Association for Computing Machinery (2007). https://doi.org/10.1145/1240624.1240727

  24. Vogel, D., Cudmore, M., Casiez, G., Balakrishnan, R., Keliher, L.: Hand occlusion with tablet-sized direct pen input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2009, New York, NY, USA, pp. 557–566. ACM (2009). https://doi.org/10.1145/1518701.1518787. Event-place: Boston, MA, USA

  25. Wacker, P., Nowak, O., Voelker, S., Borchers, J.: Evaluating menu techniques for handheld AR with a smartphone & mid-air pen. In: 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2020, New York, NY, USA. Association for Computing Machinery (2020). https://doi.org/10.1145/3379503.3403548

  26. Wacker, P., Wagner, A., Voelker, S., Borchers, J.: Physical guides: an analysis of 3D sketching performance on physical objects in augmented reality. In: Proceedings of the Symposium on Spatial User Interaction, SUI 2018, New York, NY, USA, pp. 25–35. ACM (2018). https://doi.org/10.1145/3267782.3267788. Event-place: Berlin, Germany

  27. Walker, J., Li, B., Vertanen, K., Kuhl, S.: Efficient typing on a visually occluded physical keyboard. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 5457–5461. ACM (2017). https://doi.org/10.1145/3025453.3025783

  28. Wellner, P.D.: Interacting with paper on the DigitalDesk. Technical report, University of Cambridge, Computer Laboratory (1994). https://doi.org/10.1145/159544.159630

  29. Zhai, S., Buxton, W., Milgram, P.: The partial-occlusion effect: utilizing semitransparency in 3D human-computer interaction. ACM Trans. Comput. Hum. Interact. 3(3), 254–284 (1996). https://doi.org/10.1145/234526.234532

    Article  Google Scholar 

  30. Zhang, Y., et al.: Sensing posture-aware Pen+Touch interaction on tablets. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, New York, NY, USA, pp. 55:1–55:14. ACM (2019). https://doi.org/10.1145/3290605.3300285. Event-place: Glasgow, Scotland, UK

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jonas Auda .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 85419 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Auda, J., Heger, R., Gruenefeld, U., Schneegass, S. (2021). VRSketch: Investigating 2D Sketching in Virtual Reality with Different Levels of Hand and Pen Transparency. In: Ardito, C., et al. Human-Computer Interaction – INTERACT 2021. INTERACT 2021. Lecture Notes in Computer Science(), vol 12936. Springer, Cham. https://doi.org/10.1007/978-3-030-85607-6_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85607-6_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85606-9

  • Online ISBN: 978-3-030-85607-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics