CHARM: Cord-Based Haptic Augmented Reality Manipulation

  • Konstantin KlamkaEmail author
  • Patrick Reipschläger
  • Raimund Dachselt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11574)


The recent trend of emerging high-quality Augmented Reality (AR) glasses offered the possibility for visually exciting application scenarios. However, the interaction with these devices is often challenging since current input methods most of the time lack haptic feedback and are limited in their user interface controls. With this work, we introduce Open image in new window , a combination of a belt-worn interaction device, utilizing a retractable cord, and a set of interaction techniques to enhance AR input capabilities with physical controls and spatial constraints. Building on our previous research, we created a fully-functional prototype to investigate how body-worn string devices can be used to support generic AR tasks. We contribute a radial widget menu for system control as well as transformation techniques for 3D object manipulation. To validate our interaction concepts for system control, we implemented a mid-air gesture interface as a baseline and evaluated our prototype in two formative user studies. Our results show that our approach provides flexibility regarding possible interaction mappings and was preferred for manipulation tasks compared to mid-air gesture input.


Augmented reality Haptic feedback Elastic input Cord input Radial menu 3D interaction 3D transformation Wearable computing 



We would like to thank Andreas Peetz for helping us to improve the CHARM handle and Thomas Schwab as well as Paul Riedel for working on the AR menu and gesture control. This work was partly funded by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) as part of Germany’s Excellence Strategy – EXC 2050/1 – Project ID 390696704 – Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop” (CeTI) of Technische Universität Dresden.


  1. 1.
    Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21(6), 34–47 (2001). Scholar
  2. 2.
    Blasko, G., Narayanaswami, C., Feiner, S.: Prototyping retractable string-based interaction techniques for dual-display mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 369–372. ACM, New York (2006).
  3. 3.
    Bowman, D., Wingrave, C.: Design and evaluation of menu systems for immersive virtual environments. In: Proceedings IEEE Virtual Reality 2001, pp. 149–156 (2001).
  4. 4.
    Bowman, D.A., Kruijff, E., LaViola, J.J., Poupyrev, I.: 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc., Redwood City (2004)Google Scholar
  5. 5.
    Coquillart, S., Wesche, G.: The virtual palette and the virtual remote control panel: a device and an interaction paradigm for the responsive workbench(tm). In: Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pp. 213–216, March 1999.
  6. 6.
    Dachselt, R., Hübner, A.: A survey and taxonomy of 3D menu techniques. In: Eurographics Symposium on Virtual Environments. The Eurographics Association (2006).
  7. 7.
    Davis, M.M., Gabbard, J.L., Bowman, D.A., Gracanin, D.: Depth-based 3D gesture multi-level radial menu for virtual object manipulation. In: 2016 IEEE Virtual Reality (VR), pp. 169–170, March 2016.
  8. 8.
    Dobbelstein, D., Winkler, C., Haas, G., Rukzio, E.: PocketThumb: a wearable dual-sided touch interface for cursor-based control of smart-eyewear. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 1(2), 9:1–9:17 (2017). Scholar
  9. 9.
    Fröhlich, B., Plate, J.: The cubic mouse: a new device for three-dimensional input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2000, pp. 526–531. ACM, New York (2000).
  10. 10.
    Gebhardt, S., Pick, S., Leithold, F., Hentschel, B., Kuhlen, T.: Extended pie menus for immersive virtual environments. IEEE Trans. Visual. Comput. Graph. 19(4), 644–651 (2013). Scholar
  11. 11.
    Hart, S.G.: NASA-task load index (NASA-TLX); 20 years later. Proc. Hum. Factors Ergonomics Soc. Annu. Meet. 50(9), 904–908 (2006). Scholar
  12. 12.
    Hinckley, K., Pausch, R., Goble, J.C., Kassell, N.F.: A survey of design issues in spatial input. In: Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology, UIST 1994, pp. 213–222. ACM, New York (1994).
  13. 13.
    Hyeongmook, L., Dongchul, K., Woontack, W.: Graphical menus using a mobile phone for wearable AR systems. In: 2011 International Symposium on Ubiquitous Virtual Reality, pp. 55–58, July 2011.
  14. 14.
    Klamka, K., Dachselt, R.: Elasticcon: elastic controllers for casual interaction. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2015, pp. 410–419. ACM, New York (2015).
  15. 15.
    Klamka, K., Dachselt, R.: ARCord: visually augmented interactive cords for mobile interaction. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA 2018, pp. LBW623:1–LBW623:6. ACM, New York (2018).
  16. 16.
    Koch, E., Witt, H.: Prototyping a chest-worn string-based wearable input device. In: 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks, pp. 1–6, June 2008.
  17. 17.
    Lee, H., Woo, W.: Tangible spin cube for 3D ring menu in real space. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems, CHI EA 2010, pp. 4147–4152. ACM, New York (2010).
  18. 18.
    Olwal, A., Moeller, J., Priest-Dorman, G., Starner, T., Carroll, B.: I/O braid: scalable touch-sensitive lighted cords using spiraling, repeating sensing textiles and fiber optics. In: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, UIST 2018, pp. 485–497. ACM, New York (2018).
  19. 19.
    Piekarski, W., Thomas, B.H.: The tinmith system: demonstrating new techniques for mobile augmented reality modelling. In: Proceedings of the Third Australasian Conference on User Interfaces, AUIC 2002, vol. 7, pp. 61–70. Australian Computer Society Inc., Darlinghurst (2002).
  20. 20.
    Pohl, N., Hodges, S., Helmes, J., Villar, N., Paek, T.: An interactive belt-worn badge with a retractable string-based input mechanism. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2013, pp. 1465–1468. ACM, New York (2013).
  21. 21.
    Schwarz, J., Harrison, C., Hudson, S., Mankoff, J.: Cord input: an intuitive, high-accuracy, multi-degree-of-freedom input method for mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2010, pp. 1657–1660. ACM, New York (2010).
  22. 22.
    Simon, A., Froehlich, B.: The YoYo: a handheld device combining elastic and isotonic input. In: Human-Computer Interaction INTERACT 2003: IFIP TC13 International Conference on Human-Computer Interaction, 1st–5th September 2003, Zurich, Switzerland (2003)Google Scholar
  23. 23.
    Szalavári, Z.: The personal interaction panel - a two-handed interface for augmented reality. Ph.D. thesis, Institute of Computer Graphics and Algorithms, Vienna University of Technology, Vienna, Austria (1999).
  24. 24.
    Tsai, H.R., Rekimoto, J.: ElasticVR: providing multi-level active and passive force feedback in virtual reality using elasticity. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA 2018, pp. D300:1–D300:4. ACM, New York (2018).
  25. 25.
    Weigel, M., Steimle, J.: Deformwear: deformation input on tiny wearable devices. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1(2), 28:1–28:23 (2017). Scholar
  26. 26.
    White, S., Feng, D., Feiner, S.: Interaction and presentation techniques for shake menus in tangible augmented reality. In: 2009 8th IEEE International Symposium on Mixed and Augmented Reality, pp. 39–48, October 2009.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Konstantin Klamka
    • 1
    Email author
  • Patrick Reipschläger
    • 1
  • Raimund Dachselt
    • 1
  1. 1.Interactive Media Lab DresdenTechnische Universität DresdenDresdenGermany

Personalised recommendations