Skip to main content

The activation of modality in virtual objects assembly


Manual assembly of virtual 3D objects is required in several application fields. We focus on tangible user interfaces which offer the opportunity to the user to perform virtual assemblies efficiently and easily. In each hand, the user manipulates a tracked prop, and the translations and rotations are directly mapped to the corresponding virtual object. However, with such interfaces, both hands are requisitioned, and the user cannot drop the props without changing the action or the expected results. We list and discuss the choice of four possible modalities to activate/deactivate the assembly modality: vocal modality, gestural modality, buttons, and foot pedals. We conclude that when using the foot pedals, the user’s gesture is closer to the real-world behaviour.

This is a preview of subscription content, access via your institution.


  1. Bolt R (1980) “Put-that-there”: Voice and gesture at the graphics interface. In: SIGGRAPH ’80: proceedings of the 7th annual conference on computer graphics and interactive techniques. ACM, New York, pp 262–270

    Chapter  Google Scholar 

  2. Dix A, Ghazali M, Ramduny-Ellis D (2008) Modelling devices for natural interaction. Electron Not Theor Comput Sci 208:23–40

    Article  Google Scholar 

  3. English WK, Engelbart DC, Berman ML (1967) Display-selection techniques for text manipulation. IEEE Trans Hum Factors Electron HFE-8(1):5–15

    Article  Google Scholar 

  4. Garreau L, Legardeur J, Couture N (2005) Une plate-forme basée sur les interfaces tangibles pour l’assemblage en CFAO. Rev Int Ing Numér 1(2):133–148

    Google Scholar 

  5. Hinckley K, Pausch R, Goble JC, Kassell NF (1994) Passive real-world interface props for neurosurgical visualization. In: CHI ’94: proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 452–458

    Google Scholar 

  6. Ishii H, Ren S, Frei P (2001) Pinwheels: visualizing information flow in an architectural space. In: CHI ’01: extended abstracts on Human factors in computing systems. ACM, New York, pp 111–112

    Chapter  Google Scholar 

  7. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77-D(12):1321–1329

    Google Scholar 

  8. Nigay L, Coutaz J (1997) The CARE properties and their impact on software design. AAAI Press, Menlo Park

    Google Scholar 

  9. Qi W, Martens JB, van Liere R, Kok A (2005) Reach the virtual environment: 3D tangible interaction with scientific data. In: OZCHI ’05: proceedings of the 19th conference of the computer-human interaction special interest group (CHISIG) of Australia on Computer-human interaction. Computer-Human Interaction Special Interest Group (CHISIG) of Australia, pp 1–10

  10. Reuter P, Rivière G, Couture N, Sorraing N, Espinasse L, Vergnieux R (2007) ArcheoTUI—A tangible user interface for the virtual reassembly of fractured archeological objects. In: VAST2007: proceedings of the 8th EuroGraphics international symposium on virtual reality, archaeology and cultural heritage. EuroGraphics Association, pp 15–22

  11. Ryokai K, Marti S, Ishii H (2004) I/O brush: drawing with everyday objects as ink. In: CHI ’04: proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 303–310

    Google Scholar 

  12. Sellen A, Kurtenbach G, Buxton W (1990) The role of visual and kinesthetic feedback in the prevention of mode errors. In: INTERACT ’90: proceedings of the IFIP TC13 third international conference on human-computer interaction. North-Holland, Amsterdam, pp 667–673

    Google Scholar 

  13. Ullmer B, Ishii H (2000) Emerging frameworks for tangible user interfaces. IBM Syst J 39(3/4):915–931

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Guillaume Rivière.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Rivière, G., Couture, N. & Reuter, P. The activation of modality in virtual objects assembly. J Multimodal User Interfaces 3, 189–196 (2010).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: