Skip to main content
SpringerLink
Log in

Inside-reachable and see-through augmented reality Shell for 3D visualization and tangible interaction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Augmented reality (AR) is considered as one of the most essential technologies for providing 3D information visualization superimposed in the physical object. Thus, AR has been applied to various applications such as entertainment, education, and human-computer interaction. In particular, AR cubes have also been widely used since they can provide multi-perspective and immersive 3D visualization. However, an inherent problem in current AR cubes is that they cannot support practical and tangible interactions with both virtual and physical artifacts. Thus, they can only provide simple manipulations with a limited interaction metaphor. In particular, they cannot support internal accessibility and direct manipulation with virtual and physical objects inside the cube. This paper presents a new and innovative form of an AR cube, called inside-reachable and see-through AR shell. The AR shell can support more intuitive and tangible interaction with AR objects than conventional AR cubes. One of the unique characteristics of the AR shell is that the physical tool is accessible inside the shell. To make the AR shell be inside-reachable, each face of the AR shell may have a physical hole. This fabrication of the AR shell supports not only multi-perspective views but also provides tangible and natural interaction through the physical holes. In addition, faces with holes may prevent the AR camera from tracking AR markers consistently and adequately due to the reduced tracking areas. Thus, the proposed AR shell can be made of some faces without holes and the other faces with holes. An occlusion rendering is used for showing depth cues to visualize occlusions, which makes the faces without holes see-through. Therefore, the user can interact with AR objects inside the shell through physical holes while the user views the inside of the shell through physically occluded faces. We have also extended the proposed approach to run in a mobile HMD. We have performed formal quantitative and quantitative analyses by evaluating task performance and questionnaire. Several implementations will be presented to prove the originality and advantage of the proposed AR shell.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Avery B, Sandor C, Thomas BH (2009) Improving spatial perception for augmented reality x-ray vision. Proc. IEEE VR’09

  2. Bergig O, Sereq E, Hagbi N, Pevzner K, Levi N, Blau S, Smelansky Y, El-Sana J (2001) Out of the cube: augmented rubik’s cube. International Journal of Computer Games Technology. Article ID 570210

  3. Brooke J (1996) SUS-A quick and dirty usability scale. Usability evaluation in industry 189(194):4–7

    Google Scholar 

  4. Chakraborty A, Gross R, McIntee S, Hong K-H, Lee JY, St. Amant R (2014) Captive: a cube with augmented physical tools. Proc. CHI’14 EA. 1315–1320

  5. Coma-Tatay I, Casas-Yrurzum S, Casanova-Salas P, Fernández-Marín M (2018) FI-AR learning: a web-based platform for augmented reality educational content. Multimed Tools Appl 78:6093–6118. https://doi.org/10.1007/s11042-018-6395-5

    Article  Google Scholar 

  6. Elmqvist N, Tsigas P (2008) A taxonomy of 3D occlusion management for visualization. IEEE Trans Vis Comput Graph 14(5):1095–1109

    Article  Google Scholar 

  7. Fenu C, Pittarello F (2017) Svevo tour: the design and the experimentation of an augmented reality application for engaging visitors of a literary museum. Int J Human-Comput Stud 114:20–35

    Article  Google Scholar 

  8. Grubert J, Kranz M (2017) mpCubee: towards a mobile perspective cubic display using mobile phones. Proc. IEEE VR’17

  9. Harish P, Narayanan PJ (2013) Designing perspectively correct multiplanar displays. IEEE Trans Vis Comput Graph 19(3):407–419

    Article  Google Scholar 

  10. Heun V, Kasahara S, Maes P (2013) Smarter objects: using AR technology to program physical objects and their interactions. Proc. CHI'13 pp. 961-966

  11. Hinckley K, Pausch R, Goble JC, Kassell NF (1994) Passive real-world interface props for neurosurgical visualization. Proc. CHI’94. 452–458

  12. Hornecker E, Dünser A (2009) Of pages and paddles: children’s expectations and mistaken interactions with physical-digital tools. Interact Comput 21(1–2):95–107

    Article  Google Scholar 

  13. Ichida H, Itoh Y, Kitamura Y, Kishino F (2004) ActiveCube and its 3D applications. Proc. IEEE VR’04

  14. Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. Proc. CHI’97. 234–241

  15. Issartel P, Besançon L, Isenberg T, Ammi M (2016) A tangible volume for portable 3D interaction. Proc. ISMAR-Adjunct’16

  16. Juan C, Canu R, Gimenez M (2008) Augmented reality interactive storytelling systems using tangible cubes for edutainment. Proc. 8th IEEE international Conf. on advanced learning technologies. 233-235

  17. Juan CM, Toffetti G, Abad F, Cano J (2010) Tangible cubes used as the user interface in an augmented reality game for edutainment. Proc. IEEE 10th international Conf. On advanced learning technologies. 599-603

  18. Kim M, Lee JY (2016) Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in a mobile augmented reality. Multimed Tools Appl 75(23):16529–16550

    Article  Google Scholar 

  19. Lam B, Tang Y, Stavness I, Fels S (2011) A 3D cubic puzzle in pCubee. Proc. IEEE symposium on 3D user interfaces. 135-136

  20. Lee K, Jeong D (2014) Memorix: a tangible memory game using iSIG-blocks. Proc, IEEE Games Media Entertainment

    Google Scholar 

  21. Lee JY, Rhee GW, Park H (2009) AR/RP-based tangible interactions for collaborative design evaluation of digital products. Int J Adv Manuf Technol 45(7–8):649–665

    Article  Google Scholar 

  22. Lee JY, Rhee GW, Seo DW (2010) Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. Int J Adv Manuf Technol 51(9–12):1069–1082

    Article  Google Scholar 

  23. Lee H, Billinghurst M, Woo W (2011) Two-handed tangible interaction techniques composing augmented blocks. Virtual Reality 15(2–3):133–146

    Article  Google Scholar 

  24. Maekawa T, Itoh Y, Kawai N, Kitamura Y, Kishino F (2009) MADO interface: a window like a tangible user interface to look into the virtual world. Proc. TEI’09. 175-180

  25. Mulder JD, van Liere R (2002) The person space station: bringing interaction within reach. Proc. IEEE 4th virtual reality international Conf. 73-81

  26. Park H, Park SJ, Jung HK (2013) Note on tangible interaction using paper models for AR-based design evaluation. J Adv Mech Des, Syst, Manuf 7(5):827–835

    Article  Google Scholar 

  27. Park H, Jung HK, Park SJ (2014) Tangible AR interaction based on fingertip touch using small-sized nonsquare markers. Journal of Computational Design and Engineering 1(4):289–297

    Article  Google Scholar 

  28. Pla P, Maes P (2012) Display blocks: cubic displays for multi-perspective visualization. Proc. CHI’12 EA. 2015-2020

  29. Schweikardt E., Gross MD (2006) roBlocks: a robotic construction kit for mathematics and science education. Proc. ICMI’06, 72-75

  30. Seo DW, Lee JY (2013) Direct hand touchable interactions in augmented reality environments and intuitive user experiences. Expert Syst Appl 40(9):3784–3793

    Article  Google Scholar 

  31. Sharlin E, Itoh Y, Watson B, Kitamura Y, Sutphen S, Liu L (2002) Cognitive cubes: a tangible user interface for cognitive assessment. Proc. CHI’02. 347-354

  32. Sin AK, Zaman HB (2009) Tangible interaction in learning astronomy through augmented reality. LNCS. 5857:302–313

    Google Scholar 

  33. Stavness I, Lam B, Fels S (2010) pCubee: a perspective-corrected handheld cubic display. Proc. CHI ‘10. 1381–1390

  34. Terrenghi L, Kranz M, Holleis P, Schmidt A (2006) A cube to learn: a tangible user interface for the design of a learning appliance. J Pers Ubiquit Comput 10(2–3):53–159

    Google Scholar 

  35. Vuforia (2016). https://www.vuforia.com/

  36. Wang D, Zhang Y, Chen S (2013) E-blocks: a tangible programming tool with graphical blocks. Mathematical Problems in Engineering. Article ID 598547

  37. Watanabe R, Itoh Y, Asai M, Kitamura Y, Kishno F (2004) The soul of ActiveCube: implementing a flexible, multimodal, three-dimensional spatial tangible interface. Proc. ACE’04. 173-180

  38. Yonemoto S, Yotsumoto T, Taniguchi R-I (2007) Virtual object manipulation using physical blocks. Proc. 11th international Conf. on information visualization. 781-785

  39. You C-W, Hsieh Y-H, Cheng W-H, Hsieh Y-H (2014) AttachedShock: design of a crossing-based target selection technique on augmented reality devices and its implications. Int J Human-Comput Stud 72(7):606–626

    Article  Google Scholar 

  40. Zhou ZY, Cheok AD, Chan TT, Pan JH, Li Y (2004) Interactive entertainment systems using tangible cubes. Proc Australian Workshop on Interactive Entertainment:19–22

  41. Zhou ZY, Cheok AD, Le Y, Kato H (2005) Magic cubes for social and physical family entertainment. Proc. CHI’05 EA. 1156-1157

Download references

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2016R1D1A1B03934697 & 2019R1I1A3A01059082). The authors would like to thank Hyeon-Seok Kim for an experimental setup.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Chonnam National University, 77, Yongbong-ro, Buk-gu, Gwangju, 61186, South Korea

    Minseok Kim, Kyeong-Beom Park, Sung Ho Choi & Jae Yeol Lee

Authors
  1. Minseok Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyeong-Beom Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sung Ho Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jae Yeol Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jae Yeol Lee.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, M., Park, KB., Choi, S.H. et al. Inside-reachable and see-through augmented reality Shell for 3D visualization and tangible interaction. Multimed Tools Appl 79, 5941–5963 (2020). https://doi.org/10.1007/s11042-019-08324-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08324-3

Keywords

Search

Navigation