Abstract
When immersed in Virtual Reality (VR), real objects around us could be dangerous if the users collide with them. On the other hand, such objects could be exploited to improve VR setups and to create Extended Realities where real and virtual elements are coherently mixed up, and interaction is enhanced by passive haptics. This work proposes a system that combines state-of-the-art 3D object detection, depth data from stereo cameras, and VR environment rendering. Our system can detect specific classes of objects, estimate their position and extent, and create a virtual counterpart for each one in an immersive virtual scene. We describe the pipeline of the system and show that our system allows real-time interaction with virtual objects blended with real ones.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Beever, L., John, N.W.: Leveled SR: a substitutional reality level design workflow. In: IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2022, Christchurch, New Zealand, 12–16 March 2022, pp. 130–138. IEEE (2022)
Ghasemi, Y., Jeong, H., Choi, S.H., Park, K.B., Lee, J.Y.: Deep learning-based object detection in augmented reality: a systematic review. Comput. Ind. 139, 103661 (2022)
Hartmann, J., Holz, C., Ofek, E., Wilson, A.D.: Realitycheck: blending virtual environments with situated physical reality. In: Brewster, S.A., Fitzpatrick, G., Cox, A.L., Kostakos, V. (eds.) Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, 04–09 May 2019, p. 347. ACM (2019)
Redmon, J., Divvala, S.K., Girshick, R.B., Farhadi, A.: You only look once: unified, real-time object detection. CoRR (2015)
Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 6517–6525. IEEE Computer Society (2017)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. CoRR (2018)
Valentini, I., Ballestin, G., Bassano, C., Solari, F., Chessa, M.: Improving obstacle awareness to enhance interaction in virtual reality. In: IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2010, Atlanta, GA, USA, 22–26 March 2020, pp. 44–52. IEEE (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Martini, M., Solari, F., Chessa, M. (2023). Obstacle Avoidance and Interaction in Extended Reality: An Approach Based on 3D Object Detection. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing – ICIAP 2023. ICIAP 2023. Lecture Notes in Computer Science, vol 14234. Springer, Cham. https://doi.org/10.1007/978-3-031-43153-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-43153-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43152-4
Online ISBN: 978-3-031-43153-1
eBook Packages: Computer ScienceComputer Science (R0)