Skip to main content
SpringerLink
Log in

Pursuit of “X-Ray Vision” for Augmented Reality

  • Chapter
  • First Online:
Human Factors in Augmented Reality Environments

Abstract

The ability to visualize occluded objects or people offers tremendous potential to users of augmented reality (AR). This is especially true in mobile applications for urban environments, in which navigation and other operations are hindered by the urban infrastructure. This “X-ray vision” feature has intrigued and challenged AR system designers for many years, with only modest progress in demonstrating a useful and usable capability. The most obvious challenge is to the human visual system, which is being asked to interpret a very unnatural perceptual metaphor. We review the perceptual background to understand how the visual system infers depth and how these assumptions are or are not met by augmented reality displays. In response to these challenges, several visualization metaphors have been proposed; we survey these in light of the perceptual background. Because augmented reality systems are user-centered, it is important to evaluate how well these visualization metaphors enable users to perform tasks that benefit from X-ray vision. We summarize studies reported in the literature. Drawing upon these analyses, we offer suggestions for future research on this tantalizing capability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    1  Personal space, action space, and vista space are commonly termed as near-field, medium-field, and far-field distances, respectively. In this chapter, we will use the latter set of terms.

References

  1. Avery, B., Sandor, C., Thomas, B.H.: Improving spatial perception for augmented reality X-ray vision. In: IEEE Virtual Reality, pp. 79–82 (2009)

    Google Scholar 

  2. Bajura, M., Fuchs, H., Ohbuchi, R.: Merging virtual objects with the real world: Seeing ultrasound imagery within the patient. Computer Graphics (Proceedings of SIGGRAPH’92) 26(2), 203–210 (1992)

    Google Scholar 

  3. Bane, R., Höllerer, T.: Interactive tools for virtual X-ray vision in mobile augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 231–239 (2004)

    Google Scholar 

  4. Biocca, F.A., Rolland, J.P.: Virtual eyes can rearrange your body: Adaptation to visual displacement in see-through, head-mounted displays. Presence: Teleoperators and Virtual Environments 7(3), 262–277 (1998)

    Article  Google Scholar 

  5. Bruce, V., Green, P.R., Georgeson, M.A.: Visual Perception: Physiology, Psychology, and Ecology, 3rd edn. Psychology Press (1996)

    Google Scholar 

  6. Cutting, J.E.: Reconceiving perceptual space. In: H. Hecht, R. Schwartz, M. Atherton (eds.) Looking into Pictures: An Interdisciplinary Approach to Pictorial Space, chap. 11, pp. 215–238. MIT Press (2003)

    Google Scholar 

  7. Cutting, J.E., Vishton, P.M.: Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth. In: Handbook of perception and cognition, Vol. 5: Perception of space of motion, pp. 69–117. Academic Press (1995)

    Google Scholar 

  8. Dey, A., Cunningham, A., Sandor, C.: Evaluating depth perception of photorealistic mixed reality visualizations for occluded objects in outdoor environments. In: 17th ACM Symposium on Virtual Reality Software and Technology, pp. 211–218 (2010)

    Google Scholar 

  9. Dey, A., Jarvis, G., Sandor, C., Wibowo, A.K., Mattila, V.V.: An evaluation of augmented reality X-ray vision for outdoor navigation. In: The 21st International Conference on Artificial Reality and Telexistence, pp. 28–32 (2011)

    Google Scholar 

  10. Ellis, S.R., Bucher, U.J., Menges, B.M.: The relationship of binocular convergence and errors in judged distance to virtual objects. In: Proceedings of the International Federaion of Automatic Control, pp. 297–301 (1995)

    Google Scholar 

  11. Ellis, S.R., Menges, B.M.: Localization of virtual objects in the near visual field. Human Factors 40(3), 415–431 (1998)

    Article  Google Scholar 

  12. Feiner, S.K., Seligmann, D.D.: Cutaways and ghosting: Satisfying visibility constraints in dynamic 3D illustrations. The Visual Computer 8(5–6), 292–302 (1993)

    Google Scholar 

  13. Fidopiastis, C.M.: User-centered virtual environment assessment and design for cognitive rehabilitation applications. Ph.D. thesis, Department of Modeling and Simulation, University of Central Florida (2006)

    Google Scholar 

  14. Fuchs, H., Livingston, M.A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Meyer, A.A.: Augmented reality visualization for laparoscopic surgery. In: Proceedings of First International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 934–943 (1998)

    Google Scholar 

  15. Furness, L.T.A.: The application of head-mounted displays to airborne reconnaissance and weapon delivery. Tech. Rep. TR-69-241, U.S. Air Force Avionics Laboratory, Wright-Patterson AFB (1969)

    Google Scholar 

  16. Furness, T.A.: The super cockpit and its human factors challenges. In: Proceedings of the Human Factors Society 30th Annual Meeting, pp. 48–52. Human Factors and Ergonomics Society (1986)

    Google Scholar 

  17. Jerome, C.J., Witmer, B.G.: The perception and estimation of egocentric distance in real and augmented reality environments. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 49, pp. 2249–2252 (2005)

    Article  Google Scholar 

  18. Jones, J.A., Swan II, J.E., Singh, G., Ellis, S.R.: Peripheral visual information and its effect on distance judgments in virtual and augmented environments. In: ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization, pp. 29–35 (2011)

    Google Scholar 

  19. Jones, J.A., Swan II, J.E., Singh, G., Kolstad, E., Ellis, S.R.: The effects of virtual reality, augmented reality, and motion parallax on egocentric depth perception. In: ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization, pp. 9–14 (2008)

    Google Scholar 

  20. Julier, S., Lanzagorta, M., Baillot, Y., Rosenblum, L., Feiner, S., Hollerer, T., Sestito, S.: Information filtering for mobile augmented reality. In: Proceedings of IEEE and ACM International Symposium on Augmented Reality, pp. 3–11 (2000)

    Google Scholar 

  21. Kalkofen, D., Mendez, E., Schmalstieg, D.: Interactive focus and context visualization for augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 191–201 (2007)

    Google Scholar 

  22. Kirkley, Jr., S.E.H.: Augmented reality performance assessment battery (ARPAB: Object recognition, distance estimation, and size estimation using optical see-through head-worn displays. Ph.D. thesis, Instructional Systems Technology, Indiana University (2003)

    Google Scholar 

  23. Kiyokawa, K., Kurata, Y., Ohno, H.: An optical see-through display for mutual occlusion of real and virtual environments. Computers and Graphics 25(5), 765–779 (2001)

    Article  Google Scholar 

  24. Klein, G., Murray, D.: Simulating low-cost cameras for augmented reality compositing. IEEE Transactions on Visualization and Computer Graphics 16(3), 369–380 (2010)

    Article  Google Scholar 

  25. Liu, S., Hua, H., Cheng, D.: A novel prototype for an optical see-through head-mounted display with addressable focus cues. IEEE Transactions on Visualization and Computer Graphics 16(3), 381–393 (2010)

    Article  Google Scholar 

  26. Livingston, M.A., Ai, Z., Swan II, J.E., Smallman, H.S.: Indoor vs. outdoor depth perception for mobile augmented reality. In: IEEE Virtual Reality, pp. 55–62 (2009)

    Google Scholar 

  27. Livingston, M.A., Brown, D., Julier, S.J., Schmidt, G.S.: Military applications of augmented reality. In: NATO Human Factors and Medicine Panel Workshop on Virtual Media for Military Applications (2006)

    Google Scholar 

  28. Livingston, M.A., Karsch, K., Ai, Z., Gibson, LT G.O.: User interface design for military ar applications. Virtual Reality 15(2–3), 175–184 (2011)

    Article  Google Scholar 

  29. Livingston, M.A., Swan II, J.E., Gabbard, J.L., Höllerer, T.H., Hix, D., Julier, S.J., Baillot, Y., Brown, D.: Resolving multiple occluded layers in augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 56–65 (2003)

    Google Scholar 

  30. Livingston, M.A., Zanbaka, C., Swan II, J.E., Smallman, H.S.: Objective measures for the effectiveness of augmented reality. In: IEEE Virtual Reality, pp. 287–288 (2005)

    Google Scholar 

  31. Loomis, J.M., Knapp, J.M.: Visual perception of egocentric distance in real and virtual environments. In: L.J. Hettinger, M.W. Haas (eds.) Virtual and Adaptive Environments, pp. 21–46. Lawrence Erlbaum Associates (2003)

    Google Scholar 

  32. McCandless, J.W., Ellis, S.R.: The effect of eye position on the projected stimulus distance in a binocular head-mounted display. In: Stereoscopic Displays and Virtual Reality Systems VII, Proceedings of SPIE Vol. 3957, pp. 41–48 (2000)

    Google Scholar 

  33. McCandless, J.W., Ellis, S.R., Adelstein, B.D.: Localization of a time-delayed, monocular virtual object superimposed on a real environment. Presence: Teleoperators and Virtual Environments 9(1), 15–24 (2000)

    Article  Google Scholar 

  34. Nagata, S.: How to reinforce perception of depth in single two-dimensional pictures. In: Pictorial Communication in Virtual and Real Environments, chap. 35, pp. 527–545. Taylor & Francis, Inc. (1991)

    Google Scholar 

  35. Plumert, J.M., Kearney, J.K., Cremer, J.F., Recker, K.: Distance perception in real and virtual environments. ACM Transactions on Applied Perception 2(3), 216–233 (2005)

    Article  Google Scholar 

  36. Rolland, J.P., Ariely, D., Gibson, W.: Towards quantifying depth and size perception in virtual environments. Presence: Teleoperators and Virtual Environments 4(1), 24–49 (1995)

    Google Scholar 

  37. Rolland, J.P., Fuchs, H.: Optical versus video see-through head-mounted displays. Presence: Teleoperators and Virtual Environments 9(3), 287–309 (2000)

    Article  Google Scholar 

  38. Rolland, J.P., Meyer, C., Arthur, K., Rinalducci, E.J.: Method of adjustments versus method of constant stimuli in the quantification of accuracy and precision of rendered depth in head-mounted displays. Presence: Teleoperators and Virtual Environments 11(6), 610–625 (2002)

    Article  Google Scholar 

  39. Sandor, C., Cunningham, A., Dey, A., Mattila, V.V.: An augmented reality X-ray system based on visual saliency. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 27–36 (2010)

    Google Scholar 

  40. Sandor, C., Cunningham, A., Eck, U., Urquhart, D., Jarvis, G., Dey, A., Barbier, S., Marner, M., Rhee, S.: Egocentric space-distorting visualizations for rapid environment exploration in mobile mixed reality. In: IEEE Virtual Reality (2010)

    Google Scholar 

  41. Schall, G., Mendez, E., Kruijff, E., Veas, E., Junghanns, S., Reitinger, B., Schmalstieg, D.: Handheld augmented reality for underground infrastructure visualization. Personal and Ubiquitous Computing 13(4), 281–291 (2009)

    Article  Google Scholar 

  42. Schiffman, H.R.: Sensation and Perception, 5th edn. John Wiley & Sons (2001)

    Google Scholar 

  43. Singh, G., Swan II, J.E., Jones, J.A., Ellis, S.R.: Depth judgment measures and occluding surfaces in near-field augmented reality. In: ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization, pp. 149–156 (2010)

    Google Scholar 

  44. State, A., Hirota, G., Garrett, W.F., Chen, D.T., Livingston, M.A.: Superior augmented reality registration by integrating landmark tracking and magnetic tracking. In: Computer Graphics Proceedings, Annual Conference Series (Proceedings of SIGGRAPH’96), pp. 429–438 (1996)

    Google Scholar 

  45. Stix, G.: See-through view: Virtual reality may guide physician’s hands. Scientific American 267(3), 166 (1992)

    Article  Google Scholar 

  46. Sugano, N., Kato, H., Tachibana, K.: The effects of shadow representation of virtual objects in augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 76–83 (2003)

    Google Scholar 

  47. Swan II, J.E., Jones, A., Kolstad, E., Livingston, M.A., Smallman, H.S.: Egocentric depth judgments in optical, see-through augmented reality. IEEE Transactions on Visualization and Computer Graphics 13(3), 429–442 (2007)

    Article  Google Scholar 

  48. Swan II, J.E., Livingston, M.A., Smallman, H.S., Brown, D., Baillot, Y., Gabbard, J.L., Hix, D.: A perceptual matching technique for depth judgments in optical, see-through augmented reality. In: IEEE Virtual Reality, pp. 19–26 (2006)

    Google Scholar 

  49. Tsuda, T., Yamamoto, H., Kameda, Y., Ohta, Y.: Visualization methods for outdoor see-through vision. IEICE Transactions on Information and Systems E89–D(6), 1781–1789 (2006)

    Google Scholar 

  50. Witt, J.K., Proffitt, D.R., Epstein, W.: Perceiving distance: A role of effort and intent. Perception 33(5), 577–590 (2004)

    Article  Google Scholar 

  51. Zollmann, S., Kalkofen, D., Mendez, E., Reitmayr, G.: Image-based ghostings for single layer occlusions in augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 19–26 (2010)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Steve Ellis, Cali Fidopiastis, Henry Fuchs, LT Gregory O. Gibson, Tobias Höllerer, Christian Jerome, Sonny Kirkley, Jannick Rolland, Andrei State, and Ed Swan for their assistance in the preparation of this chapter.

Author information

Authors and Affiliations

  1. U.S. Naval Research Laboratory, Washington, DC, 20375, USA

    Mark A. Livingston

  2. University of South Australia, Adelaide, SA, 5001, Australia

    Arindam Dey, Christian Sandor & Bruce H. Thomas

Authors
  1. Mark A. Livingston
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arindam Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Sandor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruce H. Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark A. Livingston .

Editor information

Editors and Affiliations

  1. CSIRO ICT Centre, Pembroke Road, Marsfield, 1710, New South Wales, Australia

    Weidong Huang

  2. CSIRO ICT Centre, Pembroke Road, Marsfield, 1710, New South Wales, Australia

    Leila Alem

  3. Mail Code 5580, Washington, 20375, District of Columbia, USA

    Mark A. Livingston

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Livingston, M.A., Dey, A., Sandor, C., Thomas, B.H. (2013). Pursuit of “X-Ray Vision” for Augmented Reality. In: Huang, W., Alem, L., Livingston, M. (eds) Human Factors in Augmented Reality Environments. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4205-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-4205-9_4

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-4204-2

  • Online ISBN: 978-1-4614-4205-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation