Advertisement

See-Through Head Worn Display (HWD) Architectures

  • Jannick P. RollandEmail author
  • Kevin P. Thompson
  • Hakan Urey
  • Mason Thomas

Abstract

Over the past 3 decades, as computer and display technology advanced along the path laid out by Moore’s Law of miniaturization and functionality, many writers presented scenarios for augmented reality (AR) displays centered on bringing information to the individual. In that the emphasis was on the individual experience, the initial technology that did not pursue a see-through geometry seemed viable. When the initial solutions, a generation of look-at displays resting on the nose bridge appeared around 2000, the market did not embrace it. Suddenly now, social media has burst on the scene and wireless access has become ubiquitous. The result is a renewed research interest in a family of see-through head-worn displays (HWDs) enabling real-time interaction throughout the global community.

See-through HWD design inherently requires an interdisciplinary approach; optical engineering, opto-mechanics, ergonomics, and psychology all being keys to the design process. The last decade has seen a game changing technology emerge, the organic light emitting display (OLED), replacing what was thought itself to be game changing, the light emitting diode (LED) illuminator combined with a liquid crystal display (LCD) or liquid crystal on silicon (LCoS) display, which in turn had replaced the initial technology, the mini-CRT. As this chapter comes to press, the first HD-format OLED displays are becoming available for prototype development. The industry is currently working to supply a system that will receive widespread consumer acceptance (meaning millions of units need to be manufacturable in a period of months once a design point is selected). The system must be low cost (hundreds of dollars to the buyer), and approach an eyeglass format with resolution that approaches that of the human visual system extending into the peripheral FOV.

This chapter will first motivate the potential benefits of HWDs, especially in see-through mode, and examine key technology paths that build on historical highlights. Market barriers to the emergence of eyewear format HWDs will next be highlighted. We will then review optical architectures for see-through HWDs and key factors and functions required of a successful see-through HWD. Specifically, building on fundamentals of optical design, the key engineering concepts and constraints will be presented and solutions discussed. Particular emphasis will be placed on differentiating the concept of an eye pupil and an operational eyebox. Next, the Lagrange invariant (LI), which sets fundamental limits in the optical design of HWDs, will be examined. Following the presentation of see-through HWDs, two differentiated solutions will be presented; the head-mounted (worn) projection display (HMPD) and the retinal scanning display (RSD). The chapter will conclude with a brief discussion of current research that may affect the solution that the market selects, we might predict by 2020.

Keywords

Augmented Reality Pixel Count Input Coupler Exit Pupil Holographic Optical Element 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

List of Abbreviations

AHMD

Advanced Helmet Mounted Display

AlInGaP

Aluminum indium Gallium Phosphide

ARPA

Advanced Research Projects Agency

AR

Augmented Reality

CGH

Computer Generated Hologram

CM

Center of Mass

CRT

Cathode Ray Tube

D-MLA

Dual Microlenslet Array

DOE

Diffractive Optical Element

EBE

Eyebox Expansion

FLCoS

Ferroelectric Liquid Crystal on Silicon

FLIR

Forward-Looking Infrared

FOV

Field of View

GaN

Gallium Nitride

GPS

Global Positioning System

HD

High Definition

HIDSS

Helmet Integrated Display Sight System

HMD

Head or Helmet-Mounted Display

HMPD

Head-Mounted Projection Display

HOE

Holographic Optical Element

HWD

Head-Worn Display

HWV

Head-Worn Video

IHADSS

Integrated Helmet and Display Sighting System (IHADSS)

IPD

Interpupillary Distance

InGaN

Indium Gallium Nitride

LCD

Liquid Crystal Display

LCoS

Liquid Crystal on Silicon

LED

Light Emitting Diode

LI

Lagrange Invariant

MEMS

Micro-Electro-Mechanical System

MLA

Microlens Array

MR

Mixed Reality

NA

Numerical Aperture

ODALab

Optical Diagnostic and Applications Laboratory

OLED

Organic Light Emitting Display

ORA

Optical Research Associates

PDA

Personal Digital Assistant

RGB

Red–Green–Blue

RSD

Retinal Scanning Display

SGR

Substrate-Guided Relay

TIR

Total Internal Reflection

References

  1. 1.
    Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in augmented reality. IEEE Comput Graph Appl 21(6):34–47CrossRefGoogle Scholar
  2. 2.
    Rolland JP, Fuchs H (2000) Optical versus video see-through head-mounted displays in medical visualization. Presence Teleoperators Virtual Environ 9(3):287–309 (MIT Press)CrossRefGoogle Scholar
  3. 3.
    Bichlmeier C, Heining SM, Feuerstein M, Navab N (2009) The virtual mirror: a new interaction paradigm for augmented reality environments. IEEE Trans Med Imag 28(9):1498–1510CrossRefGoogle Scholar
  4. 4.
    Argotti Y, Davis L, Outters V, Rolland JP (2002) Dynamic superimposition of synthetic objects on rigid and simple-deformable objects. Comput Graph 26(6):919–930CrossRefGoogle Scholar
  5. 5.
    Rosenthal A, State A, Lee J, Hirota G, Ackerman J, Keller K, Pisano ED, Jiroutek M, Muller K, Fuchs H (2002) Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms. Med Image Anal 6(3):313–320CrossRefGoogle Scholar
  6. 6.
    Klein G, Murray D (2009) Parallel tracking and mapping on a camera phone. In: Proceedings of the 8th IEEE international symposium on mixed and augmented reality. IEEE, Washington, DC, pp 83–86Google Scholar
  7. 7.
    Nilsson S, Johansson B, Jonsson A (2009) Using AR to support cross-organizational collaboration in dynamic tasks. In: Proceedings of the IEEE international symposium on mixed and augmented reality. IEEE, Washington, DC, pp 3–12Google Scholar
  8. 8.
    Howlett EM (1992) High-resolution inserts in wide-angle head-mounted stereoscopic displays. In: Merritt JO, Fisher SS (eds) Proceedings of the SPIE, vol 1669, no. 1, pp 193–203Google Scholar
  9. 9.
    Rolland JP, Yoshida A, Davis LD, Reif JH (1998) High-resolution inset head-mounted display. Appl Opt 37(19):4183–4193CrossRefGoogle Scholar
  10. 10.
    Brown LG, Boger YS (2008) Application of the sensics panoramic HMD. SID symposium digest of technical papers 39:77–80CrossRefGoogle Scholar
  11. 11.
    Hoffman DM, Girshick AR, Akeley K, Banks MS (2008) Vergence accommodation conflicts hinder visual performance and cause visual fatigue. J Vis 8:1–30CrossRefGoogle Scholar
  12. 12.
    Biocca F, Rolland JP (1998) Virtual eyes can rearrange your body: adaptation to visual displacement in see-through head-mounted displays. Presence 7(3):262–278CrossRefGoogle Scholar
  13. 13.
    State A, Keller K, Fuchs H Simulation-based design and rapid prototyping of a parallax-free, orthoscopic video see-through head-mounted display. In: Proceedings international symposium on mixed and augmented reality (ISMAR) 2005, Vienna, pp 28–31Google Scholar
  14. 14.
    Sutherland IE (1968) A head-mounted three-dimensional display. AFIPS Proc Fall Joint Comput Conf 33:757–764Google Scholar
  15. 15.
    Rolland JP, Hua H (2005) Head-mounted displays. In: Johnson RB, Driggers RG (eds) Encyclopedia of optical engineering. Taylor and Francis, New York, pp 1–14Google Scholar
  16. 16.
    Cakmakci O, Rolland JP (2006) Head-worn displays: a review. J Display Technol 2:199–216CrossRefGoogle Scholar
  17. 17.
    Rolland JP, Kaya I, Thompson KP, Cakmakci O (2010) Head-worn displays – Lens design. In: Proceedings of SID, vol 57, 57.3, New York, NYGoogle Scholar
  18. 18.
    Chen CW (1996) Helmet visor display employing reflective, refractive, and diffractive optical elements. US Patent 5,526,183 1996Google Scholar
  19. 19.
    Sweatt WC (1977) Describing holographic optical elements as lenses. JOSA A 67:803–808CrossRefGoogle Scholar
  20. 20.
    Furness T (1995) Virtual retinal display. US Patent 5,467,104 Nov 14 1995Google Scholar
  21. 21.
    Sisodia A, Bayer AM, Smith PT, Nash B, Little J, Cassarly W, Gupta A (2007) Advanced helmet mounted display (AHMD). In: Brown RW, Reese CE, Marasco PL, Harding TH (eds) Head and Helmet-Mounted Display XII, Proceedings of the SPIE vol 6567. SPIE, pp 6567–65570NGoogle Scholar
  22. 22.
    Yaakov A (2003) Substrate-guided optical beam expander. US Patent 6,829,095Google Scholar
  23. 23.
    Cameron AA (2009) The application of holographic optical waveguide technology to the Q-Sight family of helmet-mounted displays. Proc SPIE 7326:7326HGoogle Scholar
  24. 24.
    Yamazaki S, Inoguchi K, Saito Y, Morishima H, Taniguchi N (1999) Thin wide-FOV HMD with free-form-surface prism and applications. In: Stereoscopic Displays and Virtual Reality Systems VI, 3639, Proc. of the SPIE, 3639, 453–462 (1999)Google Scholar
  25. 25.
    Mukawa H et al (2008) A full color eyewear display using holographic planar waveguides. Proc Soc Info Display 3901:89–92Google Scholar
  26. 26.
    Hua H, Ha Y, Rolland JP (2003) Design of an ultra-light and compact projection lens. Appl Opt 42(1):97–107CrossRefGoogle Scholar
  27. 27.
    Cakmakci O, Rolland JP (2007) Design and fabrication of a dual-element off-axis near-eye optical magnifier. Opt Lett 32(11):1363–1365CrossRefGoogle Scholar
  28. 28.
    Zhang R, Hua H (2008) Design of a polarized head-mounted projection display using ferroelectric liquid-crystal-on-silicon microdisplays. Appl Opt 47(15):2888–2896CrossRefGoogle Scholar
  29. 29.
    Cakmakci O, Moore B, Foroosh H, Rolland JP (2008) Optimal local shape description for rotationally non-symmetric optical surface design and analysis. Opt Express 16:1583–1589CrossRefGoogle Scholar
  30. 30.
    Cakmakci O, Vo S, Forroosh H, Rolland JP (2008) Application of radial basis functions to shape description in a dual-element off-axis magnifier. Opt Lett 33(11):1237–1239CrossRefGoogle Scholar
  31. 31.
    Cakmakci O, Vo S, Thompson KP, Rolland JP (2008) Application of radial basis functions to shape description in a dual-element off-axis eyewear display: FOV limit. J Soc Inf Display 16(11):1089–1098CrossRefGoogle Scholar
  32. 32.
    Urey H (2003) Retinal scanning displays. In: Driggers R (ed) Encyclopedia of optical engineering. Marcel Dekker, New YorkGoogle Scholar
  33. 33.
    Rolland JP, Davis LD, Baillot Y (2001) A survey of tracking technology for virtual environments. In: Barfield W, Caudell T (eds) Fundamentals of wearable computers and augmented reality. Lawrence Erlbaum, Mahwah, NJ, pp 67–112 (Chapter 3)Google Scholar
  34. 34.
    Cakmakci O, Thompson KP, Vallee P, Cote J, Rolland JP (2010) Design of a free-form single-element head-worn display. In: Proceedings of the SPIE, vol 7618, pp 7618–03Google Scholar
  35. 35.
    Kidger M (2001) Fundamentals of optical design. SPIE, Bellingham, pp 26–27CrossRefGoogle Scholar
  36. 36.
    Draper R, Wood M, Radmard B, Mahmud K, Schuler P, Sotzing GA, Seshadri V, Mino W, Padilla J, Otero TF (2005) Electrochromic variable transmission optical combiner. In: Hopper DG, Forsythe EW, Morton DC, Bradford CE, Girolamo HJ (eds) Cockpit and future displays for defense and security, proceedings of SPIE, vol 5801, Bellingham, WA, pp 268–277Google Scholar
  37. 37.
    Mortimer R, Dyer AL, Reynolds JR (2006) Electrochromic organic and polymeric materials for display applications. Displays 27:2–18CrossRefGoogle Scholar
  38. 38.
    Yoshida A, Rolland JP, Reif JH (1995) Design and applications of a high-resolution insert head-mounted display. In: Proceedings 1995 virtual reality annual international symposium (VRAIS 1995), Seattle, pp 84–93Google Scholar
  39. 39.
    Melzer JE (2001) Design evolution of a wide FOV head-mounted display for aircraft training and simulation. In: Lewandowski RJ, Haworth LA, Girolamo HJ, Rash CE (eds) Helmet- and head-mounted display VI, Proceedings of SPIE, vol 4361, OrlandoGoogle Scholar
  40. 40.
    Hoppe M, Melzer J (1999) Optical tiling for wide FOV head-mounted displays. In: Proceedings SPIE conference on current developments in optical design and optical engineering VIII. SPIE, DenverGoogle Scholar
  41. 41.
    Edwards E, Rolland JP, Keller K (1993) Video see-through design for merging of real and virtual environments. In: Proceedings of IEEE virtual reality annual international symposium (VRAIS 1993), Seattle, pp 223–233CrossRefGoogle Scholar
  42. 42.
    Urey H, Chellappan KV, Erden E, Surman P (2011) State of the art in stereoscopic and autostereoscopic displays. Proc IEEE 99(4):540–555CrossRefGoogle Scholar
  43. 43.
    Heilig M (1962) Sensorama simulator. US Patent 3,050,870, 28 Aug 1962Google Scholar
  44. 44.
    Ferrari V, Megali G, Troia E, Pietrabissa A, Mosca F (2009) A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng 56(11):2627–2633CrossRefGoogle Scholar
  45. 45.
    Fisher R (1996) Head-mounted projection display system featuring beam splitter and method of making same. US Patent 5,572,229, 5 Nov 1996Google Scholar
  46. 46.
    Kijima R, Ojika T (1997) Transition between virtual environment and workstation environment with projective head-mounted display. In: Virtual reality annual international symposium (VRAIS’97). IEEE Computer Society, Los Alamitos, CA, pp 130–137Google Scholar
  47. 47.
    Hua H, Ha Y, Rolland JP (2003) Design of an ultra-light and compact projection lens. Appl Opt 42(1):97–107CrossRefGoogle Scholar
  48. 48.
    Martins R, Shaoulov V, Ha Y, Rolland JP (2007) A mobile head-worn projection display. Opt Express 15:14530–14538CrossRefGoogle Scholar
  49. 49.
    Rolland JP, Biocca F, Hamza-Lup F, Ha Y, Martins R (2005) Development of head-mounted projection displays for distributed, collaborative augmented reality applications. Presence SI Immers Proj Technol 14(5):528–549Google Scholar
  50. 50.
    Rolland JP, Biocca F, Hua H, Ya H, Gao C, Harrisson O (2004) Teleportal augmented reality system: Integrating virtual objects, remote collaborators, and physical reality for distributed networked manufacturing. In: Ong SK, Nee AYC (eds) Virtual and augmented reality applications in manufacturing. Springer, London, p 183 (chapter 11)Google Scholar
  51. 51.
    Yalcinkaya A, Urey H, Brown D, Montague T, Sprague R (2006) Two-axis electromagnetic microscanner for high resolution displays. J Microelectromechanical Syst 15(4):786–794CrossRefGoogle Scholar
  52. 52.
    De Wit GC (1997) A retinal scanning display for virtual reality. Ph.D. Thesis. Delft University of TechnologyGoogle Scholar
  53. 53.
    Urey H et al (2005) Vibration mode frequency formulae for micromechanical scanners. J Micromech Microeng 15:1713–1721CrossRefGoogle Scholar
  54. 54.
    Brewster D (1958) The Kaleidoscope: its history, theory, and construction. John Murray, LondonGoogle Scholar
  55. 55.
    Urey H (2001) Diffractive exit-pupil expander for display applications. Appl Opt 40(32):5840–5851CrossRefGoogle Scholar
  56. 56.
    Urey H, Powell KD (2005) Microlens-array-based exit-pupil expander for full-color displays. Appl Opt 44(23):4930–4936CrossRefGoogle Scholar
  57. 57.
    Rolland JP (2000) Wide angle, off-axis, see-through head-mounted display. Opt Eng (Special Issue on Pushing the Envelope in Optical Design Software. 39(7):1760–1767Google Scholar

Further Reading

  1. Barfield W, Caudell T (2001) Fundamentals of wearable computers and augmented reality. Lawrence Erlbaum, MahwahGoogle Scholar
  2. Chellephan KV, Erden E, Urey H (2010) Laser based displays: a review. Appl Opt 49(25):F79–F98 (Feature issue on Lasers: the first fifty years)CrossRefGoogle Scholar
  3. Kalawsky RS (1993) The science of virtual reality and virtual environments. Addison-Wesley, BostonGoogle Scholar
  4. Peli E, Vargas-Martin F (2008) In-the-spectacle-lens telescopic device. J Biomed Opt 13(3):034027CrossRefGoogle Scholar
  5. Rash CE (2001) Helmet-mounted displays: design issues for rotary-wing aircraft. SPIE Press, WashingtonGoogle Scholar
  6. Reiss M (1945) The Cos4 law of illumination. JOSA 35(4):283–288MathSciNetCrossRefGoogle Scholar
  7. Santhanam AP, Willoughby TR, Kaya I, Shah AP, Meeks SL, Rolland JP, Kupelian P (2008) A display framework for visualizing real-time 3D lung tumor radiotherapy. J Display Technol 4(4):473–482 (Special issue on medical displays)CrossRefGoogle Scholar
  8. Task HL (1997) HMD image source, optics, and the visual interface. In: Melzer JE, Moffitt K (eds) Head mounted displays: designing for the user. McGraw-Hill, New York, pp 55–82 Chapter 3Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jannick P. Rolland
    • 1
    Email author
  • Kevin P. Thompson
    • 2
  • Hakan Urey
    • 3
  • Mason Thomas
    • 4
  1. 1.University of RochesterRochesterUSA
  2. 2.Synopsys, Inc., RochesterRochesterUSA
  3. 3.Koç UniversityIstanbulTurkey
  4. 4.Microvision Inc.RedmondUSA

Personalised recommendations