Skip to main content
SpringerLink
Log in

Mixed Reality: A Survey

  • Chapter
Human Machine Interaction

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5440))

Abstract

This chapter presents an overview of the Mixed Reality (MR) paradigm, which proposes to overlay our real-world environment with digital, computer-generated objects. It presents example applications and outlines limitations and solutions for their technical implementation. In MR systems, users perceive both the physical environment around them and digital elements presented through, for example, the use of semitransparent displays. By its very nature, MR is a highly interdisciplinary field engaging signal processing, computer vision, computer graphics, user interfaces, human factors, wearable computing, mobile computing, information visualization, and the design of displays and sensors. This chapter presents potential MR applications, technical challenges in realizing MR systems, as well as issues related to usability and collaboration in MR. It separately presents a section offering a selection of MR projects which have either been partly or fully undertaken at Swiss universities and rounds off with a section on current challenges and trends.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Weiser, M.: Some computer science issues in ubiquitous computing. Commun. ACM 36(7), 75–84 (1993)

    Article  Google Scholar 

  2. Sutherland, I.: A head-mounted three dimensional display. In: Proceedings of the Fall Joint Computer Conference, vol. 33, pp. 757–764 (1968)

    Google Scholar 

  3. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21(6), 34–47 (2001)

    Article  Google Scholar 

  4. Milgram, P., Kishino, F.: A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems 77, 1321–1329 (1994)

    Google Scholar 

  5. Feiner, S., Macintyre, B., Seligmann, D.: Knowledge-based augmented reality. In: Communications of the ACM, vol. 36, pp. 53–62. ACM Press, New York (1993)

    Google Scholar 

  6. Lee, J., Rhee, G.: Context-aware 3D visualization and collaboration services for ubiquitous cars using augmented reality. The International Journal of Advanced Manufacturing Technology 37, 431–442 (2008)

    Article  Google Scholar 

  7. Reiners, D., Stricker, D., Klinker, G., Muller, S.: Augmented Reality for Construction Tasks: Doorlock Assembly. In: 1st International Workshop on Augmented Reality (IWAR 1998), San Francisco (1998)

    Google Scholar 

  8. Doil, F., Schreiber, W., Alt, T., Patron, C.: Augmented reality for manufacturing planning. In: Proceedings of the workshop on Virtual environments 2003, pp. 71–76. ACM Press, New York (2003)

    Chapter  Google Scholar 

  9. Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: a transitional AR interface. In: Computers & Graphics, vol. 25, pp. 745–753. Elsevier, Amsterdam (2001)

    Google Scholar 

  10. Klinker, G., Dutoit, A., Bauer, M., Bayer, J., Novak, V., Matzke, D.: “Fata Morgana” A Presentation System for Product Design. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR 2002). IEEE Computer Society, Washington (2002)

    Google Scholar 

  11. State, A., Livingston, M.A., Garrett, W.F., Hirota, G., Whitton, M.C., Pisano, E.D., Fuchs, H.: Technologies for augmented reality systems: Realizing ultrasound-guided needle biopsies. In: SIGGRAPH, pp. 439–446 (1996)

    Google Scholar 

  12. State, A., Hirota, G., Chen, D.T., Garrett, W.F., Livingston, M.A.: Superior augmented reality registration by integrating landmark tracking and magnetic tracking. In: SIGGRAPH, pp. 429–438 (1996)

    Google Scholar 

  13. Lorensen, W.E., Cline, H.E., Nafis, C., Altobelli, D., Gleason, L.: Enhancing reality in the operating room. In: IEEE Visualization, pp. 410–415 (1993)

    Google Scholar 

  14. Rosenthal, M., State, A., Lee, J., Hirota, G., Ackerman, J., Keller, K., Pisano, E., Jiroutek, M., Muller, K., Fuchs, H.: Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms. In: Medical Image Analysis, vol. 6, pp. 313–320. Elsevier, Amsterdam (2002)

    Google Scholar 

  15. Höllerer, T., Feiner, S., Terauchi, T., Rashid, G., Hallaway, D.: Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system. In: Computers & Graphics, vol. 23, pp. 779–785. Elsevier, Amsterdam (1999)

    Google Scholar 

  16. Welch, G., Sonnenwald, D., Mayer-Patel, K., Yang, R., State, A., Towles, H., Cairns, B., Fuchs, H.: Remote 3D medical consultation. In: Proc. BROADMED Conf. 2005, University College of Borås. Swedish School of Library and Information Science, pp. 103–110 (2005)

    Google Scholar 

  17. Julier, S., Baillot, Y., Brown, D., Lanzagorta, M.: Information filtering for mobile augmented reality. IEEE Computer Graphics and Applications 22(5), 12–15 (2002)

    Article  Google Scholar 

  18. Thomas, B., Piekarski, W., Hepworth, D., Gunther, B., Demczuk, V.: A wearable computer system with augmented reality to support terrestrial navigation. In: ISWC 1998: Proceedings of the 2nd IEEE International Symposium on Wearable Computers, Washington, DC, USA, p. 168. IEEE Computer Society, Los Alamitos (1998)

    Google Scholar 

  19. Feiner, S., MacIntyre, B., Hollerer, T., Webster, A.: A touring machine: Prototyping 3d mobile augmented reality systems for exploring the urban environment. In: ISWC 1997: Proceedings of the 1st IEEE International Symposium on Wearable Computers, Washington, DC, USA, p. 74. IEEE Computer Society, Los Alamitos (1997)

    Google Scholar 

  20. Rekimoto, J.: Navicam: A magnifying glass approach to augmented reality. In: Presence, vol. 6, pp. 399–412 (1997)

    Google Scholar 

  21. Thomas, B., Close, B., Donoghue, J., Squires, J., Bondi, P., Piekarski, W.: First Person Indoor/Outdoor Augmented Reality Application: ARQuake. In: Personal and Ubiquitous Computing, vol. 6, pp. 75–86. Springer, Heidelberg (2002)

    Google Scholar 

  22. Cheok, A., Goh, K., Liu, W., Farbiz, F., Fong, S., Teo, S., Li, Y., Yang, X.: Human Pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing. In: Personal and Ubiquitous Computing, vol. 8, pp. 71–81. Springer, Heidelberg (2004)

    Google Scholar 

  23. Avery, B., Thomas, B., Velikovsky, J., Piekarski, W.: Outdoor augmented reality gaming on five dollars a day. In: Proceedings of the Sixth Australasian conference on User interface, vol. 40, pp. 79–88. Australian Computer Society, Inc, Darlinghurst (2005)

    Google Scholar 

  24. Szalavári, Z., Eckstein, E., Gervautz, M.: Collaborative gaming in augmented reality. In: Proceedings of the ACM symposium on Virtual reality software and technology 1998, pp. 195–204. ACM Press, New York (1998)

    Chapter  Google Scholar 

  25. Barakonyi, I., Weilguny, M., Psik, T., Schmalstieg, D.: Monkeybridge: autonomous agents in augmented reality games. In: Lee, N. (ed.) Advances in Computer Entertainment Technology, pp. 172–175. ACM, New York (2005)

    Google Scholar 

  26. Bungert, C.: Hmd/headset/vr-helmet comparison chart (2008), http://www.stereo3d.com/hmd.htm

  27. Rekimoto, J., Nagao, K.: The world through the computer: computer augmented interaction with real world environments. In: Proceedings of the 8th annual ACM symposium on User interface and software technology, pp. 29–36. ACM, New York (1995)

    Google Scholar 

  28. Moehring, M., Lessig, C., Bimber, O.: Video See-Through AR on Consumer Cell Phones. In: Proc. of International Symposium on Augmented and Mixed Reality (ISMAR 2004), pp. 252–253 (2004)

    Google Scholar 

  29. Olwal, A.: Lightsense: enabling spatially aware handheld interaction devices. In: ISMAR, pp. 119–122 (2006)

    Google Scholar 

  30. Wellner, P.: Interacting with paper on the DigitalDesk. In: Communications of the ACM, vol. 36, pp. 87–96. ACM Press, New York (1993)

    Google Scholar 

  31. Fitzmaurice, G., Ishii, H., Buxton, W.: Bricks: laying the foundations for graspable user interfaces. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 442–449. ACM Press/Addison-Wesley Publishing Co., New York (1995)

    Google Scholar 

  32. Pinhanez, C.: Augmenting Reality with Projected Interactive Displays. In: Virtual and Augmented Architecture (Vaa 2001): Proceedings of the International Symposium on Virtual and Augmented Architecture (Vaa 2001), Trinity College, Dublin, June 21-22 (2001)

    Google Scholar 

  33. Olwal, A., Gustafsson, J., Lindfors, C.: Spatial augmented reality on industrial cnc-machines. In: Proceedings of SPIE 2008 Electronic Imaging. The Engineering Reality of Virtual Reality 2008, vol. 6804 (2008)

    Google Scholar 

  34. Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S., Forlines, C.: iLamps: geometrically aware and self-configuring projectors. In: International Conference on Computer Graphics and Interactive Techniques. ACM Press, New York (2006)

    Google Scholar 

  35. Getting, I.: Perspective/navigation-The Global Positioning System. In: Spectrum, vol. 30. IEEE, Los Alamitos (1993)

    Google Scholar 

  36. Steed, A.: Supporting Mobile Applications with Real-Time Visualisation of GPS Availability. In: Proceedings of Mobile HCI. Springer, Heidelberg (2004)

    Google Scholar 

  37. Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., Tachibana, K.: Virtual object manipulation on a table-top AR environment. In: Proc. IEEE and ACM International Symposium on Augmented Reality (ISAR), pp. 111–119 (2000)

    Google Scholar 

  38. Costanza, E., Robinson, J.: A region adjacency tree approach to the detection and design of fiducials. In: Vision, Video and Graphics (VVG), pp. 63–70 (2003)

    Google Scholar 

  39. Lepetit, V., Fua, P.: Keypoint Recognition Using Randomized Trees. In: IEEE Transactions On Pattern Analysis And Machine Intelligence, pp. 1465–1479. IEEE Computer Society, Los Alamitos (2006)

    Google Scholar 

  40. Polhemus: Polhemus “fastrak” commercial system (retrieved, June 2008), http://www.polhemus.com/?page=Motion_Fastrak

  41. Ascension: Ascension “flock of birds” commercial system (retrieved, June 2008), http://www.ascension-tech.com/products/flockofbirds.php

  42. Foxlin, E., Harrington, M., Harrington, M., Pfeifer, G.: Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications. In: Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pp. 371–378. ACM New York, NY (1998)

    Google Scholar 

  43. Yokokohji, Y., Sugawara, Y., Yoshikawa, T.: Accurate image overlay on video see-through hmds using vision and accelerometers. In: VR 2000: Proceedings of the IEEE Virtual Reality 2000 Conference, Washington, DC, USA, p. 247. IEEE Computer Society, Los Alamitos (2000)

    Google Scholar 

  44. Kotake, D., Satoh, K., Uchiyama, S., Yamamoto, H.: A hybrid and linear registration method utilizing inclination constraint. In: Proceedings of the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 140–149 (2005)

    Google Scholar 

  45. Azuma, R., Bishop, G.: A frequency-domain analysis of head-motion prediction. In: SIGGRAPH 1995: Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pp. 401–408. ACM, New York (1995)

    Chapter  Google Scholar 

  46. Dourish, P.: Where the action is: the foundations of embodied interaction. MIT Press, Cambridge (2001)

    Google Scholar 

  47. Swan, J.E., Gabbard, J.L.: Survey of user-based experimentation in augmented reality. In: Proceedings of 1st International Conference on Virtual Reality (2005)

    Google Scholar 

  48. Fjeld, M.: Special Issue on Usability and collaborative aspects of augmented reality. Interactions 11(6), 11–15 (2004)

    Article  Google Scholar 

  49. Billinghurst, M., Belcher, D., Gupta, A., Kiyokawa, K.: Communication Behaviors in Colocated Collaborative AR Interfaces. International Journal of Human-Computer Interaction 16(3), 395–423 (2003)

    Article  Google Scholar 

  50. Wiedenmaier, S., Oehme, O., Schmidt, L., Luczak, H.: Augmented reality (ar) for assembly processes design and experimental evaluation. Int. J. Hum. Comput. Interaction 16(3), 497–514 (2003)

    Article  Google Scholar 

  51. Papagiannakis, G., Schertenleib, S., Ponder, M., Arévalo-Poizat, M., Magnenat-Thalmann, N., Thalmann, D.: Real-time virtual humans in ar sites. In: 1st European Conference on Visual Media Production (CVMP), pp. 273–276. IEEE Publisher, Los Alamitos (2004)

    Google Scholar 

  52. Bianchi, G., Knörlein, B., Szèkely, G., Harders, M.: High precision augmented reality haptics. In: Eurohaptics (2006)

    Google Scholar 

  53. Bianchi, G., Wengert, C., Harders, M., Cattin, P., Székely, G.: Camera-marker alignment framework and comparison with hand-eye calibration for augmented reality applications. In: ISMAR (2005)

    Google Scholar 

  54. Fjeld, M., Lauche, K., Bichsel, M., Voorhorst, F., Krueger, H., Rauterberg, M.: Physical and virtual tools: Activity theory applied to the design of groupware. In: Computer Supported Cooperative Work (CSCW), vol. 11, pp. 153–180 (2002)

    Google Scholar 

  55. Fjeld, M., Morf, M., Krueger, H.: Activity theory and the practice of design: evaluation of a collaborative tangible user interface. Inderscience 4, 94–116 (2004)

    Google Scholar 

  56. Ganser Schwab, C., Kennel, T., Kunz, A.: Digital support for net-based teamwork in early design stages. Journal of Design Research 6(1), 150–168 (2007)

    Article  Google Scholar 

  57. Schwab, C.G., Kennel, T., Kunz, A.: Digital support for net-based teamwork in early design stages 6(1/2), 150–168 (2007)

    Google Scholar 

  58. Gross, M.H., Würmlin, S., Näf, M., Lamboray, E., Spagno, C.P., Kunz, A.M., Koller-Meier, E., Svoboda, T., Gool, L.J.V., Lang, S., Strehlke, K., Moere, A.V., Staadt, O.G.: Blue-c: a spatially immersive display and 3d video portal for telepresence. ACM Trans. Graph. 22(3), 819–827 (2003)

    Article  Google Scholar 

  59. Kuechler, M., Kunz, A.: Holoport - a device for simultaneous video and data conferencing featuring gaze awareness. In: Proceedings of the IEEE conference on Virtual Reality (IEEE VR 2006), Washington, DC, United States, pp. 81–87. IEEE Computer Society, Los Alamitos (2006)

    Chapter  Google Scholar 

  60. Kunz, A., Fadel, G., Taiber, J., Schichtel, M.: Towards collaboration in engineering of tomorrow - building highly interactive virtual collaboration platforms. In: SAE 2006 World Congress & Exhibition, United States, SAE International (2006)

    Google Scholar 

  61. Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In: IWAR 1999: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality. IEEE Computer Society, Los Alamitos (1999)

    Google Scholar 

  62. Voegtli, B.: Augmented collaboration. M.sc. thesis, HyperWerk FHNW (2002)

    Google Scholar 

  63. Fjeld, M., Fredriksson, J., Ejdestig, M., Duca, F., Bötschi, K., Voegtli, B.M., Juchli, P.: Tangible user interface for chemistry education: comparative evaluation and re-design. In: CHI, pp. 805–808 (2007)

    Google Scholar 

  64. Iten, C., Lüthi, D.: (2006), http://www.zima.ch/tangent/

Download references

Author information

Authors and Affiliations

  1. EPFL Media and Design Laboratory, Lausanne, Switzerland

    Enrico Costanza

  2. ETHZ Innovation Center Virtual Reality (ICVR), Zurich, Switzerland

    Andreas Kunz

  3. Chalmers TableTop Interaction Lab (t2i lab), Gothenburg, Sweden

    Morten Fjeld

Authors
  1. Enrico Costanza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Kunz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morten Fjeld
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Fribourg, Bd. de Pérolles 90, CH-1700, Fribourg, Switzerland

    Denis Lalanne  & Jürg Kohlas  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Costanza, E., Kunz, A., Fjeld, M. (2009). Mixed Reality: A Survey. In: Lalanne, D., Kohlas, J. (eds) Human Machine Interaction. Lecture Notes in Computer Science, vol 5440. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00437-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-00437-7_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-00436-0

  • Online ISBN: 978-3-642-00437-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation