Virtual Reality

, 11:161 | Cite as

M2S maps: supporting real-world navigation with mobile VR

Original Article


Mobile devices are becoming increasingly integrated into our society. In addition to entertaining people with applications like pervasive games, mobile devices can also help to address cognitive challenges people face in the real world. This paper, by drawing on research findings from cognitive psychology and geography, explores a design to use mobile VR to help people overcome one cognitive barrier in navigation, which is to establish the correspondence between 2D spatial information found in maps and 3D entities they perceive from the real world. The design offers users multi-format, multi-scale, and semantic (M2S) maps, ranging from 2D maps to 3D immersive environments, and helps users to connect 2D maps to the real world through 3D environments which are equipped with semantic representation and animation techniques. Consequently, users can apply various kinds of spatial knowledge, 2D or 3D, in understanding the real world as well as assisting in navigation. This research enhances the design repertoire of mobile VR, and suggests a way to integrate virtual environments into people’s real-world life by examining the cognitive implications of 3D models on users’ activities.


Virtual environments Mobile computing Multi-scale Navigation Spatial cognition 


  1. Abowd G, Dey A, Brown P, et al (1999) Towards a better understanding of context and context-awareness. In: Proceedings of the 1st international symposium on handheld and ubiquitous computing (HUC’99), pp 304–307Google Scholar
  2. Bagrow L (1985) History of cartography. Precedent Publishers, ChicagoGoogle Scholar
  3. Barkowsky T, Freksa C (1997) Cognitive requirements on making and interpreting maps. In: Hirtle S, Frank A (eds) Spatial information theory: a theoretical basis for GIS. Proceedings of COSIT 97. Springer, Berlin Heidelberg New York, pp 347–361Google Scholar
  4. Baus J, Krüger A, Wahlster W (2002) A resource-adaptive mobile navigation system. In: Proceedings of the 7th annual international conference on intelligent user interfaces, pp 15–22Google Scholar
  5. Bellard, Fabrice, TinyGL, Retrieved December 20, 2005,
  6. Benini L, Bonfigli M, Calori L, Farella E, Ricc B (2002) Palmtop computers for managing interaction with immersive virtual heritage. In: Proceedings of EUROMEDIA, pp 183–189Google Scholar
  7. Bruntsch S, Rehrl K (2005) Vienna-SPIRIT—combining intermodal traveller information with onboard/offboard navigation services, GNSS. The European navigation conference, MunichGoogle Scholar
  8. Burigat S, Chittaro L (2005) Location-aware visualization of VRML models in GPS-based mobile guides. In: Proceedings of Web3D, pp 57–64Google Scholar
  9. Chalmers D, Dulay N, Sloman M (2004) A framework for contextual mediation in mobile and ubiquitous computing applied to the context-aware adaptation of maps. Personal Ubiquitous Comput 8:1–18CrossRefGoogle Scholar
  10. Chen JL, Stanney KM (1999) A theoretical model of wayfinding in virtual environments: proposed strategies for navigational aiding. Presence Teleoperators Virtual Environ 8(6):671–685CrossRefGoogle Scholar
  11. Darken R, Cevik H (1999) Map usage in virtual environments: orientation issues. In: Proceedings of IEEE virtual reality, pp 133–140Google Scholar
  12. Darken RP, Sibert J (1996) Navigating large virtual spaces. Int J Hum Comput Interact 8(1):49–71CrossRefGoogle Scholar
  13. Dearman D, Hawkey K, Inkpen K (2005) Effect of location-awareness on rendezvous behaviour. In: Extended abstract of ACM CHI, pp 1929–1932Google Scholar
  14. Downs R, Stea D (1973) Image and environment: cognitive mapping and spatial behavior. Aldine, ChicagoGoogle Scholar
  15. Elias B, Sester M (2002). Landmarks for routing—automatic identification, extraction and visualization. ISPRS/ICA joint workshop on multi-scale representations of spatial data, OttawaGoogle Scholar
  16. Elvins T, Nadeau D, Kirsh D (1997) Worldlets-3D thumbnails for wayfinding in virtual environments. In: Proceedings of the ACM UIST, pp 21–30Google Scholar
  17. Evans GW, Pezdek K (1980) Cognitive mapping: knowledge of real-world distance and location information. J Exp Psychol Hum Learn Mem 6(1):13–24CrossRefGoogle Scholar
  18. Finn MVN (2004) The handheld classroom : educational implications of mobile computing. Aust J Emerg Technol Soc 2(1):21–35Google Scholar
  19. Flintham M, Benford S, Anastasi R, et al (2003) Where on-line meets on the streets: experiences with mobile mixed reality games. In: Proceedings of ACM CHI, pp 569–576Google Scholar
  20. Fonseca F, Martin J (2005) Play as the way out of the newspeak-tower of babel dilemma in data modeling. In: McLean E, Monod E (eds) The 26th international conference on information systems, Las Vegas, pp 11–20Google Scholar
  21. Goble C, Wroe C (2004) The Montagues and the Capulets. Comparat Funct Genomics 5(8):618–622CrossRefGoogle Scholar
  22. Golledge R (1999) Human wayfinding and cognitive maps. In: Golledge R (ed) Wayfinding behavior: cognitive maps and other spatial processes. Johns Hopkins University Press, Baltimore, pp 5–45Google Scholar
  23. Goodman J, Gray P, Khammampad K, Brewster S (2004) Using landmarks to support older people in navigation. In: Proceedings of mobile HCI, pp 38–48Google Scholar
  24. Goose S, Wanning H, Schneider G (2003) Mobile reality: a PDA-based multimodal framework synchronizing a hybrid tracking solution with 3D graphics and location-sensitive speech interaction. Ubicomp, pp 33–47Google Scholar
  25. Hart R, Moore G (1973) The development of spatial cognition: a review. In: Stea B, Downs R (eds) Image and environment. University of Chicago Press, Chicago, pp 226–234Google Scholar
  26. Headon R, Curwen R (2002) Movement awareness for ubiquitous game control. Personal Ubiquitous Comput 6:407–415CrossRefGoogle Scholar
  27. Hommel B, Knuf L (2003). Acquisition of cognitive aspect maps. In: Freksa C, Brauer W, Habel C, Wender KF (eds) Spatial cognition III: routes and navigation, human memory and learning, spatial representation and spatial learning. Springer, Berlin Heidelberg New York, pp 157–173Google Scholar
  28. Hommel B, Gehrke J, Knuf L (2000) Hierarchical coding in the perception and memory of spatial layouts. Psychol Res 64:1–10CrossRefGoogle Scholar
  29. Jul S, Furnas GW (1997) Navigation in electronic worlds. SIGCHI Bull 29(4):44–49CrossRefGoogle Scholar
  30. Kray C, Elting C, Laakso K, Coors V (2003) Presenting route instructions on mobile devices. In: Proceedings of IUI, pp 117–124Google Scholar
  31. Krüger A, Butz A, Müller C, et al (2004) The connected user interface: realizing a personal situated navigation service. In: Proceedings of IUI, pp 161–168Google Scholar
  32. LaViola J, Feliz D, Keefe D, Zeleznick R (2001) Hands-free multi-scale navigation in virtual environments. In: Proceedings of the symposium on interactive 3D graphics, pp 9–15Google Scholar
  33. Leigh J, Johnson A, DeFanti T (1996) CALVIN: an immersimedia design environment utilizing heterogeneous perspectives. In: Proceedings of IEEE international conference on multimedia computing and systems, pp 20–23Google Scholar
  34. Levine D, Warach J, Farah M (1985) Two visual systems in mental imagery: dissociation of "what" and "where" in imagery disorders due to bilateral posterior cerebral lesions. Neurology 35:1010–1018Google Scholar
  35. Loomis JM, Beall AC (1998) Visually controlled locomotion: Its dependence on optic flow, 3-D space perception, and cognition. Ecol Psychol 10:271–285CrossRefGoogle Scholar
  36. Lynch K (1960) Image of the city. MIT Press, CambridgeGoogle Scholar
  37. Magerkurth C, Cheok A, Mandryk R, Nilsen T (2005) Pervasive games: bringing computer entertainment back to the real world. Comput Entertain (CIE) 3(3)Google Scholar
  38. Malaka R, Zipf A (2000) DEEP MAP—challenging IT research in the framework of a tourist information system. In: Proceedings of ENTER, pp 15–27Google Scholar
  39. Marsden G, Tip N (2005) Navigation control for mobile virtual environments. In: Proceedings of mobile HCI, pp 279–282Google Scholar
  40. May A, Ross T, Bayer S, Tarkiainen M (2003) Pedestrian navigation aids: information requirements and design implications. Personal Ubiquitous Comput 7:331–338CrossRefGoogle Scholar
  41. McNamara TP, Hardy JK, Hirtle SC (1989) Subjective hierarchies in spatial memory. J Exp Psychol Learn Mem Cogn 15:211–227CrossRefGoogle Scholar
  42. Narzt W, Pomberger G, Ferscha A, et al. (2003) Pervasive information acquisition for mobile AR-navigation systems. Fifth IEEE workshop on mobile computing systems & applicationsGoogle Scholar
  43. Narzt W, Pomberger G, Ferscha A, et al (2004) A new visualization concept for navigation systems. Eighth ERCIM workshop: user interfaces for all (UI4All), pp 440–451Google Scholar
  44. Newcombe N, Huttenlocher J (2000) Making space: the development of spatial representation and reasoning. MIT Press, CambridgeGoogle Scholar
  45. Parent R (2001) Computer animation: algorithms and techniques. Morgan–Kaufmann, San FranciscoGoogle Scholar
  46. Piaget J, Inhelder B (1967) The child’s conception of space. Norton, New YorkGoogle Scholar
  47. Pierce J, Pausch R (2004) Navigation with place representations and visible landmarks. In: Proceedings of VRST, pp 173–180Google Scholar
  48. Puppo E, Scopigno R (1997) Simplification, LOD, multiresolution: principles and applications. In: Proceedings of EUROGRAPHICS 16(3)Google Scholar
  49. Rakkolainen I, Vainio T (2003) A 3D city info for mobile users. Comput Graph 25(4):619–625CrossRefGoogle Scholar
  50. Raubal M, Winter S (2002). Enriching wayfinding instructions with local landmarks. In: Egenhofer M, Mark D (eds) Geographic Information Science. Lecture Notes in Computer Science. Springer, Berlin Heidelberg New York, pp 243–259Google Scholar
  51. Reid J, Lipson M, Hyams J, Shaw K (2004) Fancy a schmink?: a novel networked game in a caf. Adv Comput Entertain Technol 18–23Google Scholar
  52. Rodden T, Cheverst K, Davies K, Dix A (1998) Exploiting context in HCI design for mobile systems. Workshop on human computer interaction with mobile devicesGoogle Scholar
  53. Ruddle R, Payne S, Jones D (1997) Navigating buildings in "Desk-Top" virtual environments: experimental investigations using extended navigational experience. J Exp Psychol Appl 3(2):143–159CrossRefGoogle Scholar
  54. Schilit B, Adams N, Want R (1994) Context-aware computing applications. In: Proceedings of the workshop on mobile computing systems and applications, pp 85–90Google Scholar
  55. Schmidt A, Aidoo K, Takaluoma A, Tuomela U, van Laerhoven K, van de Velde W (1999) Advanced interaction in context. In: Proceedings of the 1st international symposium on handheld and ubiquitous computing (HUC’99), pp 89–101Google Scholar
  56. Sefelin R, Bechinie M, Müller R, et al. (2005) Landmarks: yes; but which? Five methods to select optimal landmarks for a landmark- and speech-based guiding system. In: Proceeding of mobile HCI, pp 287–290Google Scholar
  57. Slavík P, Berka R (1999) Navigation system for blind users in mobile computing environment. In: Interactive applications of mobile computing, pp 53–60Google Scholar
  58. Smith I, Consolvo S, LaMarca A, et al (2005) Social disclosure of place: from location technology to communication practices. Pervasive 134–151Google Scholar
  59. Stevens A, Coupe P (1978) Distortions in judged spatial relations. Cogn Psychol 10:422–437CrossRefGoogle Scholar
  60. Stoakley R, Conway M, Pausch R (1995) Virtual reality on a WIM: interactive worlds in miniature. In: Proceedings of the ACM CHI, pp 265–272Google Scholar
  61. SundialSoft, PocketGL, Retrieved December 20, 2005,
  62. Suomela R, Roimela K, Lehikoinen J (2003) The evolution of perspective view in WalkMap. Personal Ubiquitous Comput 7(5):249–262CrossRefGoogle Scholar
  63. Thorndyke P, Hayes-Roth B (1982) Differences in spatial knowledge acquired from maps and navigation. Cogn Psychol 14:560–589CrossRefGoogle Scholar
  64. Timpf S, Kuhn W (2003) Granularity transformations in wayfinding. Spat Cogn 77–88Google Scholar
  65. Tolman EC (1948) Cognitive maps in rats and men. Psychol Rev 55:189–208CrossRefGoogle Scholar
  66. Vinson NG (1999) Design guidelines for landmarks to support navigation in virtual environments. In: Proceedings of the ACM CHI, pp 278–285Google Scholar
  67. Watsen K, Darken R, Capps M (1999) A handheld computer as an interaction device to a virtual environment. Third immersive projection technology workshopGoogle Scholar
  68. Watt A, Policarpo F (2001). 3D games: animation and advanced real-time rendering. Addison-Wesley, New YorkGoogle Scholar
  69. Wilson PN, Foreman N, Tlauka M (1997) Transfer of spatial information from a virtual to a real environment. Hum Factors 39(4):526–531CrossRefGoogle Scholar
  70. Witmer BG, Bailey JH, Knerr BW, Parsons KC (1996) Virtual spaces and real world places: transfer of route knowledge. Int J Hum Comput Stud 45:413–428CrossRefGoogle Scholar
  71. Yue W, Mu S, Wang G, Wang H (2005) TGH: a case study of designing natural interaction for mobile guide systems. In: Proceeding of mobile HCI, pp 199–206Google Scholar
  72. Zhang X, Furnas GW (2005) mCVEs: using cross-scale collaboration to support user interaction with multiscale structures. Presence Teleoperators Virtual Environ 14(1):31–46CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  1. 1.College of Information Sciences and TechnologyThe Pennsylvania State UniversityUniversity ParkUSA

Personalised recommendations