Multimodal Location Based Services—Semantic 3D City Data as Virtual and Augmented Reality

  • José Miguel Santana
  • Jochen Wendel
  • Agustín Trujillo
  • José Pablo Suárez
  • Alexander Simons
  • Andreas Koch
Conference paper
Part of the Lecture Notes in Geoinformation and Cartography book series (LNGC)


The visualization of cross-domain spatial data sets has become an important task within the analysis of energy models. The representation of these models is especially important in urban areas, in which the under-standing of patterns of energy production and demand is key for an efficient city planning. Location Based Services (LBS) provide a valuable addition towards the analysis and visualization of those data sets as the user can explore the output of different models and simulations in the real environment at the location of interest. Towards this aim, the present research explores mobile alternatives to the visual analysis of temporal data series and 3D building models. Based on the fields of numerical simulation, GIS and computer graphics, this work presents a novel mobile service that allows exploring urban models at different Level of Details (LoDs) using well-known standards such as CityGML. Ultimately, the project enables researchers, city planners and technicians to explore urban energy datasets in an interactive and immersive manner as Virtual Globes, Virtual Reality and Augmented Reality. Using models of the city of Karlsruhe, the final service has been implemented and tested on the iOS platform providing an empirical insight on the performance of the system. In addition, this research provides a holistic approach by developing one application that is capable of seamlessly change the visualization mode.


Augmented reality Virtual reality CityGML energy data Multimodal visualization GIS Temporal datasets rendering 



The first author wants to thank Agencia Canaria de Investigación, Innovación y Sociedad de la Información, and the European Social Fund, for the grant “Formación del Personal Investigador-2012” that made possible this work.


  1. Ahas, R. and Mark, Ü. (2005). Location based services—new challenges for planning and public administration? Futures, 37 (6), 547–561. doi: 10.1016/j.futures.2004.10.012
  2. Anthopoulos, L. and Fitsilis, P. (2010). From Digital to Ubiquitous Cities: Defining a Common Architecture for Urban Development. 2010 Sixth International Conference on Intelligent Environments (S. 301–306). IEEE. doi: 10.1109/IE.2010.61
  3. Apple Inc. (2014a). AVCaptureDeviceFormat Class Reference.Google Scholar
  4. Apple Inc. (2014b). Grand Central Dispatch (GCD) Reference.Google Scholar
  5. Apple Inc. (2016a). iOS Human Interface Guidelines. IOS Developer Library. Accessed 10 June 2016
  6. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S. and MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21 (6), 34–47. doi: 10.1109/38.963459
  7. Biljecki, F., Ledoux, H. and Stoter, J. (2016). An improved LOD specification for 3D building models. Computers, Environment and Urban Systems (59). doi: 10.1016/j.compenvurbsys.2016.04.005
  8. Biljecki, F., Stoter, J., Ledoux, H., Zlatanova, S. and Çöltekin, A. (2015). Applications of 3D City Models: State of the Art Review. ISPRS International Journal of Geo-Information, 4 (4), 2842–2889. Multidisciplinary Digital Publishing Institute. doi: 10.3390/ijgi4042842
  9. Blum, J.R., Greencorn, D. and Cooperstock, J.R. (2013). Smartphone sensor reliability for augmented reality applications. Mobile and ubiquitous systems: computing, networking, and services, 127–138. doi: 10.1007/978-3-642-40238-8_11
  10. Boas, Y. (2013). Overview of virtual reality technologies. Interactive Multimedia Conference 2013.Google Scholar
  11. Bolstad, L.E., Jackson, D., Nilsson, C., Percivall, G. and Turner, D.M. (2011) DeviceOrientation Event Specificaiton. W3C Working Draft. Accessed 22 August 2016.
  12. Celik, B., Karatepe, E., Gokmen, N. and Silvestre, S. (2013). A virtual reality study of surrounding obstacles on BIPV systems for estimation of long-term performance of partially shaded PV arrays. Renewable Energy, 60, 402–414. doi: 10.1016/j.renene.2013.05.040
  13. Civil, A. (2008). Best Practices for Working with Vertex Data. Data Management.Google Scholar
  14. Clark, J. and DeRose, S. (2010). XML Path Language (XPath). Engineering. Accesed 27th September 2016.
  15. Döllner, J., Kolbe, T.H., Liecke, F., Sgouros, T. and Teichmann, K. (2006). The Virtual 3D City Model of Berlin-Managing, Integrating, and Communicating Complex Urban Information. Proceedings of the 25th Urban Data Management Symposium UDMS, (May 2006), 15–17.Google Scholar
  16. Eberly, D. (2002). Triangulation by ear clipping. Magic Software, Inc, 1–13.Google Scholar
  17. Feiner, S., MacIntyre, B., Höllerer, T. and Webster, A. (1997). A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment. Personal Technologies, 1 (4), 208–217. Springer-Verlag. doi: 10.1007/BF01682023
  18. Fernández, P., Santana, J.M., Ortega, S., Trujillo, A., Suárez, J.P., Domínguez, C. et al. (2016). SmartPort: A Platform for Sensor Data Monitoring in a Seaport Based on FIWARE. Sensors, 16 (3), 417. Multidisciplinary Digital Publishing Institute. doi: 10.3390/xx
  19. Gateau, S. (2009). 3D Vision Technology - Develop, Design, Play in 3D Stereo. Acm Siggraph 2009.Google Scholar
  20. Gröger, G., Kolbe, T.H., Czerwinski, A. and Nagel, C. (2008). OpenGIS City Geography Markup Language (CityGML) Encoding Standard, Version 1.0.0. OGC Document No. 08-007r1, 234.Google Scholar
  21. Harrower, M., and Brewer, C. A. (2003). an online tool for selecting colour schemes for maps. The Cartographic Journal40(1), 27–37.Google Scholar
  22. Heuveline, V., Ritterbusch, S. and Ronnas, S. (2011). Augmented Reality for Urban Simulation Visualization. INFOCOMP 2011 : The First International Conference on Advanced Communications and Computation, (c), 115–119.Google Scholar
  23. Heydarian, A., Carneiro, J.P., Gerber, D., Becerik-Gerber, B., Hayes, T. and Wood, W. (2015). Immersive virtual environments versus physical built environments: A benchmarking study for building design and user-built environment explorations. Automation in Construction, 54, 116–126. doi: 10.1016/j.autcon.2015.03.020
  24. Höllerer, T., Feiner, S., Terauchi, T., Rashid, G. and Hallaway, D. (1999). Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system. Computers and Graphics, 23 (6), 779–785. doi: 10.1016/S0097-8493(99)00103-X
  25. Jenks, G. F. (1977). Optimal data classification for choropleth maps. University of Kansas. Dept. of Geography.Google Scholar
  26. Klobe, T. H. Burger, B., and Berit, C. (2015). City GML goes to Broadway. Photogrammetric Week ‘15, Wichmann, 2015, 343–356Google Scholar
  27. Krevelen, D.W.F. van and Poelman, R. (2010). A Survey of Augmented Reality Technologies, Applications and Limitations. The International Journal of Virtual Reality, 9 (2), 1–20. doi: 10.1155/2011/721827
  28. Lovett, A., Appleton, K., Warren-Kretzschmar, B. and Von Haaren, C. (2015). Using 3D visualization methods in landscape planning: An evaluation of options and practical issues. Landscape and Urban Planning, 142, 85–94. doi: 10.1016/j.landurbplan.2015.02.021
  29. Megginson, D. (2004). SAX. Accessed 15 June 2016
  30. Meisters, G.H. (1975). Polygons have ears. American Mathematical Monthly, 82 (6), 648–651. Mathematical Association of America.Google Scholar
  31. Nouvel, R., Zirak, M., Dastageeri, H., Coors, V. and Eicker, U. (2014). Urban Energy Analysis Based on 3D City Model for National Scale Applications. IBPSA Germany Conference.Google Scholar
  32. Ortega, S., Trujillo, A., Santana, J.M., Suárez, J.P. and Gómez-Deck, D. (2016). Rendering large datasets of georeferenced markers in mobile devices, In Press. doi: 10.1145/2948628.2948638
  33. Paolantonio, M. Di, Fernández, C.G., Latorre, M.J. and Pedrera, F. (2015). 3D virtual representation of drones’ flights through Cesium. js and Oculus Rift. Geomatics Workbooks n 12 – „FOSS4G Europe Como 2015“, 577–578.Google Scholar
  34. Phong, B.T. (1975). Illumination for Computer-Generated Images. Communications of the ACM. The University of Utah.Google Scholar
  35. Prandi, F., Staso, U. Di, Berti, M., Giovannini, L. and Amicis, R. De. (2015). Hybrid approach for large-scale Energy Performance estimation based on 3D city model data and typological classification, (November), 329–330.Google Scholar
  36. Ratti, C., Frenchman, D., Pulselli, R.M. and Williams, S. (2006). Mobile Landscapes: Using Location Data from Cell Phones for Urban Analysis. Environment and Planning B: Planning and Design, 33 (5), 727–748. doi: 10.1068/b32047
  37. Rehman, U. and Cao, S. (2015). Augmented Reality-Based Indoor Navigation Using Google Glass as a Wearable Head-Mounted Display. Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on (S. 1452–1457).Google Scholar
  38. Rosenberg, L.B. (1993). The effect of interocular distance upon operator performance using stereoscopic displays to perform virtual depth tasks. Virtual Reality Annual International Symposium, 1993. IEEE (S. 27–32). doi: 10.1109/VRAIS.1993.380802
  39. Saed, S. and Wendel, J. (2015). Estimating heating energy consumption and CO2 production - A novel modeling approach. Proceedings of the 14th International Conference on Computers in Urban Planning and Urban Management. Google Scholar
  40. Schall, G., Schmalstieg, D. and Junghanns, S. (2010). Vidente - 3D Visualization of Underground Infrastructure using Handheld Augmented Reality. Integrating GIS and Water, 1 (4), 1–17. doi: Scholar
  41. Shen, Z. (2012). Geospatial Techniques in Urban Planning (Advances in Geographic Information Science). Springer Berlin Heidelberg.Google Scholar
  42. Simons, A. and Nichersu, A. (2014). Development of a CityGML infrastructure for the imple-mentation of an energy demand method with different data sources. GIScience 2014.Google Scholar
  43. Suárez, J.P., Trujillo, A., De La Calle, M., Gómez-Deck, D. and Santana, J.M. (2012). An open source virtual globe framework for iOS, Android and WebGL compliant browser. Proceedings of the 3rd International Conference on Computing for Geospatial Research and Applications.Google Scholar
  44. Suárez, J.P., Trujillo, A., Santana, J.M., de la Calle, M. and Gómez-Deck, D. (2015). An efficient terrain Level of Detail implementation for mobile devices and performance study. Computers, Environment and Urban Systems, 52, 21–33. Elsevier. doi: 10.1016/j.compenvurbsys.2015.02.004
  45. Tanaka, E.H., Paludo, J.A., Cordeiro, C.S., Domingues, L.R., Gadbem, E. V. and Euflausino, A. (2015). Using Immersive Virtual Reality for Electrical Substation Training. International Association for Development of the Information Society. International Association for the Development of the Information Society.Google Scholar
  46. Trapp, M. (2004). OpenGL-Performance and Bottlenecks. Hasso Plattner Institut - Universität Potsdam, (January 2004), 15 pages.Google Scholar
  47. Trujillo, A., Suárez, J.P., La Calle, M. De, Gómez, D., Pedriza, A. and Santana, J.M. (2013). Glob3 mobile: An open source framework for designing virtual globes on iOS and android mobile devices. Lecture Notes in Geoinformation and Cartography (S. 211–229). Quebec, Canada: Springer. doi: 10.1007/978-3-642-29793-9-12
  48. Trujillo, A., Suárez, J.P., Santana, J.M., De La Calle, M. and Gómez-Deck, D. (2014). An efficient architecture for automatic shaders management on virtual globes. Proceedings - 5th International Conference on Computing for Geospatial Research and Application, COM.Geo 2014 (S. 38–42). doi: 10.1109/COM.Geo.2014.23
  49. Tussyadiah, I. (2013). Expectation of Travel Experiences with Wearable Computing Devices. Information and Communication Technologies in Tourism 2014 (S. 539–552). Cham: Springer International Publishing. doi: 10.1007/978-3-319-03973-2_39
  50. Veillard, D. (2009). The XML C parser and toolkit of Gnome. Accessed 10 June 2016
  51. Wendel, J., Murshed, S.M., Sriramulu, A. and Nichersu, A. (2016). Development of a Web-Browser Based Interface for 3D Data—A Case Study of a Plug-in Free Approach for Visualizing Energy Modelling Results (S. 185–205). Springer International Publishing. doi: 10.1007/978-3-319-19602-2_12
  52. Wieland, M. and Wendel, J. (2015). Computing Solar Radiation on CityGML Building Data. 18th AGILE International conference on Geographic Information Science 2015. Lisbon, Portugal. 9th June 2015Google Scholar
  53. Wiki, O. (2013). Tile usage policy --- OpenStreetMap Wiki. Accessed 28 May 2016
  54. World Wide Web Consortium. (2006). W3C Document Object Model. Accessed 10 June 2016
  55. Zamyadi, A., Pouliot, J. and Bédard, Y. (2013). A Three Step Procedure to Enrich Augmented Reality Games with CityGML 3D Semantic Modeling (S. 261–275). Springer Berlin Heidelberg. doi: 10.1007/978-3-642-29793-9_15

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • José Miguel Santana
    • 1
  • Jochen Wendel
    • 2
  • Agustín Trujillo
    • 1
  • José Pablo Suárez
    • 1
  • Alexander Simons
    • 2
  • Andreas Koch
    • 2
  1. 1.University of Las Palmas de Gran CanariaLas Palmas de G.C.Spain
  2. 2.European Institute for Energy Research (EIFER)KarlsruheGermany

Personalised recommendations