The Skyline as a Marker for Augmented Reality in Urban Context

  • Mehdi AyadiEmail author
  • Leo Valque
  • Mihaela Scuturici
  • Serge Miguet
  • Chokri Ben Amar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11241)


In recent years, augmented reality (AR) technologies have emerged as powerful tools to help visualize the future impacts of new constructions on cities. Many approaches that use costly sensors and height-end platforms to run AR in real-time have been developed. Little efforts have been made to embed AR on mobile phones. In this paper, we present a novel approach that uses the Skyline as a marker in an AR system. This lightweight feature enables real-time matching of virtual and real skyline on smartphones.

We use device’s embedded instruments to estimate the user’s pose. This approximation is used to insert a synthetic object in the live video stream. This first approach gives a very unrealistic impression of the viewed scene: the inserted objects appear to hover and float with the user’s movements. In order to address this problem, we use the live video camera feed as additional source of information which provides a redundancy to the instruments estimation. We extract the Skyline (a set of pixels that defines the boundary between the building and the sky) as main visual feature. Our proposal is to use these automatically extracted points and track them throughout the video sequence, to allow synthetic objects to anchor these visual features, making it possible to simulate a landscape from multiple viewpoints using a smartphone. We use images of the Lyon city (France) to illustrate our proposal.


Mobile augmented reality Image to geometry registration Image comparison metric 3D models Skyline matching Urban landscape 


  1. 1.
    Schmalstieg, D., Langlotz, T., Billinghurst, M.: Augmented Reality 2.0. In: Brunnett, G., Coquillart, S., Welch, G. (eds.) Virtual Realities, pp. 13–37. Springer, Vienna (2011). Scholar
  2. 2.
    Langlotz, T., Mooslechner, S., Zollmann, S., Degendorfer, C., Reitmayr, G., Schmalstieg, D.: Sketching up the world: in situ authoring for mobile augmented reality. Pers. Ubiquitous Comput. 16(6), 623–630 (2012)CrossRefGoogle Scholar
  3. 3.
    Gotow, J.B., Zienkiewicz, K., White, J., Schmidt, D.C.: Addressing challenges with augmented reality applications on smartphones. In: Cai, Y., Magedanz, T., Li, M., Xia, J., Giannelli, C. (eds.) MOBILWARE 2010. LNICST, vol. 48, pp. 129–143. Springer, Heidelberg (2010). Scholar
  4. 4.
    Wiggenhagen, M.: Co-registration of terrestrial laser scans and close range digital images using scale invariant features. Plastverarbeiter.De, pp. 208–212 (2010)Google Scholar
  5. 5.
    Moussa, W., Abdel-Wahab, M., Fritsch, D.: An automatic procedure for combining digital images and laser scanner data. ISPRS Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. XXXIX-B5, 229–234 (2012)CrossRefGoogle Scholar
  6. 6.
    Taylor, Z., Nieto, J.: Automatic calibration of lidar and camera images using normalized mutual information, p. 8 (2012)Google Scholar
  7. 7.
    Taylor, Z., Nieto, J., Johnson, D.: Automatic calibration of multi-modal sensor systems using a gradient orientation measure. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1293–1300, November 2013Google Scholar
  8. 8.
    Hofmann, S., Eggert, D., Brenner, C.: Skyline matching based camera orientation from images and mobile mapping point clouds. ISPRS Ann. Photogramm. Remote. Sens. Spat. Inf. Sci. II-5, 181–188 (2014)CrossRefGoogle Scholar
  9. 9.
    Ramalingam, S., Bouaziz, S., Sturm, P., Brand, M.: Skyline2gps: localization in urban canyons using omni-skylines. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3816–3823, October 2010Google Scholar
  10. 10.
    Nüchter, A., Gutev, S., Borrmann, D., Elseberg, J.: Skyline-based registration of 3d laser scans. Geo Spat. Inf. Sci. 14(2), 85 (2011)CrossRefGoogle Scholar
  11. 11.
    Zhu, S., Pressigout, M., Servières, M., Morin, L., Moreau, G.: Skyline matching: a robust registration method between Video and GIS. In: Conference of the European COST Action TU0801 - Semantic Enrichment of 3D City Models for Sustainable Urban Development Location: Graduate School of Architecture, Nantes, France, pp. 1–6, October 2012Google Scholar
  12. 12.
    Zhu, S., Morin, L., Pressigout, M., Moreau, G., Servières, M.: Video/GIS registration system based on skyline matching method. In: 2013 IEEE International Conference on Image Processing, pp. 3632–3636, September 2013Google Scholar
  13. 13.
    Guislain, M., Digne, J., Chaine, R., Monnier, G.: Fine scale image registration in large-scale urban lidar point sets. Comput. Vis. Image Underst. 157, 90–102 (2017). Large-Scale 3D Modeling of Urban Indoor or Outdoor Scenes from Images and Range ScansGoogle Scholar
  14. 14.
    Yusoff, N.A.H., Noor, A.M., Ghazali, R.: City skyline conservation: sustaining the premier image of Kuala Lumpur. Procedia Environ. Sci. 20, 583–592 (2014)CrossRefGoogle Scholar
  15. 15.
    Ayadi, M., Suta, L., Scuturici, M., Miguet, S., Ben Amar, C.: A parametric algorithm for skyline extraction. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2016. LNCS, vol. 10016, pp. 604–615. Springer, Cham (2016). Scholar
  16. 16.
    Fukuda, T., Zhang, T., Yabuki, N.: Improvement of registration accuracy of a handheld augmented reality system for urban landscape simulation. Front. Arch. Res. 3, 386–397 (2014)CrossRefGoogle Scholar
  17. 17.
    Skyline Database: \(\)/\(~mayadi/ISVC^{\prime }18/skylineDatabase\)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mehdi Ayadi
    • 1
    • 2
    Email author
  • Leo Valque
    • 1
  • Mihaela Scuturici
    • 1
  • Serge Miguet
    • 1
  • Chokri Ben Amar
    • 2
  1. 1.University of Lyon, CNRS, University Lyon 2, LIRIS, UMR5205LyonFrance
  2. 2.University of Sfax, ENIS, REGIM-Lab: REsearch Groups in Intelligent MachinesSfaxTunisia

Personalised recommendations