Machine Vision and Applications

, Volume 27, Issue 6, pp 943–962 | Cite as

Fast and automatic city-scale environment modelling using hard and/or weak constrained bundle adjustments

  • Dorra LarnaoutEmail author
  • Vincent Gay-Bellile
  • Steve Bourgeois
  • Michel Dhome
Original Paper


To provide high-quality augmented reality service in a car navigation system, accurate 6 degrees of freedom (DoF) localization is required. To ensure such accuracy, most current vision-based solutions rely on an off-line large-scale modelling of the environment. Nevertheless, while existing solutions to model the environment require expensive equipments and/or a prohibitive computation time, we propose in this paper a complete framework that automatically builds an accurate large-scale database of landmarks using only a standard camera, a low-cost global positioning system (GPS) and a geographic information system (GIS). As illustrated in the experiments, only few minutes are required to model large-scale environments. The resulting databases can then be used by an on-line localization algorithm to ensure high-quality augmented reality experiences.


Simultaneous localization and mapping Constrained bundle adjustment Global localization system Geographic information system Augmented reality 


  1. 1.
    Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR (2009)Google Scholar
  2. 2.
    Li, Y., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3d point clouds. In: ECCV (2012)Google Scholar
  3. 3.
    Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: ECCV (2010)Google Scholar
  4. 4.
    Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Real time localization and 3D reconstruction. In: CVPR (2006)Google Scholar
  5. 5.
    Taneja, A., Ballan, L., Pollefeys, M.: Registration of spherical panoramic images with cadastral 3D models. In: 3DIMPVT (2012)Google Scholar
  6. 6.
    Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S.M., Szeliski, R.: Building Rome in a day. Commun. ACM 54, 105–112 (2011)CrossRefGoogle Scholar
  7. 7.
    Frahm, J.M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Y. H. Jen, E. Dunn, B. Clipp, S. Lazebnik, M. Pollefeys, Building Rome on a cloudless day. In: ECCV (2010)Google Scholar
  8. 8.
    Klingner, B., Martin, D., Roseborough, J.: Street view motion-from-structure-from-motion. In: ICCV (2013)Google Scholar
  9. 9.
    Kaminsky, R.S., Snavely, N., Seitz, S.M., Szeliski, R.: Alignment of 3D point clouds to overhead images. In: CVPR Workshop (2009)Google Scholar
  10. 10.
    Strecha, C., Pylvänäinen, T., Fua, P.: Dynamic and scalable large scale image reconstruction. In: CVPR (2010)Google Scholar
  11. 11.
    Pylvänäinen, T., Roimela, K., Vedantham, R., Itäranta, J., Wang, R., Grzeszczuk, R.: Automatic alignment and multi-view segmentation of street view data using 3D shape priors. In: 3DPVT (2010)Google Scholar
  12. 12.
    Lothe, P., Bourgeois, S., Dekeyser, F., Royer, E., Dhome, M.: Towards geographical referencing of monocular SLAM reconstruction using 3D city models: application to real-time accurate vision-based localization. In: CVPR (2009)Google Scholar
  13. 13.
    Larnaout, D., Bourgeois, S., Gay-Bellile, V., Dhome, M.: Towards bundle adjustment with GIS constraints for online geo-localization of a vehicle in urban center. In: 3DIMPVT (2012)Google Scholar
  14. 14.
    Hideyuki, K., Takafumi, T., Tomokazu, S., Naokazu, Y.: Extrinsic camera parameter estimation using video images and GPS considering GPS positioning accuracy. In: ICPR (2010)Google Scholar
  15. 15.
    Michot, J., Bartoli, A., Gaspard, F.: Bi-objective bundle adjustment with application to multi-sensor SLAM. In: 3DPVT (2010)Google Scholar
  16. 16.
    Lhuillier, M.: Incremental fusion of structure-from-motion and GPS using constrained bundle adjustments. PAMI 34, 2489–2495 (2012)Google Scholar
  17. 17.
    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Indus. Appl. Math. 11, 431–441 (1963)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Recky, M., Wendel, A., Leberl, F.: Facade segmentation in a multi-view scenario. In: 3DIMPVT (2011)Google Scholar
  19. 19.
    Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: ECCV (2008)Google Scholar
  20. 20.
    Tamaazousti, M., Gay-Bellile, V., Naudet-Collette, S., Bourgeois, S., Dhome, M.: NonLinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment. In: CVPR (2011)Google Scholar
  21. 21.
    Laneurit, J., Chapuis, R., Chausse, F.: Accurate vehicle positioning on a numerical map. Int. J. Control Autom. Syst. 3, 15–31 (2005)zbMATHGoogle Scholar
  22. 22.
    Jo, K., Chu, K., Sunwoo, M.: GPS-bias correction for precise localization of autonomous vehicles. In: IV (2013)Google Scholar
  23. 23.
    Lee, D.D., Vernaza, P.: Robust GPS/INS-aided localization and mapping via GPS bias estimation. In: ISER (2006)Google Scholar
  24. 24.
    Arth, C., Wagner, D., Klopschitz, M., Irschara, A., Schmalstieg, D.: Wide area localization on mobile phones. In: ISMAR (2009)Google Scholar
  25. 25.
    Dong, Z., Zhang, G., Jia, J., Bao, H.: Keyframe-based real-time camera tracking. In: ICCV (2009)Google Scholar
  26. 26.
    Tong, G., Wu, Z., Weng, N., Hou, W.: An omni-directional vSLAM based on spherical camera model and 3D modeling. in: World Congress on Intelligent Control and Automation (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Dorra Larnaout
    • 1
    • 2
    Email author
  • Vincent Gay-Bellile
    • 1
  • Steve Bourgeois
    • 1
  • Michel Dhome
    • 2
  1. 1.CEA, LIST, LVICGif-Sur-YvetteFrance
  2. 2.Institut Pascal, UMR 6602 Université Blaise Pascal/CNRS/IFMAClermont-FerrandFrance

Personalised recommendations