Advertisement

Machine Vision and Applications

, Volume 27, Issue 6, pp 943–962 | Cite as

Fast and automatic city-scale environment modelling using hard and/or weak constrained bundle adjustments

  • Dorra LarnaoutEmail author
  • Vincent Gay-Bellile
  • Steve Bourgeois
  • Michel Dhome
Original Paper

Abstract

To provide high-quality augmented reality service in a car navigation system, accurate 6 degrees of freedom (DoF) localization is required. To ensure such accuracy, most current vision-based solutions rely on an off-line large-scale modelling of the environment. Nevertheless, while existing solutions to model the environment require expensive equipments and/or a prohibitive computation time, we propose in this paper a complete framework that automatically builds an accurate large-scale database of landmarks using only a standard camera, a low-cost global positioning system (GPS) and a geographic information system (GIS). As illustrated in the experiments, only few minutes are required to model large-scale environments. The resulting databases can then be used by an on-line localization algorithm to ensure high-quality augmented reality experiences.

Keywords

Simultaneous localization and mapping Constrained bundle adjustment Global localization system Geographic information system Augmented reality 

References

  1. 1.
    Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR (2009)Google Scholar
  2. 2.
    Li, Y., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3d point clouds. In: ECCV (2012)Google Scholar
  3. 3.
    Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: ECCV (2010)Google Scholar
  4. 4.
    Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Real time localization and 3D reconstruction. In: CVPR (2006)Google Scholar
  5. 5.
    Taneja, A., Ballan, L., Pollefeys, M.: Registration of spherical panoramic images with cadastral 3D models. In: 3DIMPVT (2012)Google Scholar
  6. 6.
    Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S.M., Szeliski, R.: Building Rome in a day. Commun. ACM 54, 105–112 (2011)CrossRefGoogle Scholar
  7. 7.
    Frahm, J.M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Y. H. Jen, E. Dunn, B. Clipp, S. Lazebnik, M. Pollefeys, Building Rome on a cloudless day. In: ECCV (2010)Google Scholar
  8. 8.
    Klingner, B., Martin, D., Roseborough, J.: Street view motion-from-structure-from-motion. In: ICCV (2013)Google Scholar
  9. 9.
    Kaminsky, R.S., Snavely, N., Seitz, S.M., Szeliski, R.: Alignment of 3D point clouds to overhead images. In: CVPR Workshop (2009)Google Scholar
  10. 10.
    Strecha, C., Pylvänäinen, T., Fua, P.: Dynamic and scalable large scale image reconstruction. In: CVPR (2010)Google Scholar
  11. 11.
    Pylvänäinen, T., Roimela, K., Vedantham, R., Itäranta, J., Wang, R., Grzeszczuk, R.: Automatic alignment and multi-view segmentation of street view data using 3D shape priors. In: 3DPVT (2010)Google Scholar
  12. 12.
    Lothe, P., Bourgeois, S., Dekeyser, F., Royer, E., Dhome, M.: Towards geographical referencing of monocular SLAM reconstruction using 3D city models: application to real-time accurate vision-based localization. In: CVPR (2009)Google Scholar
  13. 13.
    Larnaout, D., Bourgeois, S., Gay-Bellile, V., Dhome, M.: Towards bundle adjustment with GIS constraints for online geo-localization of a vehicle in urban center. In: 3DIMPVT (2012)Google Scholar
  14. 14.
    Hideyuki, K., Takafumi, T., Tomokazu, S., Naokazu, Y.: Extrinsic camera parameter estimation using video images and GPS considering GPS positioning accuracy. In: ICPR (2010)Google Scholar
  15. 15.
    Michot, J., Bartoli, A., Gaspard, F.: Bi-objective bundle adjustment with application to multi-sensor SLAM. In: 3DPVT (2010)Google Scholar
  16. 16.
    Lhuillier, M.: Incremental fusion of structure-from-motion and GPS using constrained bundle adjustments. PAMI 34, 2489–2495 (2012)Google Scholar
  17. 17.
    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Indus. Appl. Math. 11, 431–441 (1963)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Recky, M., Wendel, A., Leberl, F.: Facade segmentation in a multi-view scenario. In: 3DIMPVT (2011)Google Scholar
  19. 19.
    Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: ECCV (2008)Google Scholar
  20. 20.
    Tamaazousti, M., Gay-Bellile, V., Naudet-Collette, S., Bourgeois, S., Dhome, M.: NonLinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment. In: CVPR (2011)Google Scholar
  21. 21.
    Laneurit, J., Chapuis, R., Chausse, F.: Accurate vehicle positioning on a numerical map. Int. J. Control Autom. Syst. 3, 15–31 (2005)zbMATHGoogle Scholar
  22. 22.
    Jo, K., Chu, K., Sunwoo, M.: GPS-bias correction for precise localization of autonomous vehicles. In: IV (2013)Google Scholar
  23. 23.
    Lee, D.D., Vernaza, P.: Robust GPS/INS-aided localization and mapping via GPS bias estimation. In: ISER (2006)Google Scholar
  24. 24.
    Arth, C., Wagner, D., Klopschitz, M., Irschara, A., Schmalstieg, D.: Wide area localization on mobile phones. In: ISMAR (2009)Google Scholar
  25. 25.
    Dong, Z., Zhang, G., Jia, J., Bao, H.: Keyframe-based real-time camera tracking. In: ICCV (2009)Google Scholar
  26. 26.
    Tong, G., Wu, Z., Weng, N., Hou, W.: An omni-directional vSLAM based on spherical camera model and 3D modeling. in: World Congress on Intelligent Control and Automation (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Dorra Larnaout
    • 1
    • 2
    Email author
  • Vincent Gay-Bellile
    • 1
  • Steve Bourgeois
    • 1
  • Michel Dhome
    • 2
  1. 1.CEA, LIST, LVICGif-Sur-YvetteFrance
  2. 2.Institut Pascal, UMR 6602 Université Blaise Pascal/CNRS/IFMAClermont-FerrandFrance

Personalised recommendations