Stereo vision-based vehicle localization in point cloud maps using multiswarm particle swarm optimization

  • V. JohnEmail author
  • Z. Liu
  • S. Mita
  • Y. Xu
Original Paper


We propose a vision-based localization algorithm with multiswarm particle swarm optimization for driving an autonomous vehicle. With stereo vision, the vehicle can be localized within a 3D point cloud map using the particle swarm optimization. For vehicle localization, the GPS (global positioning system)-based algorithms are often affected by the certain conditions resulting in intermittent missing signal. We address this issue in vehicle localization by using stereo vision in addition to the GPS information. The depth-based localization is formulated as an optimization-based tracking problem. Virtual depth images generated from the point cloud are matched with the online stereo depth images using the particle swarm optimization. The virtual depth images are generated from the point cloud using a series of coordinate transforms. We propose a novel computationally efficient tracker, i.e., a multiswarm particle swarm optimization-based algorithm. The tracker is initialized with GPS information and employs a Kalman filter in the bootstrapping phase. The Kalman filter stabilizes the GPS information in this phase and, subsequently, initializes the online tracker. The proposed localization algorithm is validated with acquired datasets from driving tests. A detailed comparative and parametric analysis is conducted in the experiments. The experimental results demonstrate the effectiveness and robustness of the proposed algorithm for vehicle localization, which advances the state of the art for autonomous driving.


Vehicle Localization Intelligent vehicles Autonomous Vehicles Particle Swarm Optimization Localization 



  1. 1.
    Aisan Technology Co Ltd: What’s MMS. (2013). Accessed 7 Nov 2018
  2. 2.
    Aulinas, J., Petillot, Y., Salvi, J., Lladó, X.: The SLAM problem: a survey. In: Proceedings of the 11th International Conference of the Catalan Association for Artificial Intelligence, IOS Press, Amsterdam, The Netherlands, pp. 363–371 (2008)Google Scholar
  3. 3.
    Bazeille, S., Battesti, E., Filliat, D.: Qualitative localization using vision and odometry for path following in topo-metric maps. In: Proceedings of the 5th European Conference on Mobile Robots, Örebro, Sweden, pp. 303–308 (2011)Google Scholar
  4. 4.
    Cappelle, C., Pomorski, D., Yang, Y.: Gps/ins data fusion for land vehicle localization. In: The Proceedings of the Multiconference on “Computational Engineering in Systems Applications”, pp. 21–27 (2006)Google Scholar
  5. 5.
    Colombo, O.: Ephemeris errors of GPS satellites. Bull. Godsique 60(1), 64–84 (1986)CrossRefGoogle Scholar
  6. 6.
    Cunha, J., Pedrosa, E., Cruz, C., Neves, A.J.R., Lau, N.: Using a depth camera for indoor robot localization and navigation. In: In Robotics Science and Systems (RSS) RGB-D Workshop, pp. 1–6 (2011)Google Scholar
  7. 7.
    Dailey, M.N., Parnichkun, M.: Simultaneous localization and mapping with stereo vision. In: Ninth International Conference on Control, Automation, Robotics and Vision, Singapore, pp. 1–6 (2006)Google Scholar
  8. 8.
    Deutscher, J., Blake, A., Reid, I.D.: Articulated body motion capture by annealed particle filtering. In: Conference on Computer Vision and Pattern Recognition, Hilton Head, USA, pp. 2126–2133 (2000)Google Scholar
  9. 9.
    El Jaafari, I., El Ansari, M., Koutti, L.: Fast edge-based stereo matching approach for road applications. Signal Image Video Process. 11(2), 267–274 (2017)CrossRefGoogle Scholar
  10. 10.
    Franconi, L., Jennison, C.: Comparison of a genetic algorithm and simulated annealing in an application to statistical image reconstruction. Stat. Comput. 7(3), 193–207 (1997)CrossRefGoogle Scholar
  11. 11.
    Grewal, M.S., Weill, L.R., Andrews, A.P.: Global Positioning Systems, Inertial Navigation, and Integration. Wiley, New York (2007)CrossRefGoogle Scholar
  12. 12.
    Jia, B., Feng, W., Zhu, M.: Obstacle detection in single images with deep neural networks. Signal Image Video Process. 10(6), 1033–1040 (2016)Google Scholar
  13. 13.
    John, V., Trucco, E., Ivekovic, S.: Markerless human articulated tracking using hierarchical particle swarm optimisation. Image Vis. Comput. 28(11), 1530–1547 (2010)CrossRefGoogle Scholar
  14. 14.
    John, V., Long, Q., Liu, Z., Mita, S.: Automatic calibration and registration of lidar and stereo camera without calibration objects. In: IEEE International Conference on Vehicular Electronics and Safety, ICVES 2015, Yokohama, Japan, November 5–7, 2015, pp. 231–237 (2015)Google Scholar
  15. 15.
    John, V., Long, Q., Xu, Y., Liu, Z., Mita, S.: Registration of GPS and stereo vision for point cloud localization in intelligent vehicles using particle swarm optimization. In: Advances in Swarm Intelligence—8th International Conference, ICSI, Fukuoka, Japan, Proceedings, Part I, pp. 209–217 (2017a)Google Scholar
  16. 16.
    John, V., Tsuchizawa, S., Liu, Z., Mita, S.: Fusion of thermal and visible cameras for the application of pedestrian detection. Signal Image Video Process. 11(3), 517–524 (2017b)CrossRefGoogle Scholar
  17. 17.
    John, V., Liu, Z., Mita, S., Guo, C., Kidono, K.: Real-time road surface and semantic lane estimation using deep features. Signal Image Video Process. 12(6), 1133–1140 (2018)CrossRefGoogle Scholar
  18. 18.
    Long, Q., Xie, Q., Mita, S., Tehrani, H., Ishimaru, K., Guo, C.: Real-time dense disparity estimation based on multi-path viterbi for intelligent vehicle applications. In: British Machine Vision Conference, BMVC 2014, Nottingham, UK, pp. 1–8 (2014)Google Scholar
  19. 19.
    Maier, D., Hornung, A., Bennewitz, M.: Real-time navigation in 3d environments based on depth camera data. In: 12th IEEE-RAS International Conference on Humanoid Robots, Osaka, Japan, pp. 692–697 (2012)Google Scholar
  20. 20.
    Mattern, N., Schubert, R., Wanielik, G.: High-accurate vehicle localization using digital maps and coherency images. In: IEEE Intelligent Vehicles Symposium, La Jolla, CA, USA, pp. 462–469 (2010)Google Scholar
  21. 21.
    Niknejad, H.T., Takeuchi, A., Mita, S., McAllester, D.: On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation. IEEE Trans. Intell. Transp. Syst. 13, 748–758 (2012)CrossRefGoogle Scholar
  22. 22.
    Nistér, D., Naroditsky, O., Bergen, J.R.: Visual odometry. In: 2004 Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 652–659 (2004)Google Scholar
  23. 23.
    Oh, S.M., Tariq, S., Walker, B.N., Dellaert, F.: Map-based priors for localization. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan, pp. 2179–2184 (2004)Google Scholar
  24. 24.
    Panzieri, S., Pascucci, F., Ulivi, G.: An outdoor navigation system using GPS and inertial platform. IEEE/ASME Trans. Mechatron. 7(2), 134–142 (2002)CrossRefGoogle Scholar
  25. 25.
    Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: International Conference on Evolutionary Computation, pp. 69–73 (1998)Google Scholar
  26. 26.
    Yoneda, K., Tehrani, H., Ogawa, T., Hukuyama, N., Mita, S.: Lidar scan feature for localization with highly precise 3-d map. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, June 8–11, 2014, pp. 1345–1350 (2014)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Toyota Technological InstituteNagoyaJapan
  2. 2.University of British ColumbiaVancouverCanada

Personalised recommendations