Concept Study for Vehicle Self-Localization Using Neural Networks for Detection of Pole-Like Landmarks

  • Achim Kampker
  • Jonas HatzenbuehlerEmail author
  • Lars Klein
  • Mohsen Sefati
  • Kai D. Kreiskoether
  • Denny Gert
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 867)


This paper discusses and showcases a software framework for the self-localization of autonomous vehicles in an urban environment. The general concept of this framework is based on the semantic detection and observation of objects in the surrounding environment. For the object detection three different perception approaches are compared; LiDAR based, stereo camera based and mono camera based using a neural net. The investigated objects all share the same geometrical shape; they are vertical with a high aspect ratio. To compute the pose of the vehicle an Adaptive Monte-Carlo Algorithm has been implemented. Hence it is necessary to create a high-precision digital map this is done with a dense map, the detected objects and the LiDAR point cloud. Comparison with an earlier paper have shown that this approach keeps the global positioning accuracy around 0.50 m and leads to more robust results in highly dynamic scenarios where a small amount of objects can be detected.


Object detection Mapping Localization Neural networks 


  1. 1.
    Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Trans. Sig. Process. 50(2), 174–188 (2002)CrossRefGoogle Scholar
  2. 2.
    Bais, A., Sablatnig, R., Gu, J.: Single landmark based self-localization of mobile robots. In: The 3rd Canadian Conference on Computer and Robot Vision, p. 67. IEEE, Piscataway (2006)Google Scholar
  3. 3.
    Balali, V., Golparvar-Fard, M.: Recognition and 3D localization of traffic signs via image-based point cloud models. In: Ponticelli, S., O’Brien, W.J. (eds.) Computing in Civil Engineering 2015, pp. 206–214. American Society of Civil Engineers, Reston (2015)Google Scholar
  4. 4.
    Bappy, J.H., Roy-Chowdhury, A.K.: CNN based region proposals for efficient object detection. In: 2016 IEEE International Conference on Image Processing, pp. 3658–3662. IEEE, Piscataway (2016)Google Scholar
  5. 5.
    Bazin, J., Laffont, P., Kweon, I., Demonceaux, C., Vasseur, P.: An original approach for automatic plane extraction by omnidirectional vision. In: The IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 752–758. IEEE, Piscataway (2010)Google Scholar
  6. 6.
    Bernay-Angeletti, C., Chabot, F., Aynaud, C., Aufrere, R., Chapuis, R.: A top-down perception approach for vehicle pose estimation. In: 2015 IEEE International Conference on Robotics and Biomimetics, pp. 2240–2245. IEEE, Piscataway (2015)Google Scholar
  7. 7.
    Biber, P., Strasser, W.: The normal distributions transform: a new approach to laser scan matching. In: 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2743–2748. IEEE, Piscataway (2003)Google Scholar
  8. 8.
    Bora, D.J., Gupta, A.K., Khan, F.A.: Comparing the performance of L*A*B* and HSV color spaces with respect to color image segmentation, 04 June 2015.
  9. 9.
    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding, 07 April 2016.
  10. 10.
    Dailey, M.N., Parnichkun, M.: Landmark-based simultaneous localization and mapping with stereo vision (2005)Google Scholar
  11. 11.
    Denzler, J., Notni, G., Süße, H.: Pattern Recognition, vol. 5748. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Everingham, M., van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  13. 13.
    Feng, C., Taguchi, Y., Kamat, V.R.: Fast plane extraction in organized point clouds using agglomerative hierarchical clustering. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 6218–6225. IEEE, Piscataway (2014)Google Scholar
  14. 14.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the kitti dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)CrossRefGoogle Scholar
  15. 15.
    Girshick, R.: Fast R-CNN (2015).
  16. 16.
    Green, W.R., Grobler, H.: Normal distribution transform graph-based point cloud segmentation. In: Proceedings of the 2015 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech), pp. 54–59. IEEE, Piscataway (2015)Google Scholar
  17. 17.
    He, Z., Wang, Y., Yu, H.: Feature-to-feature based laser scan matching in polar coordinates with application to pallet recognition. Procedia Eng. 15, 4800–4804 (2011)CrossRefGoogle Scholar
  18. 18.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Patt. Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  19. 19.
    Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., Murphy, K.: Speed/accuracy trade-offs for modern convolutional object detectors (2016).
  20. 20.
    Jang, C., Kim, Y.K.: A feasibility study of vehicle pose estimation using road sign information. In: ICCAS 2016, pp. 397–401. IEEE, Piscataway (2016)Google Scholar
  21. 21.
    Jayatilleke, L., Zhang, N.: Landmark-based localization for unmanned aerial vehicles. In: 2013 IEEE International Systems Conference (SysCon 2013), pp. 448–451. IEEE, Piscataway (2013)Google Scholar
  22. 22.
    Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: single shot multibox detector (2015).
  23. 23.
    Macukow, B.: Neural networks - state of art, brief history, basic models and architecture. In: Saeed, K., Homenda, W. (eds.) Computer Information Systems and Industrial Management. Lecture Notes in Computer Science, pp. 3–14. Springer, Cham (2016)CrossRefGoogle Scholar
  24. 24.
    Pandey, G., McBride, J.R., Eustice, R.M.: Ford campus vision and lidar data set. Int. J. Robot. Res. 30(13), 1543–1552 (2011)CrossRefGoogle Scholar
  25. 25.
    Rademakers, E., de Bakker, P., Tiberius, C., Janssen, K., Kleihorst, R., Ghouti, N.E.: Obtaining real-time sub-meter accuracy using a low cost GNSS device. In: 2016 European Navigation Conference (ENC), pp. 1–8. IEEE, Piscataway (2016)Google Scholar
  26. 26.
    Rapp, M., Barjenbruch, M., Hahn, M., Dickmann, J., Dietmayer, K.: Clustering improved grid map registration using the normal distribution transform. In: 2015 IEEE Intelligent Vehicles Symposium (IV), pp. 249–254. IEEE, Piscataway (2015)Google Scholar
  27. 27.
    Reddy, B.S., Chatterji, B.N.: An FFT-based technique for translation, rotation, and scale-invariant image registration. IEEE Trans. Image Process. 5(8), 1266–1271 (1996)CrossRefGoogle Scholar
  28. 28.
    Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger (2016).
  29. 29.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 91–99. Curran Associates, Inc (2015).
  30. 30.
    Rohde, J., Jatzkowski, I., Mielenz, H., Zöllner, J.M.: Vehicle pose estimation in cluttered urban environments using multilayer adaptive monte carlo localization. In: 2016 19th International Conference on Information Fusion (FUSION), pp. 1774–1779 (2016)Google Scholar
  31. 31.
    Rossmann, J., Sondermann, B., Emde, M.: Virtual testbeds for planetary exploration: the self-localization aspect. 11th Symposium on Advanced Space Technologies in Robotics and Automation. ASTRA, pp. 1–8. ESA/ESTEC, Noordwijk (2011)Google Scholar
  32. 32.
    Schindler, A.: Vehicle self-localization with high-precision digital maps. In: 2013 IEEE Intelligent Vehicles Symposium workshops (IV workshops), pp. 134–139. IEEE, Piscataway (2013)Google Scholar
  33. 33.
    Sefati, M., Daum, M., Sondermann, B., Kreiskother, K.D., Kampker, A.: Improving vehicle localization using semantic and pole-like landmarks. In: 28th IEEE Intelligent Vehicles Symposium, pp. 13–19. IEEE, Piscataway (2017)Google Scholar
  34. 34.
    Shu, L., Xu, H., Huang, M.: High-speed and accurate laser scan matching using classified features. In: Ben-Tzvi, P. (ed.) 2013 IEEE International Symposium on Robotic and Sensors Environments (ROSE), pp. 61–66. IEEE, Piscataway (2013)Google Scholar
  35. 35.
    Sindagi, V.A., Patel, V.M.: A survey of recent advances in CNN-based single image crowd counting and density estimation. Patt. Recogn. Lett. 107, 3–6 (2018). Scholar
  36. 36.
    Spangenberg, R., Goehring, D., Rojas, R.: Pole-based localization for autonomous vehicles in urban scenarios. In: IROS 2016, pp. 2161–2166. IEEE, Piscataway (2016)Google Scholar
  37. 37.
    Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. Intelligent Robotics and Autonomous Agents. MIT Press, Cambridge (2010). Mass. [u.a.], [nachdr.] ednzbMATHGoogle Scholar
  38. 38.
    Xie, P., Petovello, M.G.: Measuring GNSS multipath distributions in urban canyon environments. IEEE Trans. Instrum. Measur. 64(2), 366–377 (2015)CrossRefGoogle Scholar
  39. 39.
    Zhang, H., Zhang, L., Dai, J.: Landmark-based localization for indoor mobile robots with stereo vision. In: 2012 Second International Conference on Intelligent System Design and Engineering Application (ISDEA), pp. 700–702. IEEE, Piscataway (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Achim Kampker
    • 1
  • Jonas Hatzenbuehler
    • 1
    Email author
  • Lars Klein
    • 1
  • Mohsen Sefati
    • 1
  • Kai D. Kreiskoether
    • 1
  • Denny Gert
    • 1
  1. 1.PEM RWTH AachenAachenGermany

Personalised recommendations