Machine Vision and Applications

, Volume 25, Issue 7, pp 1893–1911 | Cite as

Exploiting street-level panoramic images for large-scale automated surveying of traffic signs

  • Lykele HazelhoffEmail author
  • Ivo M. Creusen
  • Peter H. N. de With
Original Paper


Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.


Object detection Object classification  Traffic sign recognition Remote sensing 


  1. 1.
    Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SODA ’07, pp. 1027–1035. Society for Industrial and Applied Mathematics, Philadelphia (2007)Google Scholar
  2. 2.
    Bonaci, I., Kusalic, I., Kovacek, I., Kalafatic, Z., Segvic, S.: Addressing false alarms and localization inaccuracy in traffic sign detection and recognition. In: Proceeding of the 16th Computer Vision Winter Workshop, pp. 1–8 (2011)Google Scholar
  3. 3.
    Burghouts, G.J., Geusebroek, J.M.: Performance evaluation of local colour invariants. Comput. Vis. Image Underst. 113, 48–62 (2009)CrossRefGoogle Scholar
  4. 4.
    Comanicu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)CrossRefGoogle Scholar
  5. 5.
    Creusen, I., Hazelhoff, L., de With, P.: Color transformation for improved traffic sign detection. In: Proceeding of the 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 461–464 (2012). doi: 10.1109/ICIP.2012.6466896
  6. 6.
    Creusen, I., Wijnhoven, R.G.J., Herbschleb, E., de With, P.: Color exploitation in hog-based traffic sign detection. In: Proceeding of the 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 2669–2672 (2010). doi: 10.1109/ICIP.2010.5651637
  7. 7.
    Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Proceeding of the European Conference on Computer Vision (ECCV) (2004)Google Scholar
  8. 8.
    Dalal, N., Triggs, B.: Histogram of oriented gradients for human detection. In: Proceeding of the IEEE Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 886–893 (2005)Google Scholar
  9. 9.
    Hazelhoff, L., Creusen, I., de With, P.: Robust classification of traffic signs using multi-view cues. In: Proceeding of the 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 457–460 (2012). doi: 10.1109/ICIP.2012.6466895
  10. 10.
    Hazelhoff, L., Creusen, I.M., De With, P.H.N.: Robust detection, classification and positioning of traffic signs from street-level panoramic images for inventory purposes. In: Proceeding of the Workshop on Applications of Computer Vision (WACV), pp. 313–320 (2012)Google Scholar
  11. 11.
    Hazelhoff, L., Creusen, I.M., van de Wouw, D.W.J.M., de With, P.H.N.: Large-scale classification of traffic signs under real-world conditions. In: Proceeding of the SPIE 8304B–34 (2012)Google Scholar
  12. 12.
    Houben, S., Stallkamp, J., Salmen, J., Schlipsing, M., Igel, C.: Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark. In: Proceeding of the International Joint Conference on Neural Networks, p. 1288 (2013)Google Scholar
  13. 13.
    Jurie, F., Triggs, B.: Creating efficient codebooks for visual recognition. In: ICCV, pp. 604–610 (2005)Google Scholar
  14. 14.
    Lafuente-Arroyo, S., Maldonado-Bascon, S., Gil-Jimenez, P., Acevedo-Rodriguez, J., Lopez-Sastre, R.: A tracking system for automated inventory of road signs. In: Proceeding of the 2007 IEEE Intelligent Vehicles Symposium, pp. 166–171 (2007). doi: 10.1109/IVS.2007.4290109
  15. 15.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2) (2004)Google Scholar
  16. 16.
    Maldonado-Bascon, S., Lafuente-Arroyo, S., Gil-Jimenez, P., Gomez-Moreno, H., Lopez-Ferreras, F.: Road-sign detection and recognition based on support vector machines. IEEE Trans. Intell. Transp. Syst. 8(2), 264–278 (2007). doi: 10.1109/TITS.2007.895311
  17. 17.
    Mathias, M., Timofte, R., Benenson, R., Gool, L.J.V.: Traffic sign recognition—how far are we from the solution? In: IJCNN, pp. 1–8. IEEE, New York (2013)Google Scholar
  18. 18.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005). doi: 10.1109/TPAMI.2005.188 CrossRefGoogle Scholar
  19. 19.
    Mogelmose, A., Trivedi, M., Moeslund, T.: Vision-based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans. Intell. Transp. Syst. 13(4), 1484–1497 (2012). doi: 10.1109/TITS.2012.2209421 CrossRefGoogle Scholar
  20. 20.
    Overett, G., Petersson, L.: Large scale sign detection using hog feature variants. In: Proceeding of the 2011 IEEE Intelligent Vehicles Symposium (IV), pp. 326–331 (2011). doi: 10.1109/IVS.2011.5940549
  21. 21.
    Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Netw. 0 (2012). doi: 10.1016/j.neunet.2012.02.016
  22. 22.
    Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the condor experience. Concurr. Pract. Exp. 17(2–4), 323–356 (2005)CrossRefGoogle Scholar
  23. 23.
    Timofte, R., Gool, L.V.: Sparse representation based projections. In: Proceedings of the British Machine Vision Conference, pp. 61.1–61.12. BMVA Press, USA (2011).
  24. 24.
    Timofte, R., Zimmermann, K., Van Gool, L.: Multi-view traffic sign detection, recognition, and 3d localisation. In: Proceeding of the 2009 Workshop on Applications of Computer Vision (WACV), pp. 1–8 (2009). doi: 10.1109/WACV.2009.5403121
  25. 25.
    Timofte, R., Zimmermann, K., Van Gool, L.: Multi-view traffic sign detection, recognition, and 3d localisation. Mach. Vis. Appl. (2011). doi: 10.1007/s00138-011-0391-3

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Lykele Hazelhoff
    • 1
    Email author
  • Ivo M. Creusen
    • 1
  • Peter H. N. de With
    • 2
  1. 1.CycloMedia Technology B.V.ZaltbommelThe Netherlands
  2. 2.Eindhoven University of TechnologyEindhovenThe Netherlands

Personalised recommendations