Skip to main content
SpringerLink
Log in

Multi-view traffic sign detection, recognition, and 3D localisation

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Several applications require information about street furniture. Part of the task is to survey all traffic signs. This has to be done for millions of km of road, and the exercise needs to be repeated every so often. We used a van with eight roof-mounted cameras to drive through the streets and took images every meter. The paper proposes a pipeline for the efficient detection and recognition of traffic signs from such images. The task is challenging, as illumination conditions change regularly, occlusions are frequent, sign positions and orientations vary substantially, and the actual signs are far less similar among equal types than one might expect. We combine 2D and 3D techniques to improve results beyond the state-of-the-art, which is still very much preoccupied with single view analysis. For the initial detection in single frames, we use a set of colour- and shape-based criteria. They yield a set of candidate sign patterns. The selection of such candidates allows for a significant speed up over a sliding window approach while keeping similar performance. A speedup is also achieved through a proposed efficient bounded evaluation of AdaBoost detectors. The 2D detections in multiple views are subsequently combined to generate 3D hypotheses. A Minimum Description Length formulation yields the set of 3D traffic signs that best explains the 2D detections. The paper comes with a publicly available database, with more than 13,000 traffic signs annotations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bahlmann, C., Zhu, Y., Ramesh, V., Pellkofer, M., Koehler, T.: A system for traffic sign detection, tracking, and recognition using color, shape, and motion information. In: IEEE Intelligent Vehicles Symposium (2005)

  2. Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the 6th ACM international conference on Image and video retrieval, pp. 401–408. ACM Press, New York, NY, USA (2007)

  3. Broggi, A., Cerri, P., Medici, P., Porta, P., Ghisio, G.: Real time road signs recognition. In: Intelligent Vehicles Symposium, 2007 IEEE, pp. 981–986 (2007)

  4. Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: European Conference on Computer Vision (2008)

  5. Doubek, P., Perdoch, M., Matas, J., Sochman, J.: Mobile mapping of vertical traffic infrastructure. In: Proceedings of the 13th Computer Vision Winter Workshop, pp. 115–122. Slovenian Pattern Recognition Society, Ljubljana, Slovenia (2008)

  6. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2009 (VOC2009) Results. http://www.pascal-network.org/challenges/VOC/voc2009/workshop/index.html

  7. Felzenszwalb, P.F., Girshick, R.B., McAllester, D.: Cascade object detection with deformable part models. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)

  8. Felzenszwalb P.F., Girshick R.B., McAllester D., Ramanan D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  9. Fleuret, F., Berclaz, J., Fua, P.: Multicamera people tracking with a probabilistic occupancy map. IEEE Trans. Pattern Anal. Mach. Intell.30, (2), 267–282 (2008). http://ieeecomputersociety.org. doi:10.1109/TPAMI.2007.1174

  10. Golovinskiy, A., Kim, V.G., Funkhouser, T.: Shape-based recognition of 3d point clouds in urban environments. In: Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto, Japan (2009)

  11. Hoiem, D., Efros, A.A., Hebert, M.: Putting objects in perspective. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2137–2144 (2006)

  12. Miura, J., Kanda, T., Shirai, Y.: An active vision system for real-time traffic sign recognition. In: IEEE Conf. on Intelligent Transportation Systems (ITS), pp. 52–57 (2000)

  13. Kingston, T., Gikas, V., Laflamme, C., Larouche, C.: An integrated mobile mapping system for data acquisition and automated asset extraction. In: Proceedings of the 5th Mobile Mapping Technologies Symposium. Padua, Italy (2007)

  14. Lafuente, S., Gil, P., Maldonado, R., López, F., Maldonado, S.: Traffic sign shape classification evaluation i: Svm using distance to borders. In: IEEE Intelligent Vehicles Symposium, pp. 654–658 (2005)

  15. Leibe B., Schindler K., Cornelis N., Van Gool L.: Coupled object detection and tracking from static cameras and moving vehicles. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1683–1698 (2008)

    Article  Google Scholar 

  16. Leonardis A., Gupta A., Bajcsy R.: Segmentation of range images as the search for geometric parametric models. Int. J. Comput. Vis. 14(3), 253–277 (1995)

    Google Scholar 

  17. Maldonado S., Lafuente S., Gil P., Gómez H., López F.: Road-sign detection and recognition based on support vector machines. IEEE Trans. Intell. Transp. Syst. 8(2), 264–278 (2007)

    Article  Google Scholar 

  18. Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: British Machine Vision Conference, vol. 1, pp. 384–393 (2002)

  19. Moutarde, F., Bargeton, A., Herbin, A., Chanussot, L.: Robust on-vehicle real-time visual detection of american and european speed limit signs, with a modular traffic signs recognition system. In: IEEE Intelligent Vehicles Symposium, pp. 1122–1126 (2007)

  20. Munoz, D., Vandapel, N., Hebert, M.: Directional associative markov network for 3-d point cloud classification. In: 3D Data Processing Visualization and Transmission (2008)

  21. Nunn, C., Kummert, A., Muller-Schneiders, S.: A novel region of interest selection approach for traffic sign recognition based on 3d modelling. In: IEEE Intelligent Vehicles Symposium, pp. 654–658 (2008)

  22. Pettersson, N., Petersson, L., Andersson, L.: The histogram feature—a resource-efficient weak classifier. In: IEEE Intelligent Vehicles Symposium, pp. 678–683 (2008)

  23. Prisacariu, V.A., Timofte, R., Zimmermann, K., Reid, I., Van Gool, L.J.: Integrating object detection with 3d tracking towards a better driver assistance system. In: 20th International Conference on Pattern Recognition, ICPR 2010, Istanbul, Turkey, pp. 3344–3347 (2010)

  24. Ruta, A., Li, Y., Liu, X.: Towards real-time traffic sign recognition by class-specific discriminative features. In: British Machine Vision Conference (2007)

  25. Ruta A., Porikli F., Watanabe S., Li Y.: In-vehicle camera traffic sign detection and recognition. Mach. Vis. Appl. 22(2), 359–375 (2011)

    Article  Google Scholar 

  26. Schapire R.E., Freund Y., Bartlett P., Lee W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Stat. 26(5), 1651–1686 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  27. Timofte R., Prisacariu V.A., Van Gool L.J., Reid I.: Chapter 3.5: Combining traffic sign detection with 3d tracking towards better driver assistance. In: Chen, C.H. (ed) Emerging Topics in Computer Vision and its Applications., World Scientific Publishing, Cleveland (2011)

    Google Scholar 

  28. Timofte, R., Van Gool, L.: Multi-view manhole detection, recognition, and 3d localisation. In: 1st IEEE/ISPRS Workshop on Computer Vision for Remote Sensing of the Environment (CVRS) in conjunction with the 13th International Conference on Computer Vision (ICCV) (2011)

  29. Timofte, R., Zimmermann, K., Van Gool, L.: Multi-view traffic sign detection, recognition, and 3d localisation. In: Proceedings of the IEEE Computer Society’s Workshop on Applications of Computer Vision (WACV). IEEE Computer Society, Snowbird, Utah, USA (2009)

  30. Viola, P., Jones, M.: Robust real-time face detection. In: IEEE International Conference on Computer Vision, vol. 2, pp. 747–757 (2001)

  31. Wojek, C., Schiele, B.: A dynamic conditional random field model for joint labeling of object and scene classes. In: European Conference on Computer Vision, pp 733–747 (2008)

Download references

Author information

Author notes

  1. Karel Zimmermann

    Present address: CMP, Czech Technical University in Prague, Prague, Czech Republic

Authors and Affiliations

  1. ESAT-PSI/IBBT, Katholieke Universiteit Leuven, Leuven, Belgium

    Radu Timofte & Luc Van Gool

Authors
  1. Radu Timofte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karel Zimmermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luc Van Gool
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radu Timofte.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Timofte, R., Zimmermann, K. & Van Gool, L. Multi-view traffic sign detection, recognition, and 3D localisation. Machine Vision and Applications 25, 633–647 (2014). https://doi.org/10.1007/s00138-011-0391-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-011-0391-3

Keywords

Search

Navigation