Machine Vision and Applications

, Volume 25, Issue 3, pp 649–665 | Cite as

Exploiting temporal and spatial constraints in traffic sign detection from a moving vehicle

  • Siniša Šegvić
  • Karla Brkić
  • Zoran Kalafatić
  • Axel Pinz
Special Issue Paper
First Online:
Received:
Revised:
Accepted:

Abstract

This paper addresses detection, tracking and recognition of traffic signs in video. Previous research has shown that very good detection recalls can be obtained by state-of-the-art detection algorithms. Unfortunately, satisfactory precision and localization accuracy are more difficultly achieved. We follow the intuitive notion that it should be easier to accurately detect an object from an image sequence than from a single image. We propose a novel two-stage technique which achieves improved detection results by applying temporal and spatial constraints to the occurrences of traffic signs in video. The first stage produces well-aligned temporally consistent detection tracks by managing many competing track hypotheses at once. The second stage improves the precision by filtering the detection tracks by a learned discriminative model. The two stages have been evaluated in extensive experiments performed on videos acquired from a moving vehicle. The obtained experimental results clearly confirm the advantages of the proposed technique.

Keywords

Video analysis Object detection Object tracking Discriminative models Supervised learning Traffic signs 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arnoul, P., Viala M., Guerin J.P., Mergy, M.: Traffic signs localisation for highways inventory from a video camera on board a moving collection van. In: Proceedings of IV, pp. 141–146, Tokyo (1996)Google Scholar
  2. 2.
    Bahlmann, C., Zhu, Y., Ramesh, V., Pellkofer, M., Koehler, T.: A system for traffic sign detection, tracking, and recognition using color, shape, and motion information. In Proceedings of IV, pp. 255–260, Las Vegas (2005)Google Scholar
  3. 3.
    Baker S., Matthews I.: Lucas–Kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)Google Scholar
  4. 4.
    Baro, X., Vitria, J.: Fast traffic sign detection on greyscale images. Recent Advances in Artificial Intelligence Research and Development, pp. 131–138 (2005)Google Scholar
  5. 5.
    Baró X., Escalera S., Vitrià J., Pujol O., Radeva P.: Traffic sign recognition using evolutionary adaboost detection and forest-eco classification. IEEE Trans. ITS 10(1), 113–126 (2009)Google Scholar
  6. 6.
    Benesova, W., Lypetskyy, Y., Andreu, J.-Ph., Paletta, L., Jeitler, A., Hdl, E.: A mobile system for vision based road sign inventory. In: Proceedings of the 5th international symposium on mobile mapping technology, Padova (2007)Google Scholar
  7. 7.
    Bradski G.R., Kaehler A.: Learning OpenCV. O’Reilly Media, Inc, Sebastopol (2008)Google Scholar
  8. 8.
    Brkić, K., Pinz, A., Šegvić, S.: Traffic sign detection as a component of an automated traffic infrastructure inventory system. In: Proceedings of AAPR/ÖAGM, Stainz (2009)Google Scholar
  9. 9.
    Brkić, K., Šegvić, S., Zoran, K., Sikirić, I., Pinz, A.: Generative modeling of spatio-temporal traffic sign trajectories. In: Proceedings of UCVP, pp. 25–31, held in conjuction with CVPR2010, San Francisco (2010)Google Scholar
  10. 10.
    Cardoso, J.L., Stefan, C., Elvik, R., Srensen, M.: Road safety inspection—best practice guidelines and implementation steps. Technical report, deliverable D5 of the EU FP6 project RIPCORD, ISEREST (2007)Google Scholar
  11. 11.
    Casas J.R., Sitjes A.P., Folch P.P.: Mutual feedback scheme for face detection and tracking aimed at density estimation in demonstrations. Vis. Image Signal Process. IEE Proc. 152(3), 334–346 (2005)CrossRefGoogle Scholar
  12. 12.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of CVPR, pp. 886–893 (2005)Google Scholar
  13. 13.
    Davison A.J., Reid I.D., Molton N.D., Stasse O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. PAMI 26(6), 1052–1067 (2007)CrossRefGoogle Scholar
  14. 14.
    Duda R.O., Hart P.E., Stork D.G.: Pattern classification, 2nd edn. Wiley, New York (2001)MATHGoogle Scholar
  15. 15.
    de la Escalera A., Moreno L.E., Salichs M.A., Armingol J.M.: Road traffic sign detection and classification. IEEE Trans. Ind. Electron. 44(6), 848–859 (1997)CrossRefGoogle Scholar
  16. 16.
    Enzweiler M., Gavrila D.M.: Monocular pedestrian detection: survey and experiments. IEEE Trans. PAMI 31(12), 2179–2195 (2009)CrossRefGoogle Scholar
  17. 17.
    Ess A., Leibe B., Schindler K., Van Gool L.: Robust multi-person tracking from a mobile platform. IEEE Trans. PAMI 31(10), 1831–1846 (2009)CrossRefGoogle Scholar
  18. 18.
    Fang C.-Y., Chen S.-W., Fuh C.-S.: Road-sign detection and tracking. IEEE Trans. Veh. Technol. 52(5), 1329–1341 (2003)CrossRefGoogle Scholar
  19. 19.
    Fletcher L., Apostoloff N., Petersson L., Zelinsky A.: Vision in and out of vehicles. IEEE Intell. Syst. 18(3), 12–17 (2003)CrossRefGoogle Scholar
  20. 20.
    Gao X.W., Podladchikova L.N., Shaposhnikov D.G., Hong K., Shevtsova N.: Recognition of traffic signs based on their colour and shape features extracted using human vision models. J. Vis. Commun. Image Represent. 17(4), 675–685 (2006)CrossRefGoogle Scholar
  21. 21.
    Garcia-Garrido, M.A., Sotelo, M.A., Martin-Gorostiza, E.: Fast traffic sign detection and recognition under changing lighting conditions. In: Proceedings of ITSC, pp. 811–816, Toronto (2006)Google Scholar
  22. 22.
    Grabner, H., Beleznai, C., Bischof, H.: Improving adaboost detection rate by wobble and mean shift. In: Proceedings of CVWW, pp. 23–32, Zell an der Pram (2005)Google Scholar
  23. 23.
    Hoiem D., Efros A.A., Hebert M.: Putting objects in perspective. Int. J. Comput. Vis. 80(1), 3–15 (2008)CrossRefGoogle Scholar
  24. 24.
    Inland transport comitee (1968) Convention on road signs and signals. Economic comission for EuropeGoogle Scholar
  25. 25.
    Larsson, F., Felsberg, M.: Using fourier descriptors and spatial models for traffic sign recognition. In: Proceedings of SCIA, vol. 6688, pp. 238–249 (2011)Google Scholar
  26. 26.
    Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: Proceedings of DAGM, pp. 297–304, Magdeburg (2003)Google Scholar
  27. 27.
    Liu, W., Chen, X., Duan, B., Dong, H., Fu, P., Yuan, H., Zhao, H.: A system for road sign detection, recognition and tracking based on multi-cues hybrid. In: Proceedings of IV, pp. 562–567 (2009)Google Scholar
  28. 28.
    Loy, G., Barnes, N.M.: Fast shape-based road sign detection for a driver assistance system. In: Proceedings of IROS, pp. 70–75, Sendai (2004)Google Scholar
  29. 29.
    Madeira, S.R., Bastos, L.C., Sousa, A.M., Sobral, J.F., Santos, L.P.: Automatic traffic signs inventory using a mobile mapping system for GIS applications. In: International conference and exhibition on geographic information, Lisboa (2005)Google Scholar
  30. 30.
    Maldonado-Bascon, S., Lafuente-Arroyo, S., Siegmann, P., Gomez-Moreno, H., Acevedo-Rodriguez, F.J.: Traffic sign recognition system for inventory purposes. In: Proceedings of IV, pp. 590–595, Eindhoven (2008)Google Scholar
  31. 31.
    Marlin, B.M.: Missing data problems in machine learning. Doctoral dissertation, University of Toronto (2008)Google Scholar
  32. 32.
    Mccarthy, T., Maynooth, N., Mcelhinney, C., Cahalane, C., Kumar, P.: Initial results from european road safety inspection (eursi) mobile mapping project. In: Proceedings of ISPRS CRIMT, Newcastle (2010)Google Scholar
  33. 33.
    Nguwi Y.-Y., Kouzani A.Z.: Detection and classification of road signs in natural environments. Neural Comput. Appl. 17(3), 265–289 (2008)CrossRefGoogle Scholar
  34. 34.
    Parada-Loira, F., Landesa-Vázquez, I., Alba-Castro, J.L.: Fast real-time multiclass traffic sign detection based on novel shape and texture descriptors. In: Proceedings of ITSC, pp. 1–8, Madeira (2010)Google Scholar
  35. 35.
    Peng J., Bhanu B.: Learning to perceive objects for autonomous navigation. Auton. Robot. 6(2), 187–201 (1999)CrossRefGoogle Scholar
  36. 36.
    Pham, M.-T., Gao, Y., Hoang, V.-D.D., Cham, T.-J.: Fast polygonal integration and its application in extending haar-like features to improve object detection. In: Proceedings of CVPR, San Francisco (2010)Google Scholar
  37. 37.
    Piccioli G., De Micheli E., Parodi P., Campani M.: Robust method for road sign detection and recognition. Image Vis. Comput. 14(3), 209–223 (1996)CrossRefGoogle Scholar
  38. 38.
    Roth, P.M.: On-line conservative learning. Ph.D. thesis, Graz university of Technology, Institute for Computer Vision and Graphics (2008)Google Scholar
  39. 39.
    Ruta, A., Li, Y., Liu, X.: Detection, tracking and recognition of traffic signs from video input. In Proceedings of ITSC, pp. 55–60 (2008)Google Scholar
  40. 40.
    Ruta A., Li Y., Liu X.: Real-time traffic sign recognition from video by class-specific discriminative features. Pattern Recognit. 43(1), 416–430 (2010)CrossRefMATHGoogle Scholar
  41. 41.
    Ruta, A., Porikli, F., Watanabe, S., Li, Y.: In-vehicle camera traffic sign detection and recognition. Mach. Vis. Appl. (2009, accepted for publication)Google Scholar
  42. 42.
    Šegvić, S., Remazeilles, A., Chaumette, F.: Enhancing the point feature tracker by adaptive modelling of the feature support. In: Proceedings of ECCV, Springer LNCS, pp. 112–124, Graz (2006)Google Scholar
  43. 43.
    Šegvić S., Remazeilles A., Diosi A., Chaumette F.: A scalable mapping and localization framework for robust appearance-based navigation. Comput. Vis. Image Underst. 113(2), 172–187 (2009)CrossRefGoogle Scholar
  44. 44.
    Šegvić, S., Brkić, K., Kalafatić, Z., Stanisavljević, V., Ševrović, M., Budimir, D., Dadić, I.: A computer vision assisted geoinformation inventory for traffic infrastructure. In: Proceedings of ITSC, pp. 66–73, Madeira (2010)Google Scholar
  45. 45.
    Shi, J., Tomasi, C.: Good features to track. In: Proceedings of CVPR, pp. 593–600, Seattle (1994)Google Scholar
  46. 46.
    Timofte, R., Zimmermann, K., Van Gool, L.: Multi-view traffic sign detection, recognition, and 3d localisation. In: Proceedings of WACV, pp. 69–76, Snowbird (2009)Google Scholar
  47. 47.
    Tuzel, O., Porikli, F., Meer, P.: Learning on lie groups for invariant detection and tracking. In: Proceedings of CVPR, pp. 1–8, Anchorage (2008)Google Scholar
  48. 48.
    Viola P., Jones M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)Google Scholar
  49. 49.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann (2005)Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Siniša Šegvić
    • 1
  • Karla Brkić
    • 1
  • Zoran Kalafatić
    • 1
  • Axel Pinz
    • 2
  1. 1.Faculty of Electrical Engineering and ComputingUniversity of ZagrebZagrebCroatia
  2. 2.Institute of Electrical Measurement and Measurement Signal ProcessingGraz University of TechnologyGrazAustria

Personalised recommendations