Abstract
In this paper, we discuss theoretical foundations and a practical realization of a real-time traffic sign detection, tracking and recognition system operating on board of a vehicle. In the proposed framework, a generic detector refinement procedure based on mean shift clustering is introduced. This technique is shown to improve the detection accuracy and reduce the number of false positives for a broad class of object detectors for which a soft response’s confidence can be sensibly estimated. The track of an already established candidate is maintained over time using an instance-specific tracking function that encodes the relationship between a unique feature representation of the target object and the affine distortions it is subject to. We show that this function can be learned on-the-fly via regression from random transformations applied to the image of the object in known pose. Secondly, we demonstrate its capability of reconstructing the full-face view of a sign from substantial view angles. In the recognition stage, a concept of class similarity measure learned from image pairs is discussed and its realization using SimBoost, a novel version of AdaBoost algorithm, is analyzed. Suitability of the proposed method for solving multi-class traffic sign classification problems is shown experimentally for different feature representations of an image. Overall performance of our system is evaluated based on a prototype C++ implementation. Illustrative output generated by this demo application is provided as a supplementary material attached to this paper.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Piccioli G., De Micheli E., Parodi P., Campani M.: A robust method for road sign detection and recognition. Image Vis. Comput. 14(3), 209–223 (1996)
de la Escalera A., Moreno L.E., Salichs M.A., Armingol J.M.: Road traffic sign detection and classification. IEEE Trans. Indus. Electron. 44(6), 848–859 (1997)
Barnes, N., Loy, G., Shaw, D., Robles-Kelly, A.: Regular polygon detection. In: Proc. of the 10th IEEE International Conference on Computer Vision, vol.1, pp.778–785 (2005)
Ruta, A., Li, Y., Liu, X.: Towards real-time traffic sign recognition by class-specific discriminative features. In: Proc. of the 18th British Machine Vision Conference, vol. 1, pp. 399–408 (2007)
Fang C-Y., Chen S-W., Fuh C-S.: Road-sign detection and tracking. IEEE Trans. Veh. Technol. 52(5), 1329–1341 (2003)
Bahlmann, C., Zhu, Y., Ramesh, V., Pellkofer, M., Koehler, T.: A system for traffic sign detection, tracking and recognition using color, shape, and motion information. In: Proc. of the IEEE Intelligent Vehicles Symposium, pp. 255–260 (2005)
Freund Y., Schapire R.E.: A short introduction to boosting. J. Jpn. Soc. Artif. Intell. 14(5), 771–780 (1999)
Viola P., Jones M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Miura, J., Kanda T., Shirai Y.: An active vision system for real-time traffic sign recognition. In: Proc. of the IEEE Conference on Intelligent Transportation Systems, pp. 52–57 (2000)
Kang, D.S., Griswold, N.C., Kehtarnavaz, N.: An invariant traffic sign recognition system based on sequential color processing and geometrical transformation. In: Proc. of the IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 88–93 (1994)
Aoyagi, Y., Asakura, T.: A study on traffic sign recognition in scene image using genetic algorithms and neural networks. In: Proc. of the 22nd IEEE International Conference on Industrial Electronics, Control, and Instrumentation, vol.3, pp. 1838–1843 (1996)
Douville P.: Real-time classification of traffic signs. Real Time Imaging (9) 6(3), 185–193 (2000)
de la Escalera A., Armingol J.M., Mata M.: Traffic sign recognition and analysis for intelligent vehicles. Image Vis. Comput. 21(3), 247–258 (2003)
Nguwi Y.-Y., Kouzani A.Z.: Detection and classification of road signs in natural environments. Neural Comput. Appl. 17(3), 265–289 (2008)
Paclík P., Novovicova J., Pudil P., Somol P.: Road sign classification using the Laplace Kernel Classifier. Pattern Recognit. Lett. 21(13–14), 1165–1173 (2000)
Gao X.W., Podladchikova L., Shaposhnikov D., Hong K., Shevtsova N.: Recognition of traffic signs based on their colour and shape features extracted using human vision models. J. Vis. Commun. Image Represent. 17(4), 675–685 (2006)
Paclík P., Novovicová J., Duin R.P.W.: Building road-sign classifiers using a trainable similarity measure. IEEE Trans. Intell. Transport. Syst. 7(3), 309–321 (2006)
Duda R.O., Hart P.E.: Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM 15(1), 11–15 (1972)
Comaniciu D., Meer P.: Mean shift: a robust approach towards feature space analysis. IEEE Trans. Pattern Anal. Machine Intell. 24(5), 603–619 (2002)
Bayro-Corrochano E., Ortegón-Aguilar J.: Lie algebra approach for tracking and 3D motion estimation using monocular vision. Image Vis. Comput. 25(6), 907–921 (2007)
Tuzel, O., Porikli, F., Meer, P.: Learning on Lie groups for invariant detection and tracking. In: Proc. of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)
Rossmann W.: Lie Groups: An Introduction Through Linear Groups. Oxford University Press, New York (2002)
Schapire R.E., Singer Y.: Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37(3), 297–336 (1999)
Open Graphics Library. http://www.opengl.org/
Kalman R.E.: New approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(D), 35–45 (1960)
Open Computer Vision Library, http://sourceforge.net/projects/opencvlibrary/
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proc. of the 2005 IEEE International Conference on Computer Vision and Pattern Recognition vol. 1, pp. 886–893 (2005)
Tuzel, O., Porikli, F., Meer, P.: Region covariance: A fast descriptor for detection and classification. In Proc. of the 9th European Conference on Computer Vision, pp. 589–600 (2006)
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
The Below are the Electronic Supplementary Materials.
Rights and permissions
About this article
Cite this article
Ruta, A., Porikli, F., Watanabe, S. et al. In-vehicle camera traffic sign detection and recognition. Machine Vision and Applications 22, 359–375 (2011). https://doi.org/10.1007/s00138-009-0231-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-009-0231-x