Semi-automatic Training of a Vehicle Make and Model Recognition System

  • M. H. Zwemer
  • G. M. Y. E. Brouwers
  • R. G. J. Wijnhoven
  • P. H. N. de With
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10485)


We propose a system for vehicle Make and Model Recognition (MMR) that automatically detects and classifies the make and model from a live camera mounted above the highway. Our system consists of a vehicle detection and MMR classification component. The vehicle detector is based on HOG features and can locate \(98\%\) of the vehicles with minimum false detections. We use a Convolutional Neural Network (CNN) for MMR classification on the vehicle locations. We propose a semi-automatic data-selection approach for the vehicle detector and the MMR classifier, by using an Automatic Number Plate Recognition engine for annotating new images, requiring minimal human annotation effort. In our results we show that our MMR classification has a top-1 accuracy of \(98\%\) for 500 vehicle models, where more than 500 training samples per model are desired to obtain accurate classification.


  1. 1.
    Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. CoRR abs/1605.07678 (2016).
  2. 2.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893, June 2005Google Scholar
  3. 3.
    Dehghan, A., Masood, S.Z., Shu, G., Ortiz, E.G.: View independent vehicle make, model and color recognition using convolutional neural network. CoRR abs/1702.01721 (2017).
  4. 4.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  5. 5.
    LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRefGoogle Scholar
  6. 6.
    Petrovic, V.S., Cootes, T.F.: Analysis of features for rigid structure vehicle type recognition. In: BMVC, vol. 2, pp. 587–596 (2004)Google Scholar
  7. 7.
    Prokaj, J., Medioni, G.: 3-D model based vehicle recognition. In: Workshop on Applications of Computer Vision (WACV), pp. 1–7, December 2009Google Scholar
  8. 8.
    Ren, Y., Lan, S.: Vehicle make and model recognition based on convolutional neural networks. In: Proceedings of the IEEE ICSESS, pp. 692–695, August 2016Google Scholar
  9. 9.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks. CoRR abs/1312.6229 (2013).
  11. 11.
    Shi, J., Tomasi, C.: Good features to track. In: 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600, June 1994Google Scholar
  12. 12.
    Siddiqui, A.J., Mammeri, A., Boukerche, A.: Real-time vehicle make and model recognition based on a bag of surf features. IEEE Trans. Intell. Transp. Syst. 17(11), 3205–3219 (2016)CrossRefGoogle Scholar
  13. 13.
    Sochor, J., Herout, A., Havel, J.: BoxCars: 3D boxes as CNN input for improved fine-grained vehicle recognition. In: IEEE Conference on CVPR, June 2016Google Scholar
  14. 14.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015Google Scholar
  15. 15.
    Wijnhoven, R.G.J., de With, P.H.N.: Fast training of object detection using stochastic gradient descent. In: Proceedings of the IEEE ICPR, pp. 424–427. IEEE Computer Society, Washington, D.C. (2010)Google Scholar
  16. 16.
    Wijnhoven, R.G.J., de With, P.H.N.: Unsupervised sub-categorization for object detection: finding cars from a driving vehicle. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2077–2083, November 2011Google Scholar
  17. 17.
    Yang, L., Luo, P., Change Loy, C., Tang, X.: A Large-Scale Car Dataset for Fine-Grained Categorization and Verification. ArXiv e-prints, June 2015Google Scholar
  18. 18.
    Zhou, Y., Liu, L., Shao, L., Mellor, M.: DAVE: a unified framework for fast vehicle detection and annotation. CoRR abs/1607.04564 (2016).

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • M. H. Zwemer
    • 1
    • 2
  • G. M. Y. E. Brouwers
    • 2
  • R. G. J. Wijnhoven
    • 2
  • P. H. N. de With
    • 1
  1. 1.Eindhoven University of TechnologyEindhovenThe Netherlands
  2. 2.ViNotionEindhovenThe Netherlands

Personalised recommendations