3D Model-Driven Vehicle Matching and Recognition

  • Tingbo Hou
  • Sen Wang
  • Hong Qin
Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1)


Matching vehicles subject to both large pose transformations and extreme illumination variations remains a technically challenging problem in computer vision. In this chapter, we first investigate the state-of-the-art studies on vehicle matching, inverse rendering by which illumination can be factorized from the light reflectance field, and applications of the near-IR illumination in computer vision. Then a 3D model-driven framework is developed, towards matching and recognizing vehicles with varying pose and (visible or near-IR) illumination conditions. We adopt a compact set of 3D models to represent basic types of vehicle. The pose transformation is estimated by using approximated vehicle models that can effectively match objects under large viewpoint changes and partial occlusions. With the estimation of surface reflectance property, illumination conditions are approximated by a low-dimensional linear subspace using spherical harmonics representation. By estimated pose and illumination conditions, we can re-render vehicles in the reference image to generate the relit image with the same pose and illumination conditions as the target image. Finally, we compare the relit image and the re-rendered target image to match vehicles in the original reference image and target image. Furthermore, no training is needed in our framework and re-rendered vehicle images in any other viewpoints and illumination conditions can be obtained from just one single input image. In our experiments, both synthetic data and real data are used. Experimental results demonstrate the robustness and efficacy of our framework, with a potential to generalize our current method from vehicles to handle other types of objects.


Vehicle matching 3D model-driven method Inverse rendering Spherical harmonics Near-IR illumination 



Hong Qin and Tingbo Hou (Stony Brook University)’s research reported in this chapter is partially supported by NSF Grants: IIS-0949467, IIS-0710819, and IIS-0830183.


  1. 1.
    Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. TPAMI 25(2), 218–233 (2003)Google Scholar
  2. 2.
    Biswas, S., Agrawal, G., Chellappa, R.: Robust estimation of albedo for illumination-invariant matching and shape recovery. In: ICCV (2007).Google Scholar
  3. 3.
    Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: SIGGRAPH, pp. 145–156 (2000)Google Scholar
  4. 4.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: CVPR, pp. 264–271 (2006)Google Scholar
  5. 5.
    Fredembach, C., Süsstrunk, S.: Colouring the near infrared. In: Proceedings of the IS & T/SID 16th Color Imaging Conference, pp. 176–182 (2008)Google Scholar
  6. 6.
    Fredembach, C., Süsstrunk, S.: Illuminat estimation and detection using near infrared. In: SPIE/IS & T Electronic Imaging, vol. 7250 (2009)Google Scholar
  7. 7.
    Gardner, W.F., Lawton, D.T.: Interactive model-based vehicle tracking. TPAMI 18(11), 1115–1121 (1996)Google Scholar
  8. 8.
    Guo, Y., Rao, C., Samarasekera, S., Kim, J., Kumar, R., Sawhney, H.: Matching vehicles under large pose transformations using approximate 3d models and piecewise mrf model. In: CVPR (2008)Google Scholar
  9. 9.
    Guo, Y., Shan, Y., Sawhney, H., Kumar, R.: Peet: prototype embedding and embedding transition for matching vehicles over disparate viewpoints. In: CVPR (2007)Google Scholar
  10. 10.
    Hou, T., Wang, S., Qin, H.: Vehicle matching and recognition under large variations of pose and illumination. In: CVPR Workshop on Object Tracking and Classification Beyond and in the Visible Spectrum, pp. 24–29 (2009)Google Scholar
  11. 11.
    Ji, Q., Yang, X.: Real time 3d face pose discrimination based on active ir illumination. In: ICPR, vol. 4, pp. 310–313 (2002)Google Scholar
  12. 12.
    Kim, Z., Malik, J.: Fast vehicle detection with probabilistic feature grouping and its application to vehicle tracking. In: ICCV, pp. 524–531 (2003)Google Scholar
  13. 13.
    Koller, D., Daniilidis, K., Nagel, H.-H.: Model-based object tracking in monocular image sequences of road traffic scenes. IJCV 10(3), 257–281 (1993)CrossRefGoogle Scholar
  14. 14.
    Li, S.Z., Chu, R., Liao, S., Zhang, L.: Illumination invariant face recognition using near-infrared images. TPAMI 29(4), 627–639 (2007)Google Scholar
  15. 15.
    Morris, N.J.W., Avidan, S., Matusik, W., Pfister, H.: Statistics of infrared images. In: CVPR (2005)Google Scholar
  16. 16.
    Murase, H., Nayar, S.K.: Visual learning and recognition of 3-d objects from appearance. IJCV 14(1), 5–24 (1995)CrossRefGoogle Scholar
  17. 17.
    Novotny, P.M., Ferrier, N.J.: Using infrared sensors and the phong illumination model to measure distances. In: ICRA, pp. 1644–1649 (1999)Google Scholar
  18. 18.
    Pan, K., Liao, S., Zhang, Z., Li, S.Z., Zhang, P.: Part-based face recognition using near infrared images. In: CVPR (2007)Google Scholar
  19. 19.
    Ramamoorthi, R., Hanrahan, P.: A signal-processing framework for inverse rendering. In: SIGGRAPH, pp. 117–128 (2001)Google Scholar
  20. 20.
    Romdhani, S., Blanz, V., Vetter, T.: Face identification by fitting a 3d morphable model using linear shape and texture error functions. In: ECCV, pp. 3–19 (2002)Google Scholar
  21. 21.
    Sato, Y., Wheeler, M.D., Ikeuchi, K.: Object shape and reflectance modeling from observation. In: SIGGRAPH, pp. 379–388 (1997)Google Scholar
  22. 22.
    Shan, Y., Sawhney, H.S., Kumar, R.: Vehicle identification between non-overlapping cameras without direct feature matching. In: ICCV, pp. 378–385 (2005)Google Scholar
  23. 23.
    Shan, Y., Sawhney, H.S., Kumar, R.: Unsupervised learning of discriminative edge measures for vehicle matching between non-overlapping cameras. TPAMI 30(4):700–711 (2008)Google Scholar
  24. 24.
    Wang, O., Davis, J., Chuang, E., Rickard, I., de Mesa, K., Dave, C.: Video relighting using infrared illumination. In: Computer Graphics Forum (Proceedings Eurographics), vol. 27(2) (2008)Google Scholar
  25. 25.
    Wang, Y., Liu, Z.C., Hua, G., Wen, Z., Zhang, Z.Y., Samaras, D.: Face re-lighting from a single image under harsh lighting conditions. In: CVPR (2007)Google Scholar
  26. 26.
    Wen, Z., Liu, Z., Huang, T.S.: Face relighting with radiance environment maps. In: CVPR, pp. 158–165 (2003)Google Scholar
  27. 27.
    Yu, Y., Debevec, P., Malik, J., Hawkins, T.: Inverse global illumination: recovering reflectance models of real scenes from photographs. In: SIGGRAPH, pp. 215–224 (1999)Google Scholar
  28. 28.
    Zhang, L., Samars, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. TPAMI 28(3), 351–363 (2006)Google Scholar
  29. 29.
    Zhang, L., Wang, S., Samaras, D.: Face synthesis and recognition from a single image under arbitrary unknown lighting using a spherical harmonic basis morphable model. In: CVPR, pp. 209–216 (2005)Google Scholar
  30. 30.
    Zhao, S., Grigat, R.-R.: Robust eye detection under active infrared illumination. In: ICPR, vol. 4, pp. 481–484 (2006)Google Scholar
  31. 31.
    Zhu, Z., Ji, Q., Fujimura, K., Lee, K.: Combining Kalman filtering and mean shift for real time eye tracking under active ir illumination. In: ICPR, vol. 4, pp. 318–321 (2002)Google Scholar
  32. 32.
    Zou, X., Kittler, J., Messer, K.: Face recognition using active near-ir illumination. In: BMVC (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Department of Computer ScienceStony Brook University (SUNY)Stony Brook,USA
  2. 2.Kodak Research LaboratoriesEastman Kodak CompanyRochesterUSA

Personalised recommendations