Benchmarking Still-to-Video Face Recognition via Partial and Local Linear Discriminant Analysis on COX-S2V Dataset

  • Zhiwu Huang
  • Shiguang Shan
  • Haihong Zhang
  • Shihong Lao
  • Alifu Kuerban
  • Xilin Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7725)


In this paper, we explore the real-world Still-to-Video (S2V) face recognition scenario, where only very few (single, in many cases) still images per person are enrolled into the gallery while it is usually possible to capture one or multiple video clips as probe. Typical application of S2V is mug-shot based watch list screening. Generally, in this scenario, the still image(s) were collected under controlled environment, thus of high quality and resolution, in frontal view, with normal lighting and neutral expression. On the contrary, the testing video frames are of low resolution and low quality, possibly with blur, and captured under poor lighting, in non-frontal view. We reveal that the S2V face recognition has been heavily overlooked in the past. Therefore, we provide a benchmarking in terms of both a large scale dataset and a new solution to the problem. Specifically, we collect (and release) a new dataset named COX-S2V, which contains 1,000 subjects, with each subject a high quality photo and four video clips captured simulating video surveillance scenario. Together with the database, a clear evaluation protocol is designed for benchmarking. In addition, in addressing this problem, we further propose a novel method named Partial and Local Linear Discriminant Analysis (PaLo-LDA). We then evaluated the method on COX-S2V and compared with several classic methods including LDA, LPP, ScSR. Evaluation results not only show the grand challenges of the COX-S2V, but also validate the effectiveness of the proposed PaLo-LDA method over the competitive methods.


Face Recognition Video Sequence Linear Discriminant Analysis Face Image Video Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Liu, X., Cheng, T.: Video-based face recognition using adaptive hidden markov models. In: CVPR, pp. 340–345 (2003)Google Scholar
  2. 2.
    Arandjelović, O., Shakhnarovich, G., Fisher, J., Cipolla, R., Darrell, T.: Face recognition with image sets using manifold density divergence. In: CVPR, vol. 1, pp. 581–588 (2005)Google Scholar
  3. 3.
    Kim, T., Kittler, J., Cipolla, R.: Discriminative learning and recognition of image set classes using canonical correlations. IEEE T-PAMI 29, 1005–1018 (2007)CrossRefGoogle Scholar
  4. 4.
    Cevikalp, H., Triggs, B.: Face recognition based on image sets. In: CVPR, pp. 2567–2573 (2010)Google Scholar
  5. 5.
    Wang, R., Guo, H., Davis, L., Dai, Q.: Covariance discriminative learning: A natural and efficient approach to image set classification. In: CVPR, pp. 2496–2503 (2012)Google Scholar
  6. 6.
    Zhou, S., Krueger, V., Chellappa, R.: Probabilistic recognition of human faces from video. CVIU 91, 214–245 (2003)Google Scholar
  7. 7.
    Zhou, S., Chellappa, R.: Beyond one still image: Face recognition from multiple still images or a video sequence. In: Face Processing: Advanced Modeling and Methods, pp. 547–567 (2005)Google Scholar
  8. 8.
    Hwang, W., Huang, X., Noh, K., Kim, J.: Face recognition system using extended curvature gabor classifier bunch for low-resolution face image. In: CVPRW on Biometrics, pp. 15–22 (2011)Google Scholar
  9. 9.
    Baker, S., Kanade, T.: Hallucinating faces. In: AFGR, pp. 83–88 (2000)Google Scholar
  10. 10.
    Liu, C., Shum, H., Zhang, C.: A two-step approach to hallucinating faces: Global parametric model and local nonparametric model. In: CVPR (2001)Google Scholar
  11. 11.
    Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE T-IP 21, 3467–3478 (2012)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Gunturk, B., Batur, A., Altunbasak, Y., Hayes III, M., Mersereau, R.: Eigenface-domain super-resolution for face recognition. IEEE T-IP 12, 597–606 (2003)CrossRefGoogle Scholar
  13. 13.
    Hennings-Yeomans, P., Baker, S., Kumar, B.: Simultaneous super-resolution and feature extraction for recognition of low-resolution faces. In: CVPR (2008)Google Scholar
  14. 14.
    Li, B., Chang, H., Shan, S., Chen, X.: Low-resolution face recognition via coupled locality preserving mappings. Signal Processing Letters 17, 20–23 (2010)CrossRefGoogle Scholar
  15. 15.
    Huang, H., He, H.: Super-resolution method for face recognition using nonlinear mappings on coherent features. IEEE T-NN 22, 121–130 (2011)CrossRefGoogle Scholar
  16. 16.
    Arandjelović, O., Cipolla, R.: A manifold approach to face recognition from low quality video across illumination and pose using implicit super-resolution. In: ICCV (2007)Google Scholar
  17. 17.
    Jia, K., Gong, S.: Generalized face super-resolution. IEEE T-IP 17, 873–886 (2008)MathSciNetCrossRefGoogle Scholar
  18. 18.
    He, X., Niyogi, P.: Locality preserving projections. In: Advances in Neural Information Processing Systems, vol. 16, pp. 153–160 (2004)Google Scholar
  19. 19.
    Huang, G., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In: Workshop on Faces in ’Real-Life’ Images: Detection, Alignment, and Recognition (2008)Google Scholar
  20. 20.
    Fisher, R.: The use of multiple measurements in taxonomic problems. Annals of Human Genetics 7, 179–188 (1936)CrossRefGoogle Scholar
  21. 21.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE T-PAMI 19, 711–720 (1997)CrossRefGoogle Scholar
  22. 22.
    Sugiyama, M.: Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. The Journal of Machine Learning Research 8, 1027–1061 (2007)zbMATHGoogle Scholar
  23. 23.
    Perona, P., Zelnik-Manor, L.: Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems, vol. 17, pp. 1601–1608 (2004)Google Scholar
  24. 24.
    Lin, D., Tang, X.: Inter-modality Face Recognition. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 13–26. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  25. 25.
    Jolliffe, I.: MyiLibrary: Principal component analysis. vol, 2. Wiley Online Library (2002)Google Scholar
  26. 26.
    Lu, J., Tan, Y.: Locality repulsion projections for image-to-set face recognition. In: ICME, pp. 1–6 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Zhiwu Huang
    • 1
    • 2
  • Shiguang Shan
    • 1
  • Haihong Zhang
    • 3
  • Shihong Lao
    • 3
  • Alifu Kuerban
    • 4
  • Xilin Chen
    • 1
  1. 1.Key Lab of Intelligent Information Processing, Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.OMRON Social Solutions Co. LtdKyotoJapan
  4. 4.College of Information Science and EngineeringXinjiang UniversityChina

Personalised recommendations