InECCE2019 pp 425-436 | Cite as

Open-Set Face Recognition in Video Surveillance: A Survey

  • Wasseem N. Ibrahem Al-ObaydyEmail author
  • Shahrel Azmin Suandi
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 632)


Face recognition has received a substantial attention by the vision community over the past few decades. Most of the proposed frameworks have adopted the closed-set form of face recognition. However, when a novel unregistered face is presented to the system, the result will be misclassification. A more general and challenging open-set face recognition scheme is highly desirable due to its ability in dealing with the unknown persons which are not enrolled before. We observed that there is a shortage in survey papers that explore the research endeavors in open-set face recognition. In this paper, we present a literature survey of the open-set face recognition approaches that have been introduced for real-world scenarios focusing on video surveillance applications. Moreover, we discuss the current difficulties and suggest the promising directions for future research. The paper also describes the evaluation metrics and available benchmarking face video surveillance databases.


Open-set face recognition Video surveillance Identity of Interest (IoI) 


  1. 1.
    Jain AK, Li SZ (2011) Handbook of face recognition. Springer, BerlinGoogle Scholar
  2. 2.
    Pagano C, Granger E, Sabourin R, Gorodnichy DO (2012) Detector ensembles for face recognition in video surveillance. In: The 2012 international joint conference on neural networks (IJCNN), pp 1–8. IEEEGoogle Scholar
  3. 3.
    Bashbaghi S, Granger E, Sabourin R, Bilodeau GA (2017) Robust watch-list screening using dynamic ensembles of SVMs based on multiple face representations. Mach Vis Appl 28(1–2):219–241CrossRefGoogle Scholar
  4. 4.
    De-la Torre M, Granger E, Sabourin R, Gorodnichy DO (2015) An adaptive ensemble-based system for face recognition in person re-identification. Mach Vis Appl 26(6):741–773CrossRefGoogle Scholar
  5. 5.
    Gao H, Ekenel HK, Stiefelhagen R (2010) Robust open-set face recognition for small scale convenience applications. In: Joint pattern recognition symposium, pp 393–402. Springer, BerlinGoogle Scholar
  6. 6.
    Grgic M, Delac K, Grgic S (2011) SCface-surveillance cameras face database. Multimedia Tools Appl 51(3):863–879CrossRefGoogle Scholar
  7. 7.
    Kamgar-Parsi B, Lawson W, Kamgar-Parsi B (2011) Toward development of a face recognition system for watchlist surveillance. IEEE Trans Pattern Anal Mach Intell 33(10):1925–1937CrossRefGoogle Scholar
  8. 8.
    Abate AF, Nappi M, Riccio D, Sabatino G (2007) 2D and 3D face recognition: a survey. Pattern Recogn Lett 28(14):1885–1906CrossRefGoogle Scholar
  9. 9.
    Chihaoui M, Elkefi A, Bellil W, Ben Amar C (2016) A survey of 2D face recognition techniques. Computers 5(4):21CrossRefGoogle Scholar
  10. 10.
    Tan X, Chen S, Zhou ZH, Zhang F (2006) Face recognition from a single image per person: a survey. Pattern Recogn 39(9):1725–1745CrossRefGoogle Scholar
  11. 11.
    Zhao W, Chellappa R, Phillips PJ, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv (CSUR) 35(4):399–458CrossRefGoogle Scholar
  12. 12.
    Dhifli W, Diallo AB (2016) Face recognition in the wild. Procedia Comput Sci 96:1571–1580CrossRefGoogle Scholar
  13. 13.
    Sapkota A, Boult TE (2013) Large scale unconstrained open set face database. In: 2013 IEEE sixth international conference on biometrics: theory, applications and systems (BTAS), pp 1–8. IEEEGoogle Scholar
  14. 14.
    Wechsler H (2009) Reliable face recognition methods: system design, implementation and evaluation, vol 7. Springer Science & Business Media, BerlinGoogle Scholar
  15. 15.
    Scheirer WJ, de Rezende Rocha A, Sapkota A, Boult TE (2013) Toward open set recognition. IEEE Trans Pattern Anal Mach Intell 35(7):1757–1772CrossRefGoogle Scholar
  16. 16.
    dos Santos CE Jr, Schwartz WR (2014) Extending face identification to open-set face recognition. In: 2014 27th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), pp 188–195. IEEEGoogle Scholar
  17. 17.
    Stallkamp J, Ekenel HK, Stiefelhagen R (2007) Video-based face recognition on real-world data. In: IEEE 11th international conference on computer vision, ICCV 2007, pp 1–8. IEEEGoogle Scholar
  18. 18.
    Li F, Wechsler H (2005) Open set face recognition using transduction. IEEE Trans Pattern Anal Mach Intell 27(11):1686–1697CrossRefGoogle Scholar
  19. 19.
    Tax DM, Duin RP (2008) Growing a multi-class classifier with a reject option. Pattern Recogn Lett 29(10):1565–1570CrossRefGoogle Scholar
  20. 20.
    Qiu J, Zhang Y, Sun J (2013) Face recognition in open world environment. In: Visual communications and image processing (VCIP), pp 1–6. IEEEGoogle Scholar
  21. 21.
    Kohir VV, Desai U (2000) Face recognition. In: The 2000 IEEE international symposium on circuits and systems, Proceedings, ISCAS 2000, Geneva, vol 5, pp 305–308. IEEEGoogle Scholar
  22. 22.
    Hatami N, Ebrahimpour R, Ghaderi R (2008) ECOC-based training of neural networks for face recognition. In: 2008 IEEE conference on cybernetics and intelligent systems, pp 450–454. IEEEGoogle Scholar
  23. 23.
    Paliy I, Sachenko A, Koval V, Kurylyak Y (2005) Approach to face recognition using neural networks. In: Intelligent data acquisition and advanced computing systems: technology and applications, IDAACS 2005, pp 112–115. IEEEGoogle Scholar
  24. 24.
    Ravi J, Tevaramani SS, Raja K (2012) Face recognition using DT-CWT and LBP features. In: 2012 international conference on computing, communication and applications (ICCCA), pp 1–6. IEEEGoogle Scholar
  25. 25.
    Sebastiao R, Silva JA, Padilha AJ (2006) Face recognition from spatially-morphed video sequences. In: International conference image analysis and recognition, pp 365–374. Springer, BerlinGoogle Scholar
  26. 26.
    Sing JK, Basu DK, Nasipuri M, Kundu M (2007) Face recognition using point symmetry distance-based RBF network. Appl Soft Comput 7(1):58–70CrossRefGoogle Scholar
  27. 27.
    Theodorakopoulos I, Rigas I, Economou G, Fotopoulos S (2011) Face recognition via local sparse coding. In: 2011 IEEE international conference on computer vision (ICCV), pp 1647–1652. IEEEGoogle Scholar
  28. 28.
    Vareto RH, Silva S, de Oliveira Costa F, Schwartz WR (2017) Towards open-set face recognition using hashing functions. In: 2017 IEEE international joint conference on biometrics (IJCB), pp 634–641. IEEEGoogle Scholar
  29. 29.
    Zhang B, Hao H (2014) Open-set face recognition by transductive kernel associative memory. In: 2014 7th international congress on image and signal processing (CISP), pp 633–638. IEEEGoogle Scholar
  30. 30.
    Zhang H, Patel VM (2017) Sparse representation-based open set recognition. IEEE Trans Pattern Anal Mach Intell 39(8):1690–1696CrossRefGoogle Scholar
  31. 31.
    Tjahyadi R, Liu W, An S, Venkatesh S (2005) Face recognition based on ordinal correlation approach. In: Proceedings of the 2005 international conference on intelligent sensors, sensor networks and information processing conference, pp 349–354. IEEEGoogle Scholar
  32. 32.
    Khan SS, Madden MG (2009) A survey of recent trends in one class classification. In: Irish conference on artificial intelligence and cognitive science, pp 188–197. Springer, BerlinGoogle Scholar
  33. 33.
    Moeini A, Faez K, Moeini H, Safai AM (2017) Open-set face recognition across look-alike faces in real-world scenarios. Image Vis Comput 57:1–14CrossRefGoogle Scholar
  34. 34.
    Kato H, Chakraborty G, Ogata N, Chakraborty B (2011) A real-time angle aware face recognition system based on artificial neural network. In: 2011 3rd international conference on awareness science and technology (iCAST), pp 521–526. IEEEGoogle Scholar
  35. 35.
    Chen C, Zhan Y, Wen C (2009) Hierarchical face recognition based on SVDD and SVM. In: International conference on environmental science and information application technology, ESIAT 2009, vol 2, pp 692–695. IEEEGoogle Scholar
  36. 36.
    Tax DM, Duin RP (2002) Using two-class classifiers for multiclass classification. In: 16th international conference on pattern recognition. Proceedings, vol 2, pp 124–127. IEEEGoogle Scholar
  37. 37.
    Ekenel HK, Szasz-Toth L, Stiefelhagen R (2009) Open-set face recognition-based visitor interface system. In: International conference on computer vision systems, pp 43–52. Springer, BerlinGoogle Scholar
  38. 38.
    Hsu GSJ (2009) Fusion classifier for open-set face recognition with pose variations. Int J Comput Electr Autom Control Inf Eng 3(8):2099–2105Google Scholar
  39. 39.
    Xue S, Zhu H (2019) Low-resolution and open-set face recognition via recursive label propagation based on statistical classification. Int J Wavelets Multiresolut Inf Process 17(02):1940002MathSciNetCrossRefGoogle Scholar
  40. 40.
    Al-Obaydy WNI, Suandi SA (2018) Open-set single-sample face recognition in video surveillance using fuzzy ARTMAP. Neural Comput Appl. Scholar
  41. 41.
    Xie H, Du Y, Yu H, Chang Y, Xu Z, Tang Y (2018) Open set face recognition with deep transfer learning and extreme value statistics. Int J Wavelets Multiresolut Inf Process 16(04):1850034MathSciNetCrossRefGoogle Scholar
  42. 42.
    Toufiq R, Islam MR (2014) Face recognition system using PCA-ANN technique with feature fusion method. In: 2014 international conference on electrical engineering and information & communication technology (ICEEICT), pp 1–5. IEEEGoogle Scholar
  43. 43.
    Zhang B (2012) Reliable face recognition by random subspace support vector machine ensemble. In: 2012 International conference on machine learning and cybernetics (ICMLC), vol 1, pp 415–420. IEEEGoogle Scholar
  44. 44.
    Pankaj DS, Wilscy M (2011) Face recognition using fuzzy neural network classifier. In: Advances in parallel distributed computing, vol 203, pp 53–62. Springer, BerlinGoogle Scholar
  45. 45.
    Chen JC, Shi SY, Lien JJJ (2010) Face recognition and unseen subject rejection in margin-enhanced space. In: 2010 international conference on system science and engineering (ICSSE), pp 631–636. IEEEGoogle Scholar
  46. 46.
    Nakamura K, Takano H (2007) Unregistered face discrimination by the face orientation and size recognition. In: International joint conference on neural networks, IJCNN 2007, pp 1924–1928. IEEEGoogle Scholar
  47. 47.
    Mu X, Artiklar M, Watta P, Hassoun MH (2006) An RCE-based associative memory with application to human face recognition. Neural Process Lett 23(3):257–271CrossRefGoogle Scholar
  48. 48.
    Seow MJ, Gottumukkal R, Valaparla D, Asari K (2004) A robust face recognition system for real time surveillance. In: International conference on information technology: coding and computing, Proceedings, ITCC 2004, vol 1, pp 631–635. IEEEGoogle Scholar
  49. 49.
    Fawcett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874MathSciNetCrossRefGoogle Scholar
  50. 50.
    Salka TD, Hanafi M, Mashohor S, Ahamad SMS (2015) Large variability surveillance camera face database. In: 2015 seventh international conference on computational intelligence, modelling and simulation (CIMSim), pp 108–112. IEEEGoogle Scholar
  51. 51.
    Wong Y, Chen S, Mau S, Sanderson C, Lovell BC (2011) Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition. In: 2011 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 74–81. IEEEGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Wasseem N. Ibrahem Al-Obaydy
    • 1
    • 2
    Email author
  • Shahrel Azmin Suandi
    • 1
  1. 1.Intelligent Biometric Group, School of Electrical and Electronic EngineeringUniversiti Sains MalaysiaNibong TebalMalaysia
  2. 2.Computer Engineering DepartmentUniversity of TechnologyBaghdadIraq

Personalised recommendations