Face Recognition at a Distance

  • Frederick W. Wheeler
  • Xiaoming Liu
  • Peter H. Tu

Abstract

Face recognition at a distance is generally motivated by the desire to automatically recognize noncooperative subjects over a wide area. This remote biometric collection and identification problem has been addressed with high-resolution stationary cameras and active camera systems. Key challenges include optical system design, pan-tilt-zoom camera targeting and control, and face recognition with low-resolution images and no pose or illumination control. We discuss major applications, challenges and approaches in this field, and review research literature on this and closely related topics. We further describe a specific face recognition at a distance system that uses the active camera approach, algorithms for facial image modeling and alignment for low-resolution images, and a multi-frame super-resolution process for facial images.

References

  1. 1.
    Andersen, J.F., Busck, J., Heiselberg, H.: Long distance high accuracy 3-D laser radar and person identification. In: Kamerman, G.W. (ed.) Laser Radar Technology and Applications X, vol. 5791, pp. 9–16. SPIE, Bellingham (2005) Google Scholar
  2. 2.
    Bagdanov, A., Bimbo, A., Nunziati, W., Pernici, F.: Learning foveal sensing strategies in unconstrained surveillance environments. In: AVSS (2006) Google Scholar
  3. 3.
    Baker, S., Kanade, T.: Super resolution optical flow. Tech. Rep. CMU-RI-TR-99-36, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA (1999) Google Scholar
  4. 4.
    Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002) CrossRefGoogle Scholar
  5. 5.
    Bellotto, N., Sommerlade, E., Benfold, B., Bibby, C., Reid, I., Roth, D., Fernández, C., Gool, L.V., Gonzàlez, J.: A distributed camera system for multi-resolution surveillance. In: Proc. of the ACM/IEEE Intl. Conf. on Distributed Smart Cameras (ICDSC) (2009) Google Scholar
  6. 6.
    Bimbo, A.D., Pernici, F.: Towards on-line saccade planning for high-resolution image sensing. Pattern Recognit. Lett. 27(15), 1826–1834 (2006) CrossRefGoogle Scholar
  7. 7.
    Blackman, S., Popoli, R.: Design and Analysis of Modern Tracking Systems. Artech House, Norwood (1999) MATHGoogle Scholar
  8. 8.
    Borman, S.: Topics in multiframe superresolution restoration. Ph.D. thesis, University of Notre Dame, Notre Dame, IN (2004) Google Scholar
  9. 9.
    Bowyer, K.W., Chang, K., Flynn, P.: A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Comput. Vis. Image Underst. 101(1), 1–15 (2006) CrossRefGoogle Scholar
  10. 10.
    Chang, K., Bowyer, K., Flynn, P.: Face recognition using 2D and 3D facial data. In: Proc. ACM Workshop on Multimodal User Authentication, pp. 25–32 (2003) Google Scholar
  11. 11.
    Chaudhuri, S. (ed.): Super-Resolution Imaging, 3rd edn. Kluwer Academic, Dordrecht (2001) Google Scholar
  12. 12.
    Cootes, T., Cooper, D., Tylor, C., Graham, J.: A trainable method of parametric shape description. In: BMVC, pp. 54–61 (1991) Google Scholar
  13. 13.
    Cootes, T., Taylor, C., Lanitis, A.: Active shape models: Evaluation of a multi-resolution method for improving image search. In: BMVC, vol. 1, pp. 327–336 (1994) Google Scholar
  14. 14.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001) CrossRefGoogle Scholar
  15. 15.
    Costello, C.J., Diehl, C.P., Banerjee, A., Fisher, H.: Scheduling an active camera to observe people. In: Proc. of the ACM Intl. Workshop on Video Surveillance and Sensor Networks, pp. 39–45 (2004) CrossRefGoogle Scholar
  16. 16.
    Davis, J., Morison, A., Woods, D.: An adaptive focus-of-attention model for video surveillance and monitoring. Mach. Vis. Appl. 18(1), 41–64 (2007) CrossRefGoogle Scholar
  17. 17.
    Davis, J., Morison, A., Woods, D.: Building adaptive camera models for video surveillance. In: WACV (2007) Google Scholar
  18. 18.
    Dedeoglu, G., Baker, S., Kanade, T.: Resolution-aware fitting of active appearance models to low-resolution images. In: ECCV (2006) Google Scholar
  19. 19.
    Elder, J.H., Prince, S., Hou, Y., Sizintsev, M., Oleviskiy, Y.: Pre-attentive and attentive detection of humans in wide-field scenes. Int. J.Comput. Vis. 72, 47–66 (2007) CrossRefGoogle Scholar
  20. 20.
    Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super-resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004) CrossRefGoogle Scholar
  21. 21.
    Greiffenhagen, M., Ramesh, V., Comaniciu, D., Niemann, H.: Statistical modeling and performance characterization of a real-time dual camera surveillance system. In: CVPR (2000) Google Scholar
  22. 22.
    Hampapur, A., Pankanti, S., Senior, A., Tian, Y.L., Brown, L., Bolle, R.: Face cataloger: multi-scale imaging for relating identity to location. In: AVSS, pp. 13–20 (2003) Google Scholar
  23. 23.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000) MATHGoogle Scholar
  24. 24.
    Krahnstoever, N., Tu, P., Sebastian, T., Perera, A., Collins, R.: Multi-view detection and tracking of travelers and luggage in mass transit environments. In: Proc. Ninth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS) (2006) Google Scholar
  25. 25.
    Krahnstoever, N., Yu, T., Lim, S.N., Patwardhan, K., Tu, P.: Collaborative real-time control of active cameras in large scale surveillance systems. In: Proc. Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (M2SFA2) (2008) Google Scholar
  26. 26.
    Liang, L., Wen, F., Xu, Y., Tang, X., Shum, H.: Accurate face alignment using shape constrained Markov network. In: CVPR (2006) Google Scholar
  27. 27.
    Lim, S.N., Davis, L.S., Mittal, A.: Constructing task visibility intervals for a surveillance system. ACM Multimedia Systems Journal 12(3) (2006) Google Scholar
  28. 28.
    Lim, S.N., Davis, L., Mittal, A.: Task scheduling in large camera network. In: ACCV (2007) Google Scholar
  29. 29.
    Liu, X.: Discriminative face alignment. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1941–1954 (2009) CrossRefGoogle Scholar
  30. 30.
    Liu, X.: Video-based face model fitting using adaptive active appearance model. Image Vis. Comput. 28(7), 1162–1172 (2010) CrossRefGoogle Scholar
  31. 31.
    Liu, Z., Sarkar, S.: Outdoor recognition at a distance by fusing gait and face. Image Vis. Comput. 25(6), 817–832 (2007) CrossRefGoogle Scholar
  32. 32.
    Liu, K.R., Kang, M.G., Chaudhuri, S. (eds.): IEEE Signal Processing Magazine, Special edition: Super-Resolution Image Reconstruction, vol. 20, no. 3. IEEE (2003) Google Scholar
  33. 33.
    Liu, X., Tu, P.H., Wheeler, F.W.: Face model fitting on low resolution images. In: BMVC (2006) Google Scholar
  34. 34.
    Marchesotti, L., Marcenaro, L., Regazzoni, C.: Dual camera system for face detection in unconstrained environments. In: ICIP (2003) Google Scholar
  35. 35.
    Medioni, G., Choi, J., Kuo, C.H., Choudhury, A., Zhang, L., Fidaleo, D.: Non-cooperative persons identification at a distance with 3D face modeling. In: BTAS (2007) Google Scholar
  36. 36.
    Medioni, G., Fidaleo, D., Choi, J., Zhang, L., Kuo, C.H., Kim, K.: Recognition of non-cooperative individuals at a distance with 3D face modeling. In: 2007 IEEE Workshop on Automatic Identification Advanced Technologies, pp. 112–117 (2007) CrossRefGoogle Scholar
  37. 37.
    Medioni, G., Choi, J., Kuo, C.H., Fidaleo, D.: Identifying noncooperative subjects at a distance using face images and inferred three-dimensional face models. IEEE Trans. Syst. Man Cybern., Part A, Syst. Hum. 39(1), 12–24 (2009) CrossRefGoogle Scholar
  38. 38.
    Mortazavian, P., Kittler, J., Christmas, W.: A 3-D assisted generative model for facial texture super-resolution. In: BTAS, pp. 1–7 (2009) Google Scholar
  39. 39.
    NIST Multiple Biometric Grand Challenge. http://face.nist.gov/mbgc
  40. 40.
    O’Toole, A., Harms, J., Snow, S., Hurst, D., Pappas, M., Ayyad, J., Abdi, H.: A video database of moving faces and people. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 812–816 (2005) CrossRefGoogle Scholar
  41. 41.
    Prince, S., Elder, J., Hou, Y., Sizinstev, M., Olevsky, E.: Towards face recognition at a distance. In: Proc. of the IET Conf. on Crime and Security, pp. 570–575 (2006) CrossRefGoogle Scholar
  42. 42.
    Prince, S., Elder, J., Warrell, J., Felisberti, F.: Tied factor analysis for face recognition across large pose differences. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 970–984 (2008) CrossRefGoogle Scholar
  43. 43.
    Qureshi, F., Terzopoulos, D.: Surveillance in virtual reality: System design and multi-camera control. In: CVPR, pp. 1–8 (2007) Google Scholar
  44. 44.
    Qureshi, F., Terzopoulos, D.: Multi-camera control through constraint satisfaction for persistent surveillance. In: AVSS, pp. 211–218 (2008) Google Scholar
  45. 45.
    Qureshi, F., Terzopoulos, D.: Smart camera networks in virtual reality. Proc. IEEE 96(10), 1640–1656 (2008) CrossRefGoogle Scholar
  46. 46.
    Rara, H., Elhabian, S., Ali, A., Miller, M., Starr, T., Farag, A.: Distant face recognition based on sparse-stereo reconstruction. In: ICIP, pp. 4141–4144 (2009) Google Scholar
  47. 47.
    Rara, H., Elhabian, S., Ali, A., Miller, M., Starr, T., Farag, A.: Face recognition at-a-distance based on sparse-stereo reconstruction. In: CVPR Workshop on Biometrics, pp. 27–32 (2009) Google Scholar
  48. 48.
    Redman, B., Höft, T., Grow, T., Novotny, J., McCumber, P., Rogers, N., Hoening, M., Kubala, K., Sibell, R., Shald, S., Uberna, R., Havermann, R., Sandalphon, D.: Low-cost, stand-off, 2D+3D face imaging for biometric identification using Fourier transform profilometry. In: 2009 Military Sensing Symposia (MSS) Specialty Group on Active E-O Systems, vol. 1. Las Vegas, NV (2009) Google Scholar
  49. 49.
    Redman, B., Marron, J., Seldomridge, N., Grow, T., Höft, T., Novotny, J., Thurman, S.T., Embry, C., Bratcher, A., Kendrick, R.: Stand-off 3D face imaging and vibrometry for biometric identification using digital holography. In: 2009 Military Sensing Symposia (MSS) Specialty Group on Active E-O Systems, vol. 1. Las Vegas, NV (2009) Google Scholar
  50. 50.
    Ross, A.A., Nandakumar, K., Jain, A.K. (eds.): Handbook of Multibiometrics. Springer, Berlin (2006) Google Scholar
  51. 51.
    Senior, A., Hampapur, A., Lu, M.: Acquiring multi-scale images by pan-tilt-zoom control and automatic multi-camera calibration. In: WACV, vol. 1, pp. 433–438 (2005) Google Scholar
  52. 52.
    Stillman, S., Tanawongsuwan, R., Essa, I.: A system for tracking and recognizing multiple people with multiple cameras. In: Proc. of 2nd Intl. Conf. on Audio-Vision-based Person Authentication, pp. 96–101 (1998) Google Scholar
  53. 53.
    Tistarelli, M., Li, S.Z., Chellappa, R. (eds.): Handbook of Remote Biometrics for Surveillance and Security. Springer, Berlin (2009) Google Scholar
  54. 54.
    Tu, P.H., Doretto, G., Krahnstoever, N.O., Perera, A.G.A., Wheeler, F.W., Liu, X., Rittscher, J., Sebastian, T.B., Yu, T., Harding, K.G.: An intelligent video framework for homeland protection. In: Proc. of SPIE Defense & Security Symposium, Conference on Unattended Ground, Sea, and Air Sensor Technologies and Applications IX. Orlando, FL (2007) Google Scholar
  55. 55.
    Wheeler, F.W., Liu, X., Tu, P.H.: Multi-frame super-resolution for face recognition. In: BTAS (2007) Google Scholar
  56. 56.
    Wheeler, F.W., Weiss, R.L., Tu, P.H.: Face recognition at a distance system for surveillance applications. In: BTAS (2010) Google Scholar
  57. 57.
    Yan, S., Liu, C., Li, S.Z., Zhang, H., Shum, H.Y., Cheng, Q.: Face alignment using texture-constrained active shape models. Image Vis. Comput. 21(1), 69–75 (2003) CrossRefGoogle Scholar
  58. 58.
    Yao, Y., Abidi, B., Kalka, N., Schmid, N., Abidi, M.: High magnification and long distance face recognition: Database acquisition, evaluation, and enhancement. In: Proc. Biometrics Symposium (2006) Google Scholar
  59. 59.
    Yao, Y., Abidi, B., Kalka, N.D., Schmid, N., Abidi, M.: Super-resolution for high magnification face images. In: Prabhakar, S., Ross, A.A. (eds.) Proceedings of the SPIE, Biometric Technology for Human Identification IV, vol. 6539. Orlando, FL (2007) Google Scholar
  60. 60.
    Yao, Y., Abidi, B.R., Kalka, N.D., Schmid, N.A., Abidi, M.A.: Improving long range and high magnification face recognition: Database acquisition, evaluation, and enhancement. Comput. Vis. Image Underst. 111(2), 111–125 (2008) CrossRefGoogle Scholar
  61. 61.
    Yu, T., Lim, S.N., Patwardhan, K., Krahnstoever, N.: Monitoring, recognizing and discovering social networks. In: CVPR (2009) Google Scholar
  62. 62.
    Zhou, X., Bhanu, B.: Feature fusion of face and gait for human recognition at a distance in video. In: ICPR, vol. 4, pp. 529–532 (2006) Google Scholar
  63. 63.
    Zhou, X., Bhanu, B.: Integrating face and gait for human recognition at a distance in video. IEEE Trans. Syst. Man Cybern., Part B, Cybern. 37(5), 1119–1137 (2007) CrossRefGoogle Scholar
  64. 64.
    Zhou, X., Collins, R., Kanade, T., Metes, P.: A master-slave system to acquire biometric imagery of humans at distance. In: ACM International Workshop on Video Surveillance (2003) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Frederick W. Wheeler
    • 1
  • Xiaoming Liu
    • 1
  • Peter H. Tu
    • 1
  1. 1.Visualization and Computer Vision LabGE Global ResearchNiskayunaUSA

Personalised recommendations