Abstract
Characteristics of surveillance video generally include low resolution and poor quality due to environmental, storage and processing limitations. It is extremely difficult for computers and human opera- tors to identify individuals from these videos. To overcome this problem, super-resolution can be used in conjunction with an automated face recognition system to enhance the spatial resolution of video frames containing the subject and narrow down the number of manual verifications performed by the human operator by presenting a list of most likely candidates from the database. As the super-resolution reconstruction process is ill-posed, visual artifacts are often generated as a result. These artifacts can be visually distracting to humans and/or affect machine recognition algorithms. While it is intuitive that higher resolution should lead to improved recognition accuracy, the effects of super-resolution and such artifacts on face recognition performance have not been systematically studied. This paper aims to address this gap while illustrating that super-resolution allows more accurate identification of individuals from low-resolution surveillance footage. The proposed optical flow-based super-resolution method is benchmarked against Baker et al.’s hallucination and Schultz et al.’s super-resolution techniques on images from the Terrascope and XM2VTS databases. Ground truth and interpolated images were also tested to provide a baseline for comparison. Results show that a suitable super-resolution system can improve the discriminability of surveillance video and enhance face recognition accuracy. The experiments also show that Schultz et al.’s method fails when dealing surveillance footage due to its assumption of rigid objects in the scene. The hallucination and optical flow-based methods performed comparably, with the optical flow-based method producing less visually distracting artifacts that interfered with human recognition.
Chapter PDF
Similar content being viewed by others
References
Gunturk, B., Batur, A., Altunbasak, Y., III, M.H., Mersereau, R.: Eigenface-domain super-resolution for face recognition. IEEE Transactions on Image Processing 12(5), 597–606 (2003)
Lemieux, A., Parizeau, M.: Experiments on eigenfaces robustness. In: Proc. ICPR-2002, vol. 1, pp. 421–424 (August 2002)
Wang, X., Tang, X.: Face Hallucination and Recognition. In: Kittler, J., Nixon, M.S. (eds.) AVBPA 2003. LNCS, vol. 2688, pp. 486–494. Springer, Heidelberg (2003)
Jaynes, C., Kale, A., Sanders, N., Grossmann, E.: The Terrascope dataset: scripted multi-camera indoor video surveillance with ground-truth. In: Proc. Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 309–316 (October 2005)
Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3(1), 71–86 (1991)
Wiskott, L., Fellous, J., Krüger, N., Malsburg, C.: Face recognition by elastic bunch graph matching. In: Sommer, G., Daniilidis, K., Pauli, J. (eds.) CAIP 1997. LNCS, vol. 1296, pp. 456–463. Springer, Heidelberg (1997)
Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: XM2VTS: The Extended M2VTS Database. In: Proc. AVBPA-1999, pp. 72–76 (1999)
Park, S., Park, M., Kang, M.: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 25(9), 21–36 (2003)
Tsai, R., Huang, T.: Multiframe image restoration and registration. Advances in Computer Vision and image Processing 1, 317–339 (1984)
Baker, S., Kanade, T.: Limits on Super-Resolution and How to Break Them. 24(9), 1167–1183 (2002)
Baker, S., Kanade, T.: Super Resolution Optical Flow. Technical Report CMU-RI-TR-99-36, The Robotics Institute, Carnegie Mellon University (October 1999)
Lin, F., Fookes, C., Chandran, V., Sridharan, S.: Investigation into Optical Flow Super-Resolution for Surveillance Applications. In: Proc. APRS Workshop on Digital Image Computing 2005, pp. 73–78 (February 2005)
Black, M., Anandan, P.: A framework for the robust estimation of optical flow. In: Proc. ICCV-1993, pp. 231–236 (May 1993)
Schultz, R., Stevenson, R.: Extraction of High-Resolution Frames from Video Sequences. IEEE Transactions on Image Processing 5(6), 996–1011 (1996)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR (2001)
Bolme, D., Beveridge, R., Teixeira, M., Draper, B.: The CSU Face Identification Evaluation System: Its Purpose, Features and Structure. In: Proc. International Conference on Vision Systems, pp. 304–311 (April 2003)
Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. In: Proc. CVPR 2005, vol. 1, pp. 947–954 (2005)
Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The feret evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1090–1104 (2000)
Lin, F., Cook, J., Chandran, V., Sridharan, S.: Face Recognition from Super-Resolved Images. In: Proc. ISSPA 2005, pp. 667–670 (August 2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, F., Fookes, C., Chandran, V., Sridharan, S. (2007). Super-Resolved Faces for Improved Face Recognition from Surveillance Video. In: Lee, SW., Li, S.Z. (eds) Advances in Biometrics. ICB 2007. Lecture Notes in Computer Science, vol 4642. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74549-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-74549-5_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74548-8
Online ISBN: 978-3-540-74549-5
eBook Packages: Computer ScienceComputer Science (R0)