Evaluation of Face Datasets as Tools for Assessing the Performance of Face Recognition Methods
- 311 Downloads
Face datasets are considered a primary tool for evaluating the efficacy of face recognition methods. Here we show that in many of the commonly used face datasets, face images can be recognized accurately at a rate significantly higher than random even when no face, hair or clothes features appear in the image. The experiments were done by cutting a small background area from each face image, so that each face dataset provided a new image dataset which included only seemingly blank images. Then, an image classification method was used in order to check the classification accuracy. Experimental results show that the classification accuracy ranged between 13.5% (color FERET) to 99% (YaleB). These results indicate that the performance of face recognition methods measured using face image datasets may be biased. Compilable source code used for this experiment is freely available for download via the Internet.
KeywordsFace recognition Biometrics FERET
- Gabor, D. (1946). Theory of communication. Journal of IEEE, 93, 429–457. Google Scholar
- Goldberg, I. G., Allan, C., Burel, J. M., Creager, D., Falconi, A., Hochheiser, H., Johnston, J., Mellen, J., Sorger, P. K., & Swedlow, J. R. (2005). The Open Microscopy Environment (OME) Data Model and XML file: open tools for informatics and quantitative analysis in biological imaging. Genome Biology, 6, R47. CrossRefGoogle Scholar
- Gradshtein, I., & Ryzhik, I. (1994). Table of integrals, series and products (5th edn., p. 1054). New York: Academic Press. Google Scholar
- Gross, R., Baker, S., Matthews, I., & Kanade, T. (2004). Face recognition across pose and illumination. In S. Z. Lin & A. K. Jain (Eds.), Handbook of face recognition. Berlin: Springer. Google Scholar
- Hadjidementriou, E., Grossberg, M., & Nayar, S. (2001). Spatial information in multiresolution histograms. In IEEE conf. on computer vision and pattern recognition (Vol. 1, p. 702). Google Scholar
- Haralick, R. M., Shanmugam, K., & Dimstein, I. (1973). Textural features for image classification. IEEE Transaction on System, Man and Cybernetics, 6, 269–285. Google Scholar
- Jain, V., & Mukherjee, A. (2002). http://vis-www.cs.umass.edu/%7Evidit/IndianFaceDatabase.
- Lim, J. S. (1990). Two-dimensional signal and image processing. In Signals, systems, and the Fourier transform (pp. 42–45). Englewood Cliffs: Prentice Hall. Google Scholar
- Lynos, M., Akamatsu, S., Kamachi, M., & Gyboa, J. (1998). Coding facial expressions with Gabor wavelets. In Proceedings of the third IEEE international conference on automatic face and facial recognition (pp. 200–205). Google Scholar
- Murphy, R. F., Velliste, M., Yao, J., & Porreca, G. (2001). Searching online journals for fluorescence microscopy images depicting protein subcellular location patterns. In Proceedings of the second IEEE international symposium on bioinformatics and biomedical engeering (pp. 119–128). Google Scholar
- Orlov, N., Johnston, J., Macura, T., Shamir, L., & Goldberg, I. G. (2007). Computer vision for microscopy applications. In G. Obinata & A. Dutta (Eds.), Vision systems—segmentation and pattern recognition (pp. 221–242). Vienna: ARS. Google Scholar
- Orlov, N., Shamir, L., Johnston, J., Macura, T., Eckley, D. M., & Goldberg, I. G. (2008, in press). WND-CHARM: Multi-purpose image classification using compound image transforms. Pattern Recognition Letters. Google Scholar
- Otsu, N. (1979). A threshold selection method from gray level histograms. IEEE Transactions on System, Man and Cybernetics, 9, 62–66. Google Scholar
- Prewitt, J. M. (1970). Object enhancement and extraction. In B. S. Lipkin & A. Rosenfeld (Eds.), Picture processing and psychopictoris (pp. 75–149). New York: Academic Press. Google Scholar
- Rodenacker, K., & Bengsson, E. (2003). A feature set for cytometry on digitized microscopic images. Analytical Cellular Pathology, 25, 1–36. Google Scholar
- Samaria, F., & Harter, A. C. (1994). Parameterisation of a stochastic model for human face identification. In Proceedings of the second IEEE workshop on applications of computer vision. Google Scholar
- Hond, D., & Spacek, L. (1997). Distinctive descriptions for face processing. In Proceedings of 8th BMVC (pp. 320–329). Google Scholar
- Spacek, L. (2002), University of Essex face database. http://dces.essex.ac.uk/mv/allfaces/index.html.
- Tamura, H., Mori, S., & Yamavaki, T. (1978). Textural features corresponding to visual perception. IEEE Transactions on System, Man and Cybernetics, 8, 460–472. Google Scholar