Recognition of Sketches in Photos

  • Bing Xiao
  • Xinbo Gao
  • Dacheng Tao
  • Xuelong Li
Part of the Studies in Computational Intelligence book series (SCI, volume 346)

Summary

Face recognition by sketches in photos makes an important complement to face photo recognition. It is challenging because sketches and photos have geometrical deformations and texture difference. Aiming to achieve better performance in mixture pattern recognition, we reduce difference between sketches and photos by synthesizing sketches from photos, and vice versa, and then transform the sketch-photo recognition to photo-photo/sketch-sketch recognition. Pseudo-sketch/pseudo-photo patches are synthesized with embedded hidden Markov model and integrated to derive pseudo-sketch/pseudo-photo. Experiments are carried out to demonstrate that the proposed methods are effective to produce pseudo-sketch/pseudo-photo with high quality and achieve promising recognition results.

Keywords

sketch-photo recognition pseudo-sketch pseudo-photo image quilting averaging overlapping areas 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bartlett, M., Movellan, J., Sejnowski, T.: Face recognition by independent component analysis. IEEE Trans. Neural Netw. 13(6), 1450–1464 (2002)CrossRefGoogle Scholar
  2. 2.
    Baum, L., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist. 41(1), 164–171 (1970)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Bilmes, J.: A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models, Technical Report, ICSI TR-97-021, International computer science institute, University of California, Berkeley, USA (1998)Google Scholar
  4. 4.
    Davies, G.M., Willie, P.V.D., Morrison, L.J.: Facial composite production: a comparison of mechanical and computer driven systems. J. Appl. Psychol. 85(1), 119–124 (2000)CrossRefGoogle Scholar
  5. 5.
    Dempster, A., Laird, N., Rubin, D.: Likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. B 39(1), 1–38 (1977)MATHMathSciNetGoogle Scholar
  6. 6.
    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proc. ACM Conference on Computer Graphics and Interactive Techniques, pp. 341–346 (2001)Google Scholar
  8. 8.
    Gao, X., Zhong, J., Tao, D., Li, X.: Local face sketch synthesis learning. Neurocomputing 71(10-12), 1921–1930 (2008)CrossRefGoogle Scholar
  9. 9.
    Gao, X., Zhong, J., Tian, C.: Face sketch synthesis algorithm based on machine learning. IEEE Trans. Circuits Syst. Video Technol. 18(4), 487–496 (2008)CrossRefGoogle Scholar
  10. 10.
    He, X., Cai, D., Niyogi, P.: Tensor subspace analysis. In: Proc. 19th Annual Conference on Neural Information Processing Systems (2005)Google Scholar
  11. 11.
    He, X., Niyogi, P.: Locality preserving projections. In: Proc. 17th Annual Conference on Neural Information Processing Systems, pp. 153–160 (2003)Google Scholar
  12. 12.
    He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.J.: Face recognition using laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 1–13 (2005)CrossRefGoogle Scholar
  13. 13.
    Hyvärinen, A., Oja, E.: Independent component analysis: algorithms and applications. Neural Networks 13(4-5), 411–430 (2000)CrossRefGoogle Scholar
  14. 14.
    Kim, K.I., Jung, K., Kim, H.J.: Face recognition using kernel principal component analysis. IEEE Signal Processing Letters 9(2), 40–42 (2002)CrossRefGoogle Scholar
  15. 15.
    Kuo, S., Agazzi, O.: Keyword spotting in poorly printed documents using pseudo 2-D hidden Markov models. IEEE Trans. Pattern Anal. Mach. Intell. 16(8), 842–848 (1994)CrossRefGoogle Scholar
  16. 16.
    Lathauwer, L.D., DeMoor, B., Vandewalle, J.: On the best rank-1 and rank-(r1,r2. rn) approximation of higher-order tensors. SIAM J. Matrix Anal. A. 21(4), 1324–1342 (2000)MATHGoogle Scholar
  17. 17.
    Li, Y., Savvides, M., Bhagavatula, V.: Illumination tolerant face recognition using a novel face from sketch synthesis approach and advanced correlation filters. In: Proc. IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing, pp. 57–360 (2006)Google Scholar
  18. 18.
    Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1005–1010 (2005)Google Scholar
  19. 19.
    Liu, W., Tang, X., Liu, J.: Bayesian tensor inference for sketch-based facial photo hallucination. In: Proc. Int’l Joint Conf. on Artificial Intelligence, pp. 2141–2146 (2007)Google Scholar
  20. 20.
    Nagai, T., Nguyen, T.: Appearance model based face-to-face transform. In: Proc. IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing, pp. 749–752 (2004)Google Scholar
  21. 21.
    Nefian, A.: A hidden markov model-based approach for face detection and recognition, Ph.D., Georgia Institute of Technology (1999)Google Scholar
  22. 22.
    Nefian, A., Hayes, M.H.: Face recognition using an embedded HMM. In: Proc. Int’l Conf. on Audio- and Video-based Biometric Person Authentication, pp. 19–24 (1999)Google Scholar
  23. 23.
    Nefian, A., Hayes III, M.H.: Maximum likelihood training of the embedded HMM for face detection and recognition. In: Proc. IEEE Int’l Conf. on Image Processing, pp. 33–36 (2000)Google Scholar
  24. 24.
    Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2(6), 559–572 (1901)Google Scholar
  25. 25.
    Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Hoffman, K.J.C., Marques, J., Worek, W.J.M.: Overview of the face recognition grand challenge. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 947–954 (2005)Google Scholar
  26. 26.
    Robert, G.U.J., de Van Lobo, N.: A framework for recognizing a facial image from a police sketch. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 586–593 (1996)Google Scholar
  27. 27.
    Robert, G.U.J., de Lobo, N., Van Kwon, Y.H.: Recognizing a facial image from a police sketch. In: Proc. 2nd IEEE workshop on applications of computer vision, pp. 129–137 (1994)Google Scholar
  28. 28.
    Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRefGoogle Scholar
  29. 29.
    Stone, M.: Cross-validatory choice and assessment of statistical predictions. J. Roy. Statist. Soc. B 36(2), 111–147 (1974)MATHGoogle Scholar
  30. 30.
    Sun, J., Tao, D., Papadimitriou, S., Yu, P.S., Faloutsos, C.: Incremental tensor analysis: theory and applications. ACM Trans. on Knowledge Discovery from Data 2(3), 1–37 (2008)CrossRefGoogle Scholar
  31. 31.
    Tang, X., Wang, X.: Face photo recognition using sketch. In: Proc. IEEE Int’l Conf. on Image Processing, pp. 257–260 (2002)Google Scholar
  32. 32.
    Tang, X., Wang, X.: Face sketch synthesis and recognition. In: Proc. IEEE Int’l Conf. on Computer Vision, pp. 687–694 (2003)Google Scholar
  33. 33.
    Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circuits Syst.Video Technol. 14(1), 50–57 (2004)CrossRefGoogle Scholar
  34. 34.
    Tao, D., Li, X., Wu, X., Maybank, S.: General tensor discriminant analysis and gabor features for gait recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(10), 1700–1715 (2007)CrossRefGoogle Scholar
  35. 35.
    Turk, M., Pentland, A.: Face recognition using eigenfaces. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 586–591 (1991)Google Scholar
  36. 36.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neurosci. 3(1), 71–86 (1991)CrossRefGoogle Scholar
  37. 37.
    Viterbi, A.: Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Inf. Theory 13(2), 260–269 (1967)CrossRefMATHGoogle Scholar
  38. 38.
    Wang, Z., Bovik, A.: A universal image quality index. Signal Processing Lett. 9(3), 81–84 (2002)CrossRefGoogle Scholar
  39. 39.
    Xiao, B., Gao, X., Tao, D., Li, X.: A new approach for face recognition by sketches in photos. Signal Process. 89(8), 1576–1588 (2009)CrossRefMATHGoogle Scholar
  40. 40.
    Xiao, B., Gao, X., Tao, D., Li, X., Li, J.: Photo-sketch synthesis and recognition based on subspace learning. Neurocomputing (2009) (submitted)Google Scholar
  41. 41.
    Zhao, W., Chellappa, R., Rosenfeld, A., Phillips, P.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)CrossRefGoogle Scholar
  42. 42.
    Zhong, J., Gao, X., Tian, C.: Face sketch synthesis using E-HMM and selective ensemble. In: Proc. IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing, pp. 485–488 (2007)Google Scholar
  43. 43.
    Tao, D., Li, X., Wu, X., Maybank, S.: Geometric mean for subspace selection. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 260–274 (2009)CrossRefGoogle Scholar
  44. 44.
    Tao, D., Li, X., Wu, X., Hu, X., Maybank, S.: Supervised tensor learning. Knowl. Inf. Syst. 13(1), 1–42 (2007)CrossRefGoogle Scholar
  45. 45.
    Tao, D., Li, X., Wu, X., Maybank, S.: Tensor rank one discriminant analysis – a convergent method for discriminative multilinear subspace selection. Neurocomputing 71(10-12), 1866–1882 (2008)CrossRefGoogle Scholar
  46. 46.
    Zhang, T., Tao, D., Yang, J.: Discriminative locality alignment. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 725–738. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  47. 47.
    Gao, X., Lu, W., Tao, D., Li, X.: Image quality assessment based on multiscale geometric analysis. IEEE Trans. Image Processing 18(7), 1608–1622 (2009)MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Bing Xiao
    • 1
  • Xinbo Gao
    • 1
  • Dacheng Tao
    • 2
  • Xuelong Li
    • 3
  1. 1.VIPS Lab, School of Electronic EngineeringXidian UniversityXi’anP.R. China
  2. 2.School of Computer EngineeringNanyang Technological UniversitySingaporeSingapore
  3. 3.State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision MechanicsChinese Academy of SciencesXi’anP.R. China

Personalised recommendations