Face Photo-Sketch Transformation and Population Generation

  • Georgy Kukharev
  • Andrei OleinikEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9972)


The problem of the automatic recognition and processing of face photos and sketches is of significant importance for law enforcement applications. A photo-sketch transformation is a convenient way to reduce dissimilarities between a face sketch and a photo before employing one of the existing face recognition techniques. In this paper, we propose to generate a photo (sketch) population instead of a single image. This approach improves face recognition rate and reduces the effect of different appearance variations. The proposed method is based on the modified version of the Eigenface/Eigensketch approach. We carried out the experiments on the photo-sketch transformation and population generation. The obtained results confirm the capability of the proposed technique to transform sketches of various styles to photos and introduce an inter-population diversity that is sufficient to cover various face appearance deviations.


Forensic sketch Face recognition Eigenface Eigensketch Face image population 



This work was partially financially supported by the Government of the Russian Federation, Grant 074-U01. The authors express their sincere appreciation to Yuri Matveev, Head of SIS Department for his critical remarks and advice that significantly improved this paper.


  1. 1.
    Klare, B.F., Li, Z., Jain, A.K.: Matching forensic sketches to mug shot photos. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 639–646 (2011)CrossRefGoogle Scholar
  2. 2.
    Zhang, Y., Ellyson, S., Zone, A., Gangam, P., Sullins, J., McCullough, C., Canavan, S., Yin, L.: Recognizing face sketches by a large number of human subjects: a perception-based study for facial distinctiveness. In: IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, pp. 707–712. IEEE (2011)Google Scholar
  3. 3.
    Kukharev, G., Matveev, Y., Shchegoleva, N.: New solutions for face photo retrieval based on sketches. Pattern Recogn. Image Anal. 26(1), 165–175 (2016)CrossRefGoogle Scholar
  4. 4.
    Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 50–57 (2004)CrossRefGoogle Scholar
  5. 5.
    Shchegoleva, N., Kukharev, G.: Application of two-dimensional principal component analysis for recognition of face images. Pattern Recogn. Image Anal. 20(4), 513–527 (2010)CrossRefGoogle Scholar
  6. 6.
    Chen, H., Liu, Z., Rose, C., Xu, Y., Shum, H.Y., Salesin, D.: Example-based composite sketching of human portraits. In: Proceedings of the 3rd International Symposium on Non-Photorealistic Animation and Rendering, pp. 95–153. ACM (2004)Google Scholar
  7. 7.
    Yu, H., Zhang, J.J.: Mean value coordinates-based caricature and expression synthesis. Sig. Image Video Process. 7(5), 899–910 (2013)CrossRefGoogle Scholar
  8. 8.
    Gao, X., Zhong, J., Li, J., Tian, C.: Face sketch synthesis algorithm based on E-HMM and selective ensemble. IEEE Trans. Circuits Syst. Video Technol. 18(4), 487–496 (2008)CrossRefGoogle Scholar
  9. 9.
    Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009)CrossRefGoogle Scholar
  10. 10.
    Zhang, W., Wang, X., Tang, X.: Lighting and pose robust face sketch synthesis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 420–433. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15567-3_31 CrossRefGoogle Scholar
  11. 11.
    Kirby, M., Sirovich, L.: Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Trans. Pattern Anal. Mach. Intell. 12(1), 103–108 (1990)CrossRefGoogle Scholar
  12. 12.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)CrossRefGoogle Scholar
  13. 13.
    Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 1005–1010. IEEE (2005)Google Scholar
  14. 14.
    Li, Y.H., Savvides, M.: Faces from sketches: a subspace synthesis approach. Proc. SPIE 6202, 62020K (2006)CrossRefGoogle Scholar
  15. 15.
    Sharma, A., Jacobs, D.W.: Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 593–600. IEEE (2011)Google Scholar
  16. 16.
    Ouyang, S., Hospedales, T., Song, Y.-Z., Li, X.: Cross-modal face matching: beyond viewed sketches. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 210–225. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-16808-1_15 Google Scholar
  17. 17.
  18. 18.
  19. 19.
    Guo, G.: Heterogeneous face recognition: an emerging topic in biometrics. Intel Technol. J. 18(4), 80–97 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Saint Petersburg Electrotechnical University “LETI”St. PetersburgRussia
  2. 2.ITMO UniversitySt. PetersburgRussia

Personalised recommendations