Pattern Recognition and Image Analysis

, Volume 26, Issue 1, pp 165–175 | Cite as

New solutions for face photo retrieval based on sketches

  • G. A. KukharevEmail author
  • Yu. N. Matveev
  • N. L. Shchegoleva
Applied Problems


The problem of face photo retrieval using sketches constructed based on a description provided by a witness is discussed. The status of this problem from primary concepts and the used terminology, to modern technologies for constructing sketches, real scenarios and search results is reviewed. The development history of systems for constructing facial composites (identikits and sketches) and the ideas realized in these systems are provided. The task of automatically searching through a database of original photo images using a face sketch is discussed, and the reasons of low performance of such search in real-world scenarios are brought to light. Requirements to databases of sketches in addition to the existing benchmark face databases and also methods of creation of such databases are formulated. Within this framework the methods for generation a population of sketches from the initial sketch to improve the performance of sketch-based photo image retrieval systems are discussed. A method to increase the index of similarity in pairs sketch-photo based on computation of an average sketch from the generated population is provided. It is shown that such sketches are more similar to original photo images and their use in the discussed problem may lead to good results. But for all that, the created sketches meet the requirements of the truthful scenario as allow possibility of incomplete information in verbal descriptions. Results of experiments on CUHK Face Sketch and CUHK Face Sketch FERET databases and also open access sketches and corresponding photo images are discussed.


Photo-Sketch Retrieval Population of Sketches 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. Uhl and N. D. V. Lobo, “A framework for recognizing a facial image from a police sketch,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (San Francisco, June 18–20, 1996), pp. 586–593. DOI 10.1109/CVPR.1996.517132.Google Scholar
  2. 2.
    W. Konen, “Comparing facial line drawings with graylevel images: a case study on PHANTOMAS,” in Proc. Int. Conf. on Artificial Neural Networks (Bochum, July 16–19, 1996), pp. 727–734.Google Scholar
  3. 3.
    Identi-Kit, Identi-Kit Solutions. http://wwwidentikitnetGoogle Scholar
  4. 4.
    FACES 4.0. http://wwwiqbiometrixcomGoogle Scholar
  5. 5.
    C. A. Yuen Pong and C. H. Man, “Human face image searching system using sketch,” in Proc. Workshop on Machine Vision Applications (MVA2002) (Nara-ken New Public Hall, Nara, Dec. 11–13, 2002), pp. 500–503.Google Scholar
  6. 6.
    X. Tang and X. Wang, “Face photo-sketch synthesis and recognition,” in Proc. 9th IEEE Int. Conf. on Computer Vision (Nice, Oct. 13–16, 2003), Vol. 1, pp. 687–694.CrossRefGoogle Scholar
  7. 7.
    X. Wang and X. Tang, “Face photo-sketch synthesis and recognition,” IEEE Trans. PAMI 31 (11), 1955–1967 (2009).CrossRefGoogle Scholar
  8. 8.
    B. F. Klare, Li Zhifeng, and A. K. Jain, “Matching forensic sketches to mug shot photos,” IEEE Trans. PAMI 33 (3), 639–646 (2011).CrossRefGoogle Scholar
  9. 9.
    Hu Han, B. Klare, K. Bonnen, and A. K. Jain, “Matching composite sketches to face photos: a component-based approach,” IEEE Trans. Inf. Forensics Security 8 (3), 191–204 (2013).CrossRefGoogle Scholar
  10. 10.
    G. M. Davies and T. Valentine, “Facial composites: forensic utility and psychological research,” in Handbook of Eyewitness Psychology, Vol. 2: Memory for People (LEA, Mahwah, (2007), pp. 59–83.Google Scholar
  11. 11.
    S. Gibson, C. Solomon, and Bejarano Alvaro, “Synthesis of photographic quality facial composites using evolutionary algorithms,” in Proc. of the British Machine Vision Conf. (BMVA Press, University of East Anglia, Norwich, Sept. 9–11, 2003), pp. 221–230.Google Scholar
  12. 12.
    C. Â. Frowd, P. J. B. Hancock, and D. Carson, “EvoFIT: a holistic, evolutionary facial imaging technique for creating composites,” ACM Trans. Appl. Psychol. 1 (1), 19–39 (2004).CrossRefGoogle Scholar
  13. 13.
    B. George, S. J. Gibson, M. I. S. Maylin, and C. J. Solomon, “EFIT-V–interactive evolutionary strategy for the construction of photo-realistic facial composites,” in Proc. Conf. on Genetic and Evolutionary Computation (GECCO) (Atlanta, July 12–16, 2008), pp. 1485–1490.Google Scholar
  14. 14.
    C. D. Frowd, M. Pitchford, F. Skelton, and A. Petkovic, “Catching even more offenders with EvoFIT facial composites,” in Proc. IC EST-2012 (Lisbon, Sept. 5–7, 2012), pp. 20–26.Google Scholar
  15. 15.
    Student Sketch Database. http://mmlabiecuhkeduhk/ facesketchhtmlGoogle Scholar
  16. 16.
    Face Sketch FERET Database. http:// mmlabiecuhkeduhk/cufsf/Google Scholar
  17. 17.
    W. Zhang, X. Wang, and X. Tang, “Coupled informationtheoretic encoding for face photo-sketch recognition,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (Providence, RI, June 20–25, 2011), pp. 513–520. DOIGoogle Scholar
  18. 10.
    1109/CVPR.2011.5995324.Google Scholar
  19. 18.
    Li Xuewei and Cao Xiaochun, “A simple framework for face photo-sketch synthesis,” Math. Problems Eng. 2012, ID 910719 (2012). DOI:10.1155/2012/910719Google Scholar
  20. 19.
    H. K. Galoogahi and T. Sim, “Face photo retrieval by sketch example,” in Proc. Int. Conf. ACM Multimedia (Nara, Oct. 29–Nov. 02, 2012), pp. 949–952.CrossRefGoogle Scholar
  21. 20.
    A. Sharma and D. W. Jacobs, “Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch,” in Proc. 24th IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (Colorado Springs, CO, June 20–25, 2011), pp. 593–600.Google Scholar
  22. 21.
    Chang Liang, Zhou Mingquan, Han Yanjun, and Deng Xiaoming, “Face sketch synthesis via sparse representation,” in Proc. 20th Int. Conf. on Pattern Recognition (ICPR) (Istanbul, Aug. 23–26, 2010), pp. 2146–2149.Google Scholar
  23. 22.
    G. A. Kukharev, K. Buda, and N. L. Shchegoleva, “Methods of face photo-sketch comparison,” Pattern Recogn. Image Anal. 24 (1), 102–113 (2014). DOI: 10.1134/S1054661814010076.CrossRefGoogle Scholar
  24. 23.
    G. Kukharev, K. Buda, and N. Shchegoleva, “Sketch generation from photo to create test databases,” Przeglad Elektrotechniczny (Electr. Rev.) 9 (2), 97–100 (2014).Google Scholar
  25. 24.
    Yu Hongchuan and J. Zhang Jian, “Mean value coordinates-based caricature and expression synthesis,” Signal, Image Video Processing (SIViP) 7 (5), 899–910 (2013). DOI 10.1007/s11760-011-0279-8.CrossRefGoogle Scholar
  26. 25.
    Patent RU 2541132 C1, Byull. Izobret., No 4 (26.07.13).Google Scholar
  27. 26.
    Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE Signal Processing Lett. 9 (3), 81–84 (2002).CrossRefGoogle Scholar
  28. 27.
    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error measurement to structural similarity,” IEEE Trans. Image Processing 13 (1), 1–14 (2004).Google Scholar
  29. 28.
    S. Masoudnia and R. Ebrahimpour, “Mixture of experts: a literature survey,” Artificial Intellig. Rev. 42 (2), 275–293 (2014).CrossRefGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2016

Authors and Affiliations

  • G. A. Kukharev
    • 1
    Email author
  • Yu. N. Matveev
    • 2
  • N. L. Shchegoleva
    • 3
  1. 1.West Pomeranian University of TechnologySzczecinPoland
  2. 2.ITMO UniversitySt. PetersburgRussia
  3. 3.Saint Petersburg State Electrotechnical UniversitySt. PetersburgRussia

Personalised recommendations