Signal, Image and Video Processing

, Volume 9, Supplement 1, pp 295–303 | Cite as

Matching caricatures to photographs

  • Bahri Abaci
  • Tayfun Akgul
Original Paper


Facial caricatures are informative funny images that allow us to identify a subject even with a few lines and dots. Matching caricatures to photographs is a challenging cross-modal face matching problem. This paper addresses this problem by defining a set of qualitative face features both for caricatures and photographs where features are automatically extracted from photos and manually labeled in caricatures. Additionally we release a publicly available caricature-photograph database with 200 caricatures and corresponding photomates. In our experiments, we use genetic algorithms and logistic regression and achieve over \(\mathrm{30.0}\,\%\) recognition rate at 0.1 false accept rate.


Caricature recognition Facial attributes Genetic algorithms 



This work is supported by The Scientific and Technological Research Council of Turkey (TUBITAK), Project No: 112E142. The authors would like to thank Anil Jain, Brendon Klare, Serhat Bucak and Mehmet Kerem Turkcan for their interest, initial help and contribution.


  1. 1.
    Akgul, T.: Introducing the cartoonist. IEEE Antennas Propag. Mag. 49(3), 162–162 (2007)CrossRefGoogle Scholar
  2. 2.
    Akgul, T.: Can an algorithm recognize montage portraits as human faces?[in the spotlight]. IEEE Signal Process. Mag. 28(1), 150–158 (2011)CrossRefGoogle Scholar
  3. 3.
    Ayhan, O., Abaci, B., Akgul, T.: Improved active shape model for variable illumination conditions. In: IEEE 15th International Workshop on Multimedia Signal Processing (MMSP), pp. 322–327. (2013)Google Scholar
  4. 4.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.: Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)CrossRefGoogle Scholar
  5. 5.
    Bhatt, H.S., Bharadwaj, S., Singh, R., Vatsa, M.: Memetically optimized mcwld for matching sketches with digital face images. IEEE Trans. Inf. Forensics Secur. 7(5), 1522–1535 (2012)CrossRefGoogle Scholar
  6. 6.
    Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  7. 7.
    Fernández, A., Garca, R., Usamentiaga, R., Casado, R.: Glasses detection on real images based on robust alignment. Mach. Vis. Appl. 26(4), 519–531 (2015). doi: 10.1007/s00138-015-0674-1 CrossRefGoogle Scholar
  8. 8.
    Freedman, D.A.: Statistical Models: Theory and Practice. Cambridge University Press, New York (2009)CrossRefGoogle Scholar
  9. 9.
    Goldberg, D.E., Holland, J.H.: Genetic algorithms and machine learning. Mach. Learn. 3(2), 95–99 (1988)CrossRefGoogle Scholar
  10. 10.
    Grgic, M., Delac, K., Grgic, S.: Scface-surveillance cameras face database. Multimed. Tools Appl. 51(3), 863–879 (2011)CrossRefGoogle Scholar
  11. 11.
    Han, H., Klare, B.F., Bonnen, K., Jain, A.K.: Matching composite sketches to face photos: a component based approach. IEEE Trans. Inf. Forensics Secur. 8(1), 191–204 (2013)CrossRefGoogle Scholar
  12. 12.
    Hsu, R.-L., Jain, A.K.: Semantic face matching. In: IEEE International Conference on Multimedia and Expo, vol. 2, pp. 145–148. (2002)Google Scholar
  13. 13.
    Kabakcioglu, G.: In: Nasrettin Hoca’nin Torunlari: Karikaturcu Portreleri. (Pan Yayincilik, 2002). 975-17-3013-9Google Scholar
  14. 14.
    Klare, B.F., Li, Z., Jain, A.K.: Matching forensic sketches to mug shot photos. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 639–646 (2011)CrossRefGoogle Scholar
  15. 15.
    Klare, B.F., Bucak, S.S., Jain, A.K., Akgul, T.: Towards automated caricature recognition. In: 5th IAPR International Conference on Biometrics (ICB), pp. 139–146. (2012)Google Scholar
  16. 16.
    Klum, S., Han, H., Jain, A.K., Klare, B.F.: Sketch based face recognition: forensic vs. composite sketches. In: International Conference on Biometrics (ICB), pp. 1–8. (2013)Google Scholar
  17. 17.
    Krasnogor, N., Smith, J.: A tutorial for competent memetic algorithms: model, taxonomy, and design issues. IEEE Trans. Evol. Comput. 9(5), 474–488 (2005)CrossRefGoogle Scholar
  18. 18.
    Kumar, N., Berg, A., Belhumeur, P.N., Nayar, S.: Describable visual attributes for face verification and image search. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1962–1977 (2011)CrossRefGoogle Scholar
  19. 19.
    Le, T.H.N., Luu, K., Seshadri, K., Savvides, M.: Beard and mustache segmentation using sparse classifiers on self-quotient images. In: 19th IEEE International Conference on Image Processing (ICIP), pp. 165–168. (2012)Google Scholar
  20. 20.
    Lian, H.-C., Lu, B.-L.: Multi-view gender classification using local binary patterns and support vector machines. In: Wang, J., Yi, Z., Zurada, J., Lu, B.-L., Yin, H. (eds). Advances in Neural Networks-ISNN 2006, Lecture Notes in Computer Science, vol. 3972, pp. 202–209. Springer, Berlin. (2006). ISBN: 978-3-540-34437-7Google Scholar
  21. 21.
    Martinez, A.M.: The AR face database. CVC Technical Report 24 (1998)Google Scholar
  22. 22.
    Mauro, R., Kubovy, M.: Caricature and face recognition. Mem. Cognit. 20(4), 433–440 (1992)CrossRefGoogle Scholar
  23. 23.
    Milborrow, S., Morkel, J., Nicolls, F.: The muct landmarked face database. Pattern Recognit. Assoc. S. Afr. 201, 179–184 (2010)Google Scholar
  24. 24.
    Moghaddam, B., Pentland, A.: Probabilistic visual learning for object detection. In: Fifth International Conference on Computer Vision, IEEE, pp. 786–793. (1995)Google Scholar
  25. 25.
    Oral, T.: In: Yuz Yuze. Pan Yayincilik, Istanbul (2005). ISBN: 975-8434-82-9Google Scholar
  26. 26.
    Oral, T.: In: Memnu-Niyet-Sizler. Schneidertempel Sanat Merkezi, Istanbul (2013)Google Scholar
  27. 27.
    Ouyang, S., Hospedales, T., Song, Y-Z., Li, X.: Cross-modal face matching: beyond viewed sketches. In: Computer Vision – ACCV 2014, vol. 9004, pp. 210–225. Springer International Publishing, Cham (2015). ISBN: 978-3-319-16807-4Google Scholar
  28. 28.
    Poroy, S.: In: Semih Poroy Portreler. Karikatur Vakfi Yayinlari, Ankara (1998). ISBN: 975-17-3013-9Google Scholar
  29. 29.
    Roth, J., Liu, X.: On Hair Recognition in the wild by machine. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI). (2014)Google Scholar
  30. 30.
    Toseeb, U., Keeble, D.R., Bryant, E.J.: The significance of hair for face recognition. PloS One 7(3), 34144 (2012)CrossRefGoogle Scholar
  31. 31.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)CrossRefGoogle Scholar
  32. 32.
    Wang, D., Shan, S., Zeng, W., Zhang, H., Chen, X.: A novel two-tier Bayesian based method for hair segmentation. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 2401–2404. (2009)Google Scholar
  33. 33.
    Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009)CrossRefGoogle Scholar
  34. 34.
    Wu, C., Liu, C., Shum, H.-Y., Xy, Y.-Q., Zhang, Z.: Automatic eyeglasses removal from face images. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 322–336 (2004)CrossRefGoogle Scholar
  35. 35.
    Zhou, E., Fan, H., Cao, Z., Jiang, Y., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: 2013 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 386–391. (2013)Google Scholar

Copyright information

© Springer-Verlag London 2015

Authors and Affiliations

  1. 1.Department of Electronic and Communication EngineeringIstanbul Technical UniversityMaslakTurkey

Personalised recommendations