Soft Computing

, Volume 22, Issue 4, pp 1351–1360 | Cite as

Face image retrieval: super-resolution based on sketch-photo transformation

  • Shu ZhanEmail author
  • Jingjing Zhao
  • Yucheng Tang
  • Zhenzhu Xie
Methodologies and Application


Considering the crucial role of face image in modern intelligent system, face image retrieval has attracted more attention for authentication, surveillance, law enforcement, and security control. In most cases, we cannot obtain the suspect’s face image directly and the best substitute is a face sketch of criminal suspect drawn by artist according to eyewitness description. It is a key step in the criminal investigation process to narrow down criminal suspect using the face sketch. At first, the face sketch is transformed to a pseudo-photo for subsequent utilization. Transformation is performed according to the classic eigenface algorithm and enhanced by super-resolution. Matching between reconstructed pseudo-photo and real face photographs is performed by Hash encoding and iterative quantization. We carried out our ideas on two public face databases, and the sketch face images are generated by photo-shopping software program. The experimental results show that the corresponding face images can be retrieved according to the input face sketch and super-resolution can effectively enhance the image quality and detail information of the pseudo-photo. Hash encoding and iterative quantization achieve the quick search of approximate face images.


Face retrieval Super-resolution Iterative quantization Hash encoding Dictionary learning 



This work was supported by the National Natural Science Foundation of China under Grants 61371156, Anhui Province Science and Technology Research Programs under Grant 1401B042019. The authors would like to thank the anonymous reviewers for their helpful and constructive comments and suggestions.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Cao Y, Wang C, Zhang L (2011) Edgel index for large-scale sketch-based image search. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition (CVPR) Colorado Springs, CO, USA, 2011, pp 761–768Google Scholar
  2. Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2004:275–282Google Scholar
  3. Chen J, Ma H (2014) Privacy-preserving decentralized access control for cloud storage systems. In: The 2014 IEEE 7th international conference on cloud computingGoogle Scholar
  4. Chen X, Li J, Susilo W (2012) Efficient fair conditional payments for outsourcing computations. IEEE Trans Inf Forensics Secur 7(6):1687–1694CrossRefGoogle Scholar
  5. Chen B, Shu H, Coatrieux G, Chen G, Sun X, Coatrieux J (2015) Color image analysis by quaternion-type moments. J Math Imaging Vis 51(1):124–144MathSciNetCrossRefzbMATHGoogle Scholar
  6. Dong W, Zhang D, Shi G et al (2011) Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857Google Scholar
  7. Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658CrossRefGoogle Scholar
  8. Elad M, Feuer A (1999) Super-resolution reconstruction of image sequences. IEEE Trans Pattern Anal Mach Intell 21(9):817–834CrossRefGoogle Scholar
  9. Freeman T, Jones T, Pasztor EC (2002) Example based superresolution. IEEE Comput Graph Appl 22(2):56–65CrossRefGoogle Scholar
  10. Galea C, Farrugia R (2015) Fusion of intra- and inter-modality algorithms for face-sketch recognition. Comput Anal Images Patterns 9257:700–7110MathSciNetGoogle Scholar
  11. Galoogahi HK, Sim T (2012) Face sketch recognition by local radon binary pattern: Lrbp. In: The 19th IEEE international conference on image processing, 2012Google Scholar
  12. Gong Y, Lazebnik S (2011) Iterative quantization: a procrustean approach to learning binary codes. In: IEEE conference on computer vision and pattern recognition 2011Google Scholar
  13. Gruca A, Czachorski T, Kozielski S (2014) Advances in intelligent systems and computing. Springer International Publishing, BerlinGoogle Scholar
  14. Gu B, Sheng V, Tay K, Romano W, Li S (2015) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26(7):1403–1416MathSciNetCrossRefGoogle Scholar
  15. He L, Qi H, Zaretzki R (2013) Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2013, pp 345–352Google Scholar
  16. Hong C, Yu J, Tao D, Wang M (2015) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Ind Electron 62(6):3742–3751Google Scholar
  17. Irani M, Peleg S (1991) Improving resolution by image registration. Graph Models Image Process (CVGIP) 53(3):231–239CrossRefGoogle Scholar
  18. Jin Li, Kwangjo K (2010) Hidden attribute-based signatures without anonymity revocation. Inf Sci 180(9):1681–1689MathSciNetCrossRefzbMATHGoogle Scholar
  19. Jin L, Xiaofeng C, Mingqiang L (2014) Secure deduplication with efficient and reliable convergent key management. IEEE Trans Parallel Distrib Syst 25(6):1615–1625CrossRefGoogle Scholar
  20. Li W, Zhou Z (2015) Learning to hash for big data: current status and future trends (in Chinese). Chin Sci Bull 60:485490Google Scholar
  21. Li J, Wang Q, Wang C, Ren K (2009) Enhancing attribute-based encryption with attribute hierarchy. In: The fourth international conference on communications and networking in ChinaGoogle Scholar
  22. Li J, Zhao G, Chen X, Xie D, Rong C, Li W, Tang L, Tang Y (2010) Fine-grained data access control systems with user accountability in cloud computing. In: The 2010 IEEE second international conference on cloud computing technology and scienceGoogle Scholar
  23. Li J, Huang X, Li J, Chen X, Xiang Y (2014) Securely outsourcing attribute-based encryption with checkability. IEEE Trans Parallel Distrib Syst 25(8):2201–2210CrossRefGoogle Scholar
  24. Li J, Li X, Yang B, Sun X (2015) Segmentation-based image copy-move forgery detection scheme. IEEE Trans Inf Forensics Secur 10(3):507–518CrossRefGoogle Scholar
  25. Lin D, Tang X (2005) Coupled space learning of image style transformation. In: Proceedings of the 10th IEEE international conference on computer vision. Beijing: IEEE, 2005, pp 1699–1706Google Scholar
  26. Lin L, Zeng K, Wang Y, Xu YQ, Zhu SC (2013) Video stylization: painterly rendering and optimization with content extraction. IEEE Trans Circuits Syst Video Technol 23(4):577–590CrossRefGoogle Scholar
  27. Lin L, Xu Y, Liang X, Lai J (2014) Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 23(7):3191–3202MathSciNetCrossRefzbMATHGoogle Scholar
  28. Lin L, Wang X, Yang W, Lai JH (2015a) Discriminatively trained and-or graph models for object shape detection. IEEE Trans Pattern Anal Mach Intell 37(5):959–972CrossRefGoogle Scholar
  29. Lin L, Zhang R, Duan X (2015b) Adaptive scene category discovery with generative learning and compositional sampling. IEEE Trans Circuits Syst Video Technol 25(2):251–260CrossRefGoogle Scholar
  30. Liu X, Lin L, Jin H (2015) Contextualized trajectory parsing with spatio-temporal graph. In: IEEE transactions on pattern analysis and machine intelligence, PP (99) (2015) 1–1Google Scholar
  31. Lowe DG (1999) Object recognition from local scale-invariant features. Proceed Int Conf Comput Vis 2:11501157Google Scholar
  32. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  33. Matthew T, Alex P (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86CrossRefGoogle Scholar
  34. Mrozek D, Maysiak B, Kozielski S (2006) East: energy alignment search tool. Lect Notes Comput Sci 4223:696–705CrossRefGoogle Scholar
  35. Pan Z, Zhang Y, Kwong S (2015a) Efficient motion and disparity estimation optimization for low complexity multiview video coding. IEEE Trans Broadcast 61(2):166–176CrossRefGoogle Scholar
  36. Pan Z, Yu J, Xiao C, Sun W (2015b) Single image super resolution based on adaptive multi-dictionary learning (in Chinese). Acta Electron Sin 43:209–216Google Scholar
  37. Pawlowski R, Malysiak-Mrozek B, Kozielski S, Mrozek D (2011) Fast and accurate similarity searching of biopolymer sequences with GPU and CUDA. In: 11th International conference on algorithms and architectures for parallel processing (ICA3PP 2011)Google Scholar
  38. Silva MAA, Camara-Chavez G (2014) Face sketch recognition from local features. In: The 27th SIBGRAPI conference on graphics, 2014Google Scholar
  39. Smith LN, Elad M (2013) Improving dictionary learning: multiple dictionary updates and coefficient reuse. IEEE Sig Process Lett 20:79–82CrossRefGoogle Scholar
  40. Szanto B, Pozsegovics P, Vamossy Z, Sergyan S (2011) Sketch4match content-based image retrieval system using sketches. In: Proceedings of the 2011 IEEE 9th international symposium on applied machine intelligence and informatics (SAMI), Smolenice, IEEE, 2011, pp 183–188Google Scholar
  41. Tang X, Wang X (2002) Face photo recognition using sketch. In: Proceedings of the 2002 international conference on image processing. Rochester, NY: IEEE, 2002, pp 257–260Google Scholar
  42. Tang X, Wang X (2004) Face sketch recognition. IEEE Trans Circuits Syst Video Technol 14(1):50–57CrossRefGoogle Scholar
  43. Timofte R et al (2013) Anchored neighborhood regression for fastexample-based super-resolution. In: IEEE international conference oncomputer vision (ICCV), 2013, pp 1920–1927Google Scholar
  44. Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31(11):1955–1967CrossRefGoogle Scholar
  45. Wang Y, Yu M, He G, Jia Q (2011) Query by sketch: an asymmetric sketch-vs-image retrieval system. In: Proceedings of the 4th international congress on image and signal processing, Shanghai, China, 2011, pp 1368–1372Google Scholar
  46. Wang S, Zhang L, Y et al (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: Proceedings of the 25th IEEE conference on computer vision and pattern recognition. Providence, RI: IEEE, 2012, pp 2216–2223Google Scholar
  47. Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30CrossRefGoogle Scholar
  48. Xia Z, Wang X, Sun X, Liu Q, Xiong N (2016) Steganalysis of LSB matching using differences between nonadjacent pixels. Multimed Tools Appl 75(4):1947–1962CrossRefGoogle Scholar
  49. Yamasaki T, Aizawa K, Silva GC (2009) Sketch-based spatial queries for retrieving human locomotion patterns from continuously archived GPS data. IEEE Trans Multimed 11(7):1240–1253CrossRefGoogle Scholar
  50. Yang M, Wang YCF (2013) A self-learning approach to single image super-resolution. IEEE Trans Multimed 15(3):498–508Google Scholar
  51. Yang J, Wright J, Huang T (2010) Image super-resolution via sparse representation 19:2861–2873Google Scholar
  52. Yu J, Rui Y, Tang Y, Tao D (2014a) High-order distance based multiview stochastic learning in image classification. IEEE Trans Cybern 44(12):2431–2442CrossRefGoogle Scholar
  53. Yu J, Hong R, Wang M, You J (2014b) Image clustering based on sparse patch alignment framework. Pattern Recognit 47(11):3512–3519CrossRefzbMATHGoogle Scholar
  54. Yu J, Rui Y, Chen B (2014c) Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans Multimed 16(1):159–168CrossRefGoogle Scholar
  55. Yu J, Rui Y, Tao D (2014d) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23(5):2019–2032MathSciNetCrossRefzbMATHGoogle Scholar
  56. Yu J, Gao X et al (2014e) A unified learning framework for single image. IEEE Trans Neural Netw Learn Syst 25(4):780–792MathSciNetCrossRefGoogle Scholar
  57. Yu J, Tao D, Rui Y (2015) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779CrossRefGoogle Scholar
  58. Zeyde R, Elad M, Protter M et al (2012) On single image scale-up using sparse representations. In: Proceedings of the 7th international conference on curves and surfaces, Avignon, France, 2012, pp 711–730Google Scholar
  59. Zhang W, Wang X, Tang X (2011) Coupled information-theoretic encoding for face photo-sketch recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), 2011Google Scholar
  60. Zhou X, Cao L (2015) The sketch face recognition combining with adaboost and blocking lbp. J Image Graph 20(1):50–58Google Scholar
  61. Zhao J, Mao Y, Fang Q, Liang Z, Yang F, Zhan S (2015) Heterogeneous face recognition based on super resolution reconstruction by adaptive multi-dictionary learning, vol 9428. The 10th Chinese conference on biometric recognition (CCBR 2015), 2015, pp 143–150Google Scholar
  62. Zhao J, Fang Q, Liang Z, Hu C, Yang F, Zhan S (2016) Sketch face recognition based on super-resolution reconstruction. J Image Graphics 21(2):0218–0224Google Scholar
  63. Zheng Y, Jeon B, Xu D, Wu Q, Zhang H (2015) Image segmentation by generalized hierarchical fuzzy C-means algorithm. J Intell Fuzzy Syst 28(2):961–973Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Shu Zhan
    • 1
    Email author
  • Jingjing Zhao
    • 1
  • Yucheng Tang
    • 1
  • Zhenzhu Xie
    • 1
  1. 1.Hefei University of TechnologyHefeiChina

Personalised recommendations