Local appearance modeling for objects class recognition

Theoretical Advances


In this work, we propose a new formulation of the objects modeling combining geometry and appearance; it is useful for detection and recognition. The object local appearance location is referenced with respect to an invariant which is a geometric landmark. The appearance (shape and texture) is a combination of Harris–Laplace descriptor and local binary pattern (LBP), all being described by the invariant local appearance model (ILAM). We use an improved variant of LBP traits at regions located by Harris–Laplace detector to encode local appearance. We applied the model to describe and learn object appearances (e.g., faces) and to recognize them. Given the extracted visual traits from a test image, ILAM model is carried out to predict the most similar features to the facial appearance: first, by estimating the highest facial probability and then in terms of LBP histogram-based measure, by computing the texture similarity. Finally, by a geometric calculation the invariant allows to locate an appearance in the image. We evaluate the model by testing it on different face images databases. The experiments show that the model results in high accuracy of detection and provides an acceptable tolerance to the appearance variability.


Invariant descriptors Local binary patterns Features matching Probabilistic learning Appearance modeling Object class recognition Facial detection 


  1. 1.
    Burns J, Weiss R, Riseman E (1993) View variation of point set and line-segment features. PAMI 15(1):51–68CrossRefGoogle Scholar
  2. 2.
    Dorko G, Schmid C (2003) Selection of scale-invariant parts for object class recognition. In: ICCV, pp 634–640Google Scholar
  3. 3.
    Agarwal S, Awan A, Roth D (2004) Learning to detect objects in images via a sparse, part-based representation. PAMI 26(11):1475–1490CrossRefGoogle Scholar
  4. 4.
    Fei-Fei L, Fergus R, Perona P (2003) A Bayesian approach to unsupervised one-shot learning of object categories. ICCV. Nice, France, pp 1134–1141Google Scholar
  5. 5.
    Hadid A, Pietikäinen M, Ahonen T (2004) A discriminative feature space for detecting and recognizing faces. In: Computer vision and pattern recognition (CVPR), proceedings of the IEEE computer society conference, vol 2, pp 797–804Google Scholar
  6. 6.
    Ojala T, Pietikäinen M, Mäenpää T (2002) Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell (PAMI) 24:971–987CrossRefMATHGoogle Scholar
  7. 7.
    Taffar M, Miguet S (2017) Face class modeling based on local appearance for recognition. In: 6th international conference on pattern recognition applications and methods (ICPRAM’17), Porto, PortugalGoogle Scholar
  8. 8.
    Visual Object Classes database (2012) Pattern analysis, statistical modelling and computational learning (PASCAL) visual object classes challenge. http://host.robots.ox.ac.uk/pascal/VOC/voc2012/
  9. 9.
    Color FERET Face Database (2009). www.itl.nist.gov/iad/humanid/colorferet
  10. 10.
    CMU Face Group and Face Detection Project (2009) Frontal and profile face images databases. http://vasc.ri.cmu.edu/idb/html/face/
  11. 11.
    CMU-PIE Database, CMU Pose, illumination, and expression (PIE) database. http://www.ri.cmu.edu/projects/project_418.html
  12. 12.
    AT&T Database (1994) AT&T: the database of faces, Cambridge University, Computer Laboratory, Digital Technology Group. http://www.cl.cam.ac.uk/Research/DTG/attarchive:pub/data/att_faces.zip
  13. 13.
    Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2015) Object detectors emerge in deep scene CNNs. In: ICLRGoogle Scholar
  14. 14.
    Zhu Z, Luo P, Wang X, Tang X (2014) Multi-view perceptron: a deep model for learning face identity and view representations. In: NIPSGoogle Scholar
  15. 15.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning, nature, vol 521. Macmillan Publishers, London, pp 436–444Google Scholar
  16. 16.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. IJCV 60(2):91–110CrossRefGoogle Scholar
  17. 17.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of CVPR, vol 1, pp 886–893Google Scholar
  18. 18.
    Shen L, Bai L (2006) A review on Gabor wavelets for face recognition. Pattern Anal Appl 9(2–3):273–292MathSciNetCrossRefGoogle Scholar
  19. 19.
    Kumar N, Berg AC, Belhumeur PN, Nayar SK (2009) Attribute and simile classifiers for face verification. In: 12th IEEE conference on ICCV, pp 365–372Google Scholar
  20. 20.
    Huang GB, Lee H, Learned-Miller E (2012) Learning hierarchical representations for face verification with convolutional deep belief networks. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2518–2525Google Scholar
  21. 21.
    Sun Y, Wang X, Tang X (2013) Hybrid deep learning for face verification. In: ICCVGoogle Scholar
  22. 22.
    Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. In: IEEE conference on CVPR, vol 1, pp 539–546Google Scholar
  23. 23.
    Toews M, Arbel T (2006) Detection over viewpoint via the object class invariant. Proc Int Conf Pattern Recogn 1:765–768Google Scholar
  24. 24.
    Taffar M, Benmohammed M (2011) Generic face invariant model for face detection. In: Proceedings of the IP&C conference. Springer, New York, pp 39–45Google Scholar
  25. 25.
    Fergus R, Perona P, Zisserman A (2003) Object class recognition by unsupervised scale-invariant learning. In: CVPR, Madison, Wisconsin, pp 264–271Google Scholar
  26. 26.
    Lindeberg T (1998) Feature detection with automatic scale selection. Int J Comput Vis 30(2):79–116CrossRefGoogle Scholar
  27. 27.
    Mikolajczyk K, Schmid C (2004) Scale and affine invariant interest point detectors. IJCV 60(1):63–86CrossRefGoogle Scholar
  28. 28.
    Kadir T, Brady M (2001) Saliency, scale and image description. IJCV 45(2):83–105CrossRefMATHGoogle Scholar
  29. 29.
    Herbert B, Tinnr T, Gool LV (2006) SURF: speeded up robust features. In: ECCV, Springer LNCS, vol. 3951(1), pp 404–417Google Scholar
  30. 30.
    Heisele B, Poggio T, Pontil M (2000) Face detection in still gray images. Technical report 1687, Center for Biological and Computational Learning, MITGoogle Scholar
  31. 31.
    Yang M-H, Kriegman DJ, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell (PAMI) 24:34–58CrossRefGoogle Scholar
  32. 32.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the computer vision and pattern recognition (CVPR), pp 511–518Google Scholar
  33. 33.
    Taffar M, Miguet S, Benmohammed M (2012) Viewpoint invariant face detection, networked digital technologies, communications in computer and information science. Springer, New York, pp 90–402Google Scholar
  34. 34.
    Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3:71–86CrossRefGoogle Scholar
  35. 35.
    Etemad K, Chellappa R (1997) Discriminant analysis for recognition of human face images. J Opt Soc Am 14:1724–1733CrossRefGoogle Scholar
  36. 36.
    Phillips P, Moon H, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face recognition algorithms. IEEE Trans Pattern Anal Mach Intell (PAMI) 22:1090–1104CrossRefGoogle Scholar
  37. 37.
    Penev P, Atick J (1996) Local feature analysis: a general statistical theory for object representation. Netw Comput Neural Syst 7:477–500CrossRefMATHGoogle Scholar
  38. 38.
    Wiskott L, Fellous J-M, Kuiger N, Von der Malsburg C (1997) Face recognition by elastic bunch graph matching. IEEE Trans PAMI 19:775–779CrossRefGoogle Scholar
  39. 39.
    Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: Proceedings of the 8th European conference on computer vision (ECCV)Google Scholar
  40. 40.
    Hadid A, Pietikäinen M (2004) Selecting models from videos for appearance-based face recognition. In: Proceedings of the 17th international conference on pattern recognition (ICPR)Google Scholar
  41. 41.
    Pope AR, Lowe DG (2000) Probabilistic models of appearance for 3-D object recognition. IJCV 40(2):149–167CrossRefMATHGoogle Scholar
  42. 42.
    Bart E, Byvatov E, Ullman S (2004) View-invariant recognition using corresponding object fragments. In: ECCV, pp 152–165Google Scholar
  43. 43.
    Nanni L, Brahnam S, Lumini A (2012) Random interest regions for object recognition based on texture descriptors and bag of features. Expert Syst Appl 39:973–977CrossRefGoogle Scholar
  44. 44.
    Déniz O, Bueno G, Salido J, De la Torre F (2011) Face recognition using histograms of oriented gradients. Pattern Recognit Lett 32:1598–1603CrossRefGoogle Scholar
  45. 45.
    Yu J, Qin Z, Wan T, Zhang X (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364CrossRefGoogle Scholar
  46. 46.
    Pranam J, Geers G (2010) IFLT based real-time framework for image matching. In: 20th International conference on pattern recognition (ICPR), pp 2242–2245Google Scholar
  47. 47.
    Janney P, Yu Z (2007) Invariant features of local textures—a rotation invariant local texture descriptor. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–7Google Scholar
  48. 48.
    Sun Y, Wang X, Tang X (2013) Deep convolutional network cascade for facial point detection. In: CVPR, pp 3476–3483Google Scholar
  49. 49.
    Luo P, Wang X, Tang X (2012) Hierarchical face parsing via deep learning. In: Proceedings of the CVPRGoogle Scholar
  50. 50.
    Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2013) Decaf: a deep convolutional activation feature for generic visual recognition. arXivpreprint arXiv:1310.1531
  51. 51.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: NIPS, pp 1097–1105Google Scholar
  52. 52.
    Zhang N, Paluri M, Ranzato M, Darrell T, Bourdev L (2014) Panda: Pose aligned networks for deep attribute modeling, In: CVPRGoogle Scholar
  53. 53.
    Zhang N, Donahue J, Girshick R, Darrell T (2014) Part-based r-cnns for fine-grained category detection. In: ECCV, pp 834–849Google Scholar
  54. 54.
    Oyallon E, Mallat S, Sifre L (2013) Generic deep networks with wavelet scattering. arXiv preprint arXiv:1312.5940
  55. 55.
    Wu D, Wu J, Zeng R, Jiang L, Senhadji L, Shu H (2015) Kernel principal component analysis network for image classification. arXiv preprint arXiv:1512.06337
  56. 56.
    Yanga X, Liu W, Tao D, Cheng J (2017) Canonical correlation analysis networks for two-view image recognition. Inf Sci 385(C):338–352. doi: 10.1016/j.ins.2017.01.011 CrossRefGoogle Scholar
  57. 57.
    Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: Proceedings of the British machine vision conference (BMVC), SwanseaGoogle Scholar
  58. 58.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deep-face: closing the gap to human-level performance in face verification. In: Proceedings of CVPRGoogle Scholar
  59. 59.
    Huang GB, Ramesh M, Berg T, Miller EL (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49. University of Massachusetts, AmherstGoogle Scholar
  60. 60.
    Wolf L, Hassner Tal, Maoz I (2011) Face recognition in unconstrained videos with matched background similarity. In: Proceedings of the CVPRGoogle Scholar
  61. 61.
    Sun Y, Ding L, Wang X, Tang X (2015) Deepid3: Face recognition with very deep neural networks. In: CoRR. arXiv:abs/1502.00873
  62. 62.
    Chen D, Cao X, Wang L, Wen F, Sun J (2012) Bayesian face revisited: a joint formulation. In: Proceedings of the ECCV, pp 566–579Google Scholar
  63. 63.
    Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: NIPSGoogle Scholar
  64. 64.
    Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: Proceedings of the CVPRGoogle Scholar
  65. 65.
    Sun Y, Wang X, Tang X (2014) Deeply learned face representations are sparse, selective, and robust. In: CoRR. arXiv:abs/1412.1265
  66. 66.
    Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the CVPRGoogle Scholar
  67. 67.
    Mikolajczyk K, Schmid C (2002) An affine invariant interest point detector. In: Proceedings of the 7th European conference on computer vision, Copenhagen, Denmark, vol. I, pp 128–142Google Scholar
  68. 68.
    Roth D, Yang M, Ahuja N (2000) A snow-based face detector. In: Neural information processing systemsGoogle Scholar
  69. 69.
    Sung KK, Poggio T (1998) Example-based learning for view-based human face detection. IEEE Trans PAMI 20:39–51CrossRefGoogle Scholar
  70. 70.
    Schneiderman H, Kanade T (1998) Probabilistic modeling of local appearance and spatial relationships for object recognition. In: Proceedings of the computer vision and pattern recognition (CVPR), pp 45–51Google Scholar
  71. 71.
    Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell (PAMI) 20(1):23–38CrossRefGoogle Scholar
  72. 72.
    Osuna E, Freund R, Girosit F (1997) Training support vector machines: an application to face detection. In: Proceedings of the computer vision and pattern recognition (CVPR), pp 130–136Google Scholar
  73. 73.
    Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
  74. 74.
    Farfade SS, Saberian M, Li LJ (2015) Multi-view face detection using deep convolutional neural networks. In: Computer vision and pattern recognition. arXiv:1502.02766
  75. 75.
    Mikolajczyk K, Schmid C (2001) Indexing based on scale invariant interest points. In: Proceedings of the 8th international conference on computer vision, Vancouver, Canada, pp 525–531Google Scholar
  76. 76.
    Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc (Series B) 39(1):1–38MathSciNetMATHGoogle Scholar
  77. 77.
    Gupta MR, Chen Y (2010) Theory and use of the EM algorithm. Found Trends Signal Process 4(3):223–296CrossRefMATHGoogle Scholar
  78. 78.
    Liu C (2003) A bayesian discriminating features method for face detection. IEEE Trans Pattern Anal Mach Intell 25:725–740CrossRefGoogle Scholar
  79. 79.
    Elad M, HelOr Y, Keshet R (2002) Rejection based classifier for face detection. Pattern Recognit Lett 23:1459–1471CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag London Ltd. 2017

Authors and Affiliations

  1. 1.Computer Science DepartmentUniversity of JijelOuled Aissa, JijelAlgeria
  2. 2.LIRIS, Université de Lyon, UMR CNRS 5205, Université Lyon 2BronFrance

Personalised recommendations