Fusion of CNN- and COSFIRE-Based Features with Application to Gender Recognition from Face Images

  • Frans Simanjuntak
  • George AzzopardiEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 943)


Convolution neural networks (CNNs) have been demonstrated to be very effective in various computer vision tasks. The main strength of such networks is that features are learned from some training data. In cases where training data is not abundant, transfer learning can be used in order to adapt features that are pre-trained from other tasks. Similarly, the COSFIRE approach is also trainable as it configures filters to be selective for features selected from training data. In this study we propose a fusion method of these two approaches and evaluate their performance on the application of gender recognition from face images. In particular, we use the pre-trained VGGFace CNN, which when used as standalone, it achieved 97.45% on the GENDER-FERET data set. With one of the proposed fusion approaches the recognition rate on the same task is improved to 98.9%, that is reducing the error rate by more than 50%. Our experiments demonstrate that COSFIRE filters can provide complementary features to CNNs, which contribute to a better performance.


VGGFace COSFIRE Fusion Gender recognition 


  1. 1.
    Fukushima, K.: Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)CrossRefGoogle Scholar
  2. 2.
    Goodfellow, I., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (2015)Google Scholar
  3. 3.
    Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8(1), 98–113 (1997)CrossRefGoogle Scholar
  4. 4.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference (2015)Google Scholar
  5. 5.
    Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)CrossRefGoogle Scholar
  6. 6.
    Pinheiro, P., Collobert, R.: Recurrent convolutional neural networks for scene labeling. In: Xing, E.P., Jebara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning Research, PMLR, 22–24 June 2014, Beijing, China, vol. 32, pp. 82–90 (2014)Google Scholar
  7. 7.
    Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)CrossRefGoogle Scholar
  8. 8.
    Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., Chen, T.: Recent advances in convolutional neural networks. Pattern Recogn. 77(C), 354–377 (2018)CrossRefGoogle Scholar
  9. 9.
    Strigl, D., Kofler, K., Podlipnig, S.: Performance and scalability of GPU-based convolutional neural networks. In: 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing, pp. 317–324, February 2010Google Scholar
  10. 10.
    Uetz, R., Behnke, S.: Large-scale object recognition with CUDA-accelerated hierarchical neural networks. In: 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems, vol. 1, pp. 536–541, November 2009Google Scholar
  11. 11.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)Google Scholar
  12. 12.
    Yan, Z., Jagadeesh, V., DeCoste, D., Di, W., Piramuthu, R.: HD-CNN: hierarchical deep convolutional neural network for image classification. CoRR, abs/1410.0736 (2014)Google Scholar
  13. 13.
    Kim, H.-J., Lee, J.S., Yang, H.-S.: Human action recognition using a modified convolutional neural network. In: Proceedings of the 4th International Symposium on Neural Networks: Part II–Advances in Neural Networks, ISNN 2007, pp. 715–723. Springer, Heidelberg (2007)Google Scholar
  14. 14.
    Le, Q.V., Zou, W.Y., Yeung, S.Y., Ng, A.Y.: Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, pp. 3361–3368. IEEE Computer Society, Washington (2011)Google Scholar
  15. 15.
    Wang, P., Cao, Y., Shen, C., Liu, L., Shen, H.T.: Temporal pyramid pooling based convolutional neural networks for action recognition. CoRR, abs/1503.01224 (2015)Google Scholar
  16. 16.
    Ferrari, V., Marin-Jimenez, M., Zisserman, A.: Progressive search space reduction for human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  17. 17.
    Weiss, D.J., Sapp, B., Taskar, B.: Sidestepping intractable inference with structured ensemble cascades. In: NIPS, pp. 2415–2423. Curran Associates, Inc. (2010)Google Scholar
  18. 18.
    Toshev, A., Szegedy, C.: Deeppose: human pose estimation via deep neural networks. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 1653–1660. IEEE Computer Society, Washington (2014)Google Scholar
  19. 19.
    Guyon, I., Albrecht, P., Le Cun, Y., Denker, J., Hubbard, W.: Design of a neural network character recognizer for a touch terminal. Pattern Recogn. 24(2), 105–119 (1991)CrossRefGoogle Scholar
  20. 20.
    Zhu, R., Mao, X., Zhu, Q., Li, N., Yang, Y.: Text detection based on convolutional neural networks with spatial pyramid pooling. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1032–1036, September 2016Google Scholar
  21. 21.
    Bengio, Y., LeCun, Y., Henderson, D.: Globally trained handwritten word recognizer using spatial representation, convolutional neural networks, and hidden Markov models. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6, pp. 937–944. Morgan-Kaufmann (1994)Google Scholar
  22. 22.
    Yin, X., Yin, X., Huang, K., Hao, H.: Robust text detection in natural scene images. IEEE Trans. Pattern Anal. Mach. Intell. 36(5), 970–983 (2014)CrossRefGoogle Scholar
  23. 23.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  24. 24.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278–2324 (1998)CrossRefGoogle Scholar
  25. 25.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR (2014)Google Scholar
  26. 26.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR, abs/1512.03385 (2015)Google Scholar
  27. 27.
    Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., Yu, D.: Convolutional neural networks for speech recognition. IEEE/ACM Trans. Audio Speech Lang. Proc. 22(10), 1533–1545 (2014)CrossRefGoogle Scholar
  28. 28.
    Mao, Q., Dong, M., Huang, Z., Zhan, Y.: Learning salient features for speech emotion recognition using convolutional neural networks. IEEE Trans. Multimedia 16, 2203–2213 (2014)CrossRefGoogle Scholar
  29. 29.
    Santos, R.M., Matos, L.N., Macedo, H.T., Montalvão, J.: Speech recognition in noisy environments with convolutional neural networks. In: 2015 Brazilian Conference on Intelligent Systems (BRACIS), pp. 175–179, November 2015Google Scholar
  30. 30.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, 25–29 October 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1746–1751 (2014)Google Scholar
  31. 31.
    Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), June 2014, Baltimore, Maryland, pp. 655–665. Association for Computational Linguistics (2014)Google Scholar
  32. 32.
    Wang, P., Xu, J., Xu, B., Liu, C., Zhang, H., Wang, F., Hao, H.: Semantic clustering and convolutional neural network for short text categorization. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 352–357. Association for Computational Linguistics (2015)Google Scholar
  33. 33.
    Johnson, R., Zhang, T.: Effective use of word order for text categorization with convolutional neural networks. In: The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2015, 31 May–5 June 2015, Denver, Colorado, USA, pp. 103–112 (2015)Google Scholar
  34. 34.
    Wasenmüller, O., Stricker, D.: Comparison of kinect v1 and v2 depth images in terms of accuracy and precision, November 2016Google Scholar
  35. 35.
    Huang, J., Li, J., Gong, Y.: An analysis of convolutional neural networks for speech recognition. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4989–4993, April 2015Google Scholar
  36. 36.
    Gecer, B., Azzopardi, G., Petkov, N.: Color-blob-based COSFIRE filters for object recognition. Image Vis. Comput. 57(C), 165–174 (2017)CrossRefGoogle Scholar
  37. 37.
    Azzopardi, G., Petkov, N.: A shape descriptor based on trainable COSFIRE filters for the recognition of handwritten digits. In: Wilson, R., Hancock, E., Bors, A., Smith, W. (eds.) Computer Analysis of Images and Patterns, pp. 9–16. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  38. 38.
    Guo, J., Shi, C., Azzopardi, G., Petkov, N.: Recognition of architectural and electrical symbols by COSFIRE filters with inhibition. In: CAIP (2015)Google Scholar
  39. 39.
    Fernández-Robles, L., Azzopardi, G., Alegre, E., Petkov, N., Castejón-Limas, M.: Identification of milling inserts in situ based on a versatile machine vision system. J. Manuf. Syst. 45, 48–57 (2017)CrossRefGoogle Scholar
  40. 40.
    Azzopardi, G., Rodríguez-Sánchez, A., Piater, J., Petkov, N.: A push-pull CORF model of a simple cell with antiphase inhibition improves SNR and contour detection. PLOS One 9(7), 1–13 (2014)CrossRefGoogle Scholar
  41. 41.
    Strisciuglio, N., Petkov, N.: Delineation of line patterns in images using B-COSFIRE filters. In: 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI), pp. 1–6, July 2017Google Scholar
  42. 42.
    Strisciuglio, N., Azzopardi, G., Petkov, N.: Detection of curved lines with B-COSFIRE filters: a case study on crack delineation. CoRR, abs/1707.07747 (2017)Google Scholar
  43. 43.
    Azzopardi, G., Strisciuglio, N., Vento, M., Petkov, N.: Trainable COSFIRE filters for vessel delineation with application to retinal images. Med. Image Anal. 19(1), 46–57 (2015)CrossRefGoogle Scholar
  44. 44.
    Azzopardi, G., Greco, A., Vento, M.: Gender recognition from face images with trainable COSFIRE filters. In: 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 235–241, August 2016Google Scholar
  45. 45.
    Azzopardi, G., Greco, A., Saggese, A., Vento, M.: Fusion of domain-specific and trainable features for gender recognition from face images. IEEE Access 6, 24171–24183 (2018)CrossRefGoogle Scholar
  46. 46.
    Azzopardi, G., Petkov, N.: Trainable COSFIRE filters for keypoint detection and pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 490–503 (2013)CrossRefGoogle Scholar
  47. 47.
    Azzopardi, G., Petkov, N.: Ventral-stream-like shape representation: from pixel intensity values to trainable object-selective COSFIRE models. Front. Comput. Neurosci. 8, 80 (2014)Google Scholar
  48. 48.
    Liew, S.S., Khalil-Hani, M., Radzi, F., Bakhteri, R.: Gender classification: a convolutional neural network approach. Turkish J. Electr. Eng. Comput. Sci. 24, 1248–1264 (2016)CrossRefGoogle Scholar
  49. 49.
    Levi, G., Hassncer, T.: Age and gender classification using convolutional neural networks. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 34–42, June 2015Google Scholar
  50. 50.
    Dhomne, A., Kumar, R., Bhan, V.: Gender recognition through face using deep learning. Procedia Comput. Sci. 132, 2–10 (2018). International Conference on Computational Intelligence and Data ScienceGoogle Scholar
  51. 51.
    Narodytska, N., Kasiviswanathan, S.P.: Simple black-box adversarial perturbations for deep networks. CoRR, abs/1612.06299 (2016)Google Scholar
  52. 52.
    Moosavi-Dezfooli, S.-M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. CoRR, abs/1511.04599 (2015)Google Scholar
  53. 53.
    Tang, W., Li, B., Tan, S., Barni, M., Huang, J.: CNN based adversarial embedding with minimum alteration for image steganography. CoRR, abs/1803.09043 (2018)Google Scholar
  54. 54.
    Uricar, M., Franc, V., Hlavac, V.: Facial landmark tracking by tree-based deformable part model based detector. In: 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 963–970, December 2016Google Scholar
  55. 55.
  56. 56.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07-49, University of Massachusetts, Amherst, October 2007Google Scholar
  57. 57.
    Shan, C.: Learning local binary patterns for gender classification on real-world face images. Pattern Recogn. Lett. 33(4), 431–437 (2012). Intelligent Multimedia InteractivityCrossRefGoogle Scholar
  58. 58.
    Azzopardi, G., Greco, A., Vento, M.: Gender recognition from face images using a fusion of SVM classifiers. In: Campilho, A., Karray, F. (eds.) Image Analysis and Recognition, pp. 533–538. Springer, Cham (2016)Google Scholar
  59. 59.
    Tapia, J.E., Perez, C.A.: Gender classification based on fusion of different spatial scale features selected by mutual information from histogram of LBP, intensity, and shape. IEEE Trans. Inf. Forensics Secur. 8(3), 488–499 (2013)CrossRefGoogle Scholar
  60. 60.
    Dago-Casas, P., González-Jiménez, D., Yu, L.L., Alba-Castro, J.L.: Single- and cross- database benchmarks for gender classification under unconstrained settings. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2152–2159, November 2011Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of GroningenGroningenThe Netherlands

Personalised recommendations