Skip to main content
SpringerLink
Log in

Fusion of Texture and Optical Flow Using Convolutional Neural Networks for Gender Classification in Videos

  • Conference paper
  • First Online:
Intelligent Technologies and Applications (INTAP 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1382))

Included in the following conference series:

  • 542 Accesses

Abstract

Automatic Gender Classification (AGC) is an essential problem due to its growing demand in commercial applications, including social media and security environments such as the airport. AGC is a well-researched topic both in the field of Computer Vision and Biometrics. In this paper, we propose the use of decision-level fusion for AGC in videos. Our approach does a decision-level fusion of labels obtained from two fine-tuned deep-networks based on a color image and optical-flow image, respectively, based on Resnet-18 architecture. We compare our proposed method with handcrafted features, which includes the concatenation of the Histogram of Optical Flow (HOF) and Histogram of Oriented Gradients (HOG). We compare it with deep-networks, which includes pre-trained & fine-tuned Resnet-18 based on a color image, and pre-trained & fine-tuned Resnet-18 based on optical flow image. Our fusion-based approach considerably outperforms both the handcrafted features, and the deep-networks previously mentioned. Another advantage of our proposed method is that it can work when the visual features are hidden. We used 98 videos from the HMDB51 action recognition dataset, specifically from the cart-wheel action with an almost 50% training, and testing split without validation set. We achieve an overall accuracy of 79.59% with Resnet-18 network architecture with the proposed method, compared to fine-tuned single-stream Resnet-18 for Color-Stream at 65.30%, and Optical-Flow at 55.10% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bagher Oskuie, F., Faez, K.: Gender classification using a novel gait template: radon transform of mean gait energy image. In: Kamel, M., Campilho, A. (eds.) Image Analysis and Recognition, pp. 161–169. Springer, Berlin Heidelberg, Berlin, Heidelberg (2011)

    Chapter  Google Scholar 

  2. Bourdev, L., Maji, S., Malik, J.: Describing people: a poselet-based approach to attribute classification. In: 2011 International Conference on Computer Vision, pp. 1543–1550. IEEE (2011)

    Google Scholar 

  3. Chen, J., Liu, S., Chen, Z.: Gender classification in live videos. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1602–1606. IEEE (2017)

    Google Scholar 

  4. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893. IEEE (2005)

    Google Scholar 

  5. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 428–441. Springer, Heidelberg (2006). https://doi.org/10.1007/11744047_33

    Chapter  Google Scholar 

  6. Dantcheva, A., Elia, P., Ross, A.: What else does your biometric data reveal? a survey on soft biometrics. IEEE Trans. Inf. Forensics Secur. 11(3), 441–467 (2015)

    Article  Google Scholar 

  7. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  8. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997). https://doi.org/10.1006/jcss.1997.1504. http://www.sciencedirect.com/science/article/pii/S002200009791504X

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  11. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49, University of Massachusetts, Amherst, October 2007

    Google Scholar 

  12. Huh, M., Agrawal, P., Efros, A.A.: What makes imagenet good for transfer learning? arXiv preprint arXiv:1608.08614 (2016)

  13. Kannala, J., Rahtu, E.: BSIF: binarized statistical image features. In: Proceeding of 21st International Conference on Pattern Recognition (ICPR 2012), Tsukuba, Japan, pp. 1363–1366 (2012)

    Google Scholar 

  14. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

  15. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)

    Google Scholar 

  16. Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 34–42 (2015)

    Google Scholar 

  17. Liew, S.S., Hani, M.K., Radzi, S.A., Bakhteri, R.: Gender classification: a convolutional neural network approach. Turkish J. Electr. Eng. Comput. Sci. 24(3), 1248–1264 (2016)

    Article  Google Scholar 

  18. Liu, P., Lyu, M., King, I., Xu, J.: Selflow: self-supervised learning of optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4571–4580 (2019)

    Google Scholar 

  19. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  20. Makinen, E., Raisamo, R.: Evaluation of gender classification methods with automatically detected and aligned faces. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 541–547 (2008)

    Article  Google Scholar 

  21. Moghaddam, B., Yang, M.H.: Gender classification with support vector machines. In: Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), pp. 306–311. IEEE (2000)

    Google Scholar 

  22. Ng, C.B., Tay, Y.H., Goi, B.-M.: Recognizing human gender in computer vision: a survey. In: Anthony, P., Ishizuka, M., Lukose, D. (eds.) PRICAI 2012. LNCS (LNAI), vol. 7458, pp. 335–346. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32695-0_31

    Chapter  Google Scholar 

  23. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002). https://doi.org/10.1109/TPAMI.2002.1017623

    Article  MATH  Google Scholar 

  24. Rai, P., Khanna, P.: Gender classification techniques: a review. In: Wyld, D.C., Zizka, J., Nagamalai, D. (eds.) Advances in Computer Science, Engineering & Applications, pp. 51–59. Springer, Berlin Heidelberg (2012)

    Chapter  Google Scholar 

  25. Ramachandra, R., Busch, C.: Presentation attack detection methods for face recognition systems: a comprehensive survey. ACM Comput. Surv. (CSUR) 50(1), 1–37 (2017)

    Article  Google Scholar 

  26. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems, pp. 568–576 (2014)

    Google Scholar 

  27. Singh, A., Kumar, A., Jain, A.: Bayesian gait-based gender identification (BGGI) network on individuals wearing loosely fitted clothing. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)

    Google Scholar 

  28. Sun, N., Zheng, W., Sun, C., Zou, C., Zhao, L.: Gender classification based on boosting local binary pattern. In: Wang, J., Yi, Z., Zurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3972, pp. 194–201. Springer, Heidelberg (2006). https://doi.org/10.1007/11760023_29

    Chapter  Google Scholar 

  29. Szegedy, C., et al.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition (CVPR) (2015). http://arxiv.org/abs/1409.4842

  30. Tammina, S.: Transfer learning using VGG-16 with deep convolutional neural network for classifying images. Int. J. Sci. Res. Publ. (IJSRP) 9, p9420, October 2019. https://doi.org/10.29322/IJSRP.9.10.2019.p9420

  31. Sun, Z., Yuan, X., Bebis, G., Louis, S.J.: Neural-network-based gender classification using genetic search for eigen-feature selection. In: Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN 2002 (Cat. No. 02CH37290), vol. 3, pp. 2433–2438 (2002)

    Google Scholar 

  32. Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H.: Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1 - Volume 01, pp. 786–791. ICCV 2005. IEEE Computer Society, USA (2005). DOIurl10.1109/ICCV.2005.147. https://doi.org/10.1109/ICCV.2005.147

  33. Zhang, Z., Hu, M., Wang, Y.: A survey of advances in biometric gait recognition. In: Sun, Z., Lai, J., Chen, X., Tan, T. (eds.) Biometric Recognition, pp. 150–158. Springer, Berlin Heidelberg, Berlin, Heidelberg (2011)

    Chapter  Google Scholar 

  34. Zheng, J., Lu, B.L.: A support vector machine classifier with automatic confidence and its application to gender classification. Neurocomputer 74(11), 1926–1935, May 2011. https://doi.org/10.1016/j.neucom.2010.07.032. https://doi.org/10.1016/j.neucom.2010.07.032

Download references

Author information

Authors and Affiliations

  1. Norwegian Biometrics Laboratory, Norwegian University of Science and Technology (NTNU), Trondheim, Norway

    Jag Mohan Singh, Raghavendra Ramachandra & Patrick Bours

Authors
  1. Jag Mohan Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raghavendra Ramachandra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Bours
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jag Mohan Singh .

Editor information

Editors and Affiliations

  1. NTNU, Gjøvik, Norway

    Sule Yildirim Yayilgan

  2. The Islamia University of Bahawalpur, Punjab, Pakistan

    Imran Sarwar Bajwa

  3. University of Agder, Kristiansand, Norway

    Filippo Sanfilippo

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Singh, J.M., Ramachandra, R., Bours, P. (2021). Fusion of Texture and Optical Flow Using Convolutional Neural Networks for Gender Classification in Videos. In: Yildirim Yayilgan, S., Bajwa, I.S., Sanfilippo, F. (eds) Intelligent Technologies and Applications. INTAP 2020. Communications in Computer and Information Science, vol 1382. Springer, Cham. https://doi.org/10.1007/978-3-030-71711-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-71711-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-71710-0

  • Online ISBN: 978-3-030-71711-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation