Abstract
This paper proposes a new video-based face recognition method that uses video frames of a subject rotating his/her head. In the experiment discussed here, the manifolds of video frames embedded in a high-dimensional video space were extracted using neural network (NN)-based models. This increased the recognition rate in comparison with a simple NN architecture (from 72.9 to 81.3 %). These models were inspired by manifold interpretations of the brain’s visual perception. Next, the pose and person manifolds were separated using the neurons trained in the hidden or bottleneck layer of the network. Finally, the separated manifolds were used to synthesize face images at different angles from a single frontal image. Using video frames to extract these manifolds produces higher-quality images than using manifolds extracted from single images. This improvement in image quality was verified using the structural similarity index measure.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arandjelović O, Cipolla R (2013) Achieving robust face recognition from video by combining a weak photometric model and a learnt generic face invariant. Pattern Recogn 46(1):9–23
Mian A (2011) Online learning from local features for video-based face recognition. Pattern Recogn 44(5):1068–1075
Cui Z, Chang H, Shan S, Ma B, Chen X (2014) Joint sparse representation for video-based face recognition. Neurocomputing 135:306–312
Zhao W et al (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458
Satoh SI (2000) Comparative evaluation of face sequence matching for content-based video access. In: Proceedings of fourth IEEE international conference on automatic face and gesture recognition, 2000. IEEE
Belhumeur PN, Hespanha JP, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intel 19(7):711–720
Liu X, Cheng T (2003) Video-based face recognition using adaptive hidden markov models. In: Proceedings of 2003 IEEE computer society conference on computer vision and pattern recognition. IEEE
Liu X, Chen T (2003) Video-based face recognition using adaptive hidden markov models. In: Proceedings of 2003 IEEE computer society conference on computer vision and pattern recognition. IEEE
Hadid A, Pietikäinen M (2009) Manifold learning for video-to video face recognition. In: Biometric ID management and multimodal communication. Springer, Berlin. pp 9–16
Hamedani K, Seyyedsalehi SA (2012) Video-based face recognition using manifold learning by neural networks. In: 2012 20th Iranian conference on electrical engineering (ICEE). IEEE
Seung HS, Lee DD (2000) The manifold ways of perception. Science 290(5500):2268–2269
Hadid A, Pietikäinen M (2013) Demographic classification from face videos using manifold learning. Neurocomputing 100:197–205
Hira ZM, Trigeorgis G, Gillies DF (2014) An algorithm for finding biologically significant features in microarray data based on a priori manifold learning. PLoS ONE 9(3):e90562
Qiao Hong, Zhang Peng, Wang Di, Zhang Bo (2013) An explicit nonlinear mapping for manifold learning. IEEE Trans Cybern 43(1):51–63
Wang C, Song X (2014) Robust head pose estimation via supervised manifold learning. Neural Networks 53:15–25
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396
Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Saul LK, Roweis ST (2003) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4:119–155
Hou Chenping, Nie Feiping, Wang Hua, Yi Dongyun, Zhang Changshui (2014) Learning high-dimensional correspondence via manifold learning and local approximation. Neural Comput Appl 24(7–8):1555–1568
Wang L, Suter D (2007) Learning and matching of dynamic shape manifolds for human action recognition. IEEE Trans Image Process 16(6):1646–1661
Chen J et al (2007) Enhancing human face detection by resampling examples through manifolds. IEEE Trans Syst Man Cybern A Syst Humans 37(6):1017–1028
Wang Q, Xu G, Ai H (2003) Learning object intrinsic structure for robust visual tracking. In: Proceedings of 2003 IEEE computer society conference on computer vision and pattern recognition. IEEE
Du M, Sankaranarayanan AC, Chellappa R (2014) Robust face recognition from multi-view videos
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Sanderson C, Paliwal KK (2002) Polynomial features for robust face authentication. In: Proceedings of 2002 international conference on image processing. IEEE
Ren Y, Iftekharuddin KM, White WE (2009) Recurrent network-based face recognition using image sequences. In: Computational intelligence for multimedia signal and vision processing, 2009. CIMSVP'09. IEEE Symposium, pp 41–46
Seyyedsalehi SZ, Seyyedsalehi SA (2014) Simultaneous learning of nonlinear manifolds based on the bottleneck neural network. Neural Process Lett 40(2):191–209
Aghajanian J, Prince S (2009) Face pose estimation in uncontrolled environments. In: BMVC
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hamedani, K., Seyyedsalehi, S.A. & Ahamdi, R. Video-based face recognition and image synthesis from rotating head frames using nonlinear manifold learning by neural networks. Neural Comput & Applic 27, 1761–1769 (2016). https://doi.org/10.1007/s00521-015-1975-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-015-1975-z