Skip to main content
SpringerLink
Log in

Performance Analysis and Application of Expressiveness Detection on Facial Expression Videos Using Deep Learning Techniques

  • Original Article
  • Published:
Data-Enabled Discovery and Applications

Abstract

The emergence of various social media platforms has promoted a rapid growth in multimedia generation and proliferation. The interpretation of multimedia data pose a challenge for current computer systems and methodologies. The introduction of revolutionary and sophisticated methods such as Convoluted Neural Networks (CNN) and Long Short-Term Memory (LSTM) has improved the feasibility of extracting meaningful content from various data sources. The focus of this paper is to highlight how the abovementioned methods were utilized to determine the expressiveness of a subject’s response to a video commercial. A real-time expressiveness feedback solution is explored in this paper as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. D.J. McDuff, R. el Kaliouby, T. Senechal, M. Amr, J.F. Cohn, R.W. Picard, in Affectiva-MIT facial expression dataset (AM-FED): naturalistic and spontaneous facial expressions collected In-the-Wild. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , (2013)

  2. Facial action coding system, http://www.cs.cmu.edu/~face/facs.htm

  3. Philipp Michel, Rana el Kaliouby, in Real time facial expression recognition in video using support vector machines. Proceedings of the 5th International Conference on Multimodal Interfaces (Vancouver, Canada, 2003)

  4. B.E. Boser, I.M. Guyon, V.N. Vapnik, in A training algorithm for optimal margin classifiers. Proceedings of the fifth annual workshop on Computational learning theory, Pittsburgh, (1992), pp. 144–152

  5. H. Mobahi, R. Collobert, J. Weston, in Deep learning from temporal coherence in video. Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, (2009)

  6. Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, in Gradient-based learning applied to document recognition, Vol. 86, (1998), pp. 2278–2324

  7. A. Krizhevsky, I. Sutskever, G.E. Hinton, in ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (NIPS’2012), (2012)

  8. S.A. Nene, S.K. Nayar, H. Murase, Columbia object image library (COIL-100). Technical Report CUCS-006- 96, Columbia University. http://www.cs.columbia.edu/CAVE/research/softlib/coil-100.html. (1996)

  9. A. Rao, N. Thiagarajan, Recognizing facial expressions from videos using deep belief networks. Stanford CS 229 Machine Learning Final Projects, Technical Report (2010)

  10. J.Y.-H. Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, G. Toderici, Beyond short snippets: deep networks for video classification (2015)

  11. J. Donahue, L.A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, T. Darrell, in Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2015)

  12. Ffmpeg : Desciption, https://www.ffmpeg.org/about.html

  13. P.W.D. Charles, Project Title, GitHub repository, https://github.com/charlespwd/project-title (2013)

  14. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, Y.B. Theano, in A CPU and GPU math expression compiler. Proceedings of the Python for Scientific Computing Conference (SciPy). Oral, (2010)

  15. T. Kanungo, D.M. Mount, N.S. Netanyahu, C.D. Piatko, R. Silverman, A.Y. Wu, An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002)

    Article  Google Scholar 

  16. D. Müllner, Modern hierarchical, agglomerative clustering algorithms. arXiv:1109.2378 (2011)

  17. M. Abadi, et al., Tensorflow: large-scale machine learning on heterogeneous systems. arXiv:1603.04467 (2016)

  18. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  19. M.D. Zeiler, ADADELTA: An adaptive learning rate method. arXiv:1212.5701 (2012)

  20. S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  21. J. Martens, I. Sutskever, in Learning recurrent neural networks with hessian-free optimization. ICML, (2011)

  22. LSTM Network for Semantic Analysis : Description, http://deeplearning.net/tutorial/lstm.html

  23. J. Rafal, W. Zaremba, I. Sutskever, in An empirical exploration of recurrent network architectures. Proceedings of the 32nd International Conference on Machine Learning (ICML- 15), (2015)

Download references

Author information

Authors and Affiliations

  1. CBP Government Engineering College, New Delhi, India

    K. G. Srinivasa

  2. M S Ramaiah Institute of Technology, Bangalore, India

    Sriram Anupindi

Authors
  1. K. G. Srinivasa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sriram Anupindi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. G. Srinivasa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Srinivasa, K.G., Anupindi, S. Performance Analysis and Application of Expressiveness Detection on Facial Expression Videos Using Deep Learning Techniques. Data-Enabled Discov. Appl. 2, 9 (2018). https://doi.org/10.1007/s41688-018-0018-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41688-018-0018-1

Keywords

Search

Navigation