Skip to main content
SpringerLink
Log in

WTAOF-ILPB Based Feature Learning and LFSSOA-RBFNN Based Classification for Facial Micro-Expression Recognition

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Micro-Expressions (ME) has turned out to be one amongst the most vital clues for lies, as well as numerous other applications. It is the brief Facial Expressions (FE) that the people express whilst endeavouring to conceal, hide, or repress their emotions. In a real environment, recognizing facial ME is really very hard than recognizing the general-FE, which shows rich emotions. Here, to attain the most accurate classification outcomes of Micro-Expression Recognition (MER), a combination of Feature Learning (FL) algorithms is proposed. The Video Frames (VF) is taken as the input for MER. First, the inputted-VF is preprocessed and then a Viola-Jones Algorithm is carried out to attain the face images of the VF. The SB-DCNN extracts the Facial Land-Marks as of the face images. Next, for performing the effectual FL, the combination of Weight matrix-based Temporal Accumulated Optical Flow and Improved LBP features (WTAOF-ILPB) are employed. Lastly, to recognize the ME of the input VF, the features as of SB-DCNN and WTAOF-ILPB are rendered to the optimal adaptation of Radial Basis Function Neural Network (RBFNN), i.e. Levy Flight based Shuffled Shepherd Optimization Algorithm (LFSSOA)–RBFNN (LFSSOA-RBFNN). The proposed method’s performance is analyzed by executing an experiment. The outcomes exhibited that the proposed LFSSOA-RBFNN trounces other prevailing algorithms with respect to some performance metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Weber, R., Li, J., Soladie, C., & Seguier, R. (2019). A survey on databases of facial macro expression and micro expression. Computer Vision, Imaging and Computer Graphics Theory and Applications, 997, 298–325.

    Article  Google Scholar 

  2. Sumi, K., & Ueda, T. (2016). Micro expression recognition for detecting human emotional changes. Lecture Notes in Computer Science, vol. 9733, pp. 60-70, 2016.

  3. Hurley, C. M., Anker, A. E., Frank, M. G., Matsumoto, D., & Hwang, H. C. (2014). Background factors predicting accuracy and improvement in micro expression recognition. Motivation and Emotion, 38(5), 700–714.

    Article  Google Scholar 

  4. Takalkar, M., Xu, M., Wu, Q., & Chaczko, Z. (2018). A survey facial micro expression recognition. Multimedia Tools and Applications, 77(15), 19301–19325.

    Article  Google Scholar 

  5. Sun, Z., Zheng-ping, Hu., Zhao, M., & Li, S. (2020). Multi-scale active patches fusion based on spatiotemporal LBP TOP for micro-expression recognition. Journal of Visual Communication and Image Representation, 71, 1–10.

    Article  Google Scholar 

  6. Zhang, S., Feng, B., Chen, Z., & Huang, X. (2017). Micro expression recognition by aggregating local spatio temporal patterns. Lecture Notes in Computer Science, vol. 10132, pp. 638-648.

  7. Goh, K. M., Ng, C. H., Lim, L. L., & Sheikh, U. U. (2020). Micro expression recognition an updated review of current trends challenges and solutions. The Visual Computer, 36(3), 445–468.

    Article  Google Scholar 

  8. Patel, D., Hong, X., & Zhao, G. (2016). Selective deep features for micro-expression recognition. In 2016 23rd international conference on pattern recognition (ICPR) IEEE, December 4–8, 2016, Cancún, México.

  9. Le, T. T. Q., Tran, T. K., & Rege, M. (2020). Dynamic image for micro expression recognition on region based framework. In: IEEE 21st international conference on information reuse and integration for data science (IRI), IEEE, pp. 11–13 Aug. 2020, Las Vegas, NV, USA.

  10. Pan, H., Xie, L., Lv, Z., Li, J., & Wang, Z. (2020). Hierarchical support vector machine for facial micro-expression recognition. Multimedia Tools and Applications, 79(41), 31451–31465.

    Article  Google Scholar 

  11. Verma, M., Vipparthi, S. K., Singh, G., & Murala, S. (2019). LEARNet dynamic imaging network for micro expression recognition. IEEE Transactions on Image Processing, 29, 1618–1627.

    Article  MathSciNet  MATH  Google Scholar 

  12. Takalkar, M. A., Xu, M., & Chaczko, Z. (2020). Manifold feature integration for micro-expression recognition. Multimedia Systems, 26(5), 535–551.

    Article  Google Scholar 

  13. Liong, S. T., See, J., Phan, R. C. W., Wong, K. S., & Tan, S. W. (2018). Hybrid facial regions extraction for micro expression recognition system. Journal of Signal Processing Systems, 90(4), 601–617.

    Article  Google Scholar 

  14. Qu, F., Wang, S. J., Yan, W. J., Li, H., Wu, S., & Fu, X. (2017). CAS (ME)2) a database for spontaneous macro expression and micro expression spotting and recognition. IEEE Transactions on Affective Computing, 9(4), 424–436.

    Article  Google Scholar 

  15. Choi, D. Y., & Song, B. C. (2020). Facial micro expression recognition using two-dimensional landmark feature maps. IEEE Access, 8, 121549–121563.

    Article  Google Scholar 

  16. Wang, Y., See, J., Oh, Y. H., Phan, R. C. W., Rahulamathavan, Y., Ling, H. C., Tan, S. W., & Li, X. (2017). Effective recognition of facial micro expressions with video motion magnification. Multimedia Tools and Applications, 76(20), 21665–21690.

    Article  Google Scholar 

  17. Oh, Y. H., See, J., Lengo, A. C., Phan, R. C. W., & Baskaran, V. M. (2018). A survey of automatic facial micro-expression analysis databases methods and challenges. Frontiers in Psychology, 9, 1–21.

    Article  Google Scholar 

  18. Wang, S.-J., Yan, W.-J., Li, X., Zhao, G., Zhou, C.-G., Xiaolan, Fu., Yang, M., & Tao, J. (2015). Micro expression recognition using color spaces. IEEE Transactions on Image Processing, 24(12), 6034–6047.

    Article  MathSciNet  MATH  Google Scholar 

  19. Liu, Y.-J., Zhang, J.-K., Yan, W.-J., Wang, S.-J., Zhao, G., & Xiaolan, Fu. (2015). A main directional mean optical flow feature for spontaneous micro expression recognition. IEEE Transactions on Affective Computing, 7(4), 299–310.

    Article  Google Scholar 

  20. Lai, Z., Chen, R., Jia, J., & Qian, Y. (2020). Real time micro expression recognition based on ResNet and atrous convolutions. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-020-01779-5

    Article  Google Scholar 

  21. Wang, C., Peng, M., Bi, T., & Chen, T. (2020). Micro attention for micro expression recognition. Neurocomputing, 410, 354–362.

    Article  Google Scholar 

  22. Adegun, I. P., & Vadapalli, H. B. (2020). Facial micro expression recognition a machine learning approach. Scientific African. https://doi.org/10.1016/j.sciaf.2020.e00465

    Article  Google Scholar 

  23. Wang, L., Xiao, H., Luo, S., Zhang, J., & Liu, X. (2019). A weighted feature extraction method based on temporal accumulation of optical flow for micro expression recognition. Signal Processing Image Communication, 78, 246–253.

    Article  Google Scholar 

  24. Gan, Y. S., Liong, S. T., Yau, W. C., Huang, Y. C., & Tan, L. K. (2019). Off-apexnet on micro expression recognition system. Signal Processing Image Communication, 74, 129–139.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, NIT Nagaland, Chumukedima, India

    Rudranath Banerjee & Shouvik Dey

  2. Computer Science and Engineering, CGEC, Cooch Behar, India

    Sourav De

Authors
  1. Rudranath Banerjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sourav De
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shouvik Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rudranath Banerjee.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Banerjee, R., De, S. & Dey, S. WTAOF-ILPB Based Feature Learning and LFSSOA-RBFNN Based Classification for Facial Micro-Expression Recognition. Wireless Pers Commun 127, 2285–2304 (2022). https://doi.org/10.1007/s11277-021-08794-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-08794-5

Keywords

Search

Navigation