Abstract
Facial recognition has now played a pivotal role in many applications, including biomechanics, sports, image segment, animation, and robotics, etc. Although commercial facial recognition is matured, micro-expression recognition is still in its infancy and has attracted more attention from researchers in recent years. Usually, test and training samples can be recorded by different equipment throughout a variety of conditions, or by heterologous species. As a result, it is necessary to investigate whether the common micro-expression recognition algorithm is still feasible when the test samples are different from the training samples. In the present study, a series of well-developed algorithms for multi-source domain adaptation, the basic principles of multi-source domain adaptation, and the feature representation method has been discussed. A new method called the novel super-wide regression network (SWiRN) model has been introduced. Finally, some loss functions that are commonly used in neural networks for multiple source domain adaptations have been presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Borgwardt KM, Gretton A, Rasch MJ (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14):49–57
Chattopadhyay R, Sun Q, Fan W, Davidson I, Panchanathan S, Ye J (2012) Multisource domain adaptation and its application to early detection of fatigue. ACM Trans Knowl Discov Data 6(4):1–26
Choudhury T, Clarkson B, Jebara T, Pentland A (1999) Multimodal person recognition using unconstrained audio and video. In: Proceedings, International Conference on audio-and video-based person authentication, pp 176–181.
Davison AK, Yap MH, Lansley C (2015) Micro-facial movement detection using individualised baselines and histogram-based descriptors. In 2015 IEEE international conference on systems, man, and cybernetics, pp 1864–1869
Davison AK, Lansley C, Costen N, Tan K, Yap MH (2016) Samm: a spontaneous micro-facial movement dataset. IEEE Trans Affect Comput 9(1):116–129
Duan L, Tsang IW, Xu D, Chua TS (2009) Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th Annual International Conference on machine learning, pp 289–296
Duan L, Xu D, Tsang IW (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23(3):504–518
Duan L, Xu D, Chang SF (2012) Exploiting web images for event recognition in consumer videos: A multiple source domain adaptation approach. In: Computer vision and pattern recognition, pp 1338–1345
Ekman P, Rosenberg EL (1997) What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS)
Ekman P, Friesen WV, Hager JC (2002) Facial action coding system: the manual on CD ROM. A Human Face, Salt Lake City, pp 77–254
Gao J, Fan W, Jiang J, Han J (2008) Knowledge transfer via multiple model local structure mapping. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 283–291
Hong XD, Zheng X, Xia JY, Wei L, Xue W (2019) Cross-lingual non-ferrous metals related news recognition method based on CNN with a limited bi-lingual dictionary. Comput Mater Contin 58(2):379–389
Huang J, Gretton A, Borgwardt K, Schölkopf B, Smola AJ (2007) Correcting sample selection bias by unlabeled data. In: Advances in neural information processing systems, pp 601–608
Kan M, Wu J, Shan S (2014) Domain adaptation for recognition: targetize source domain bridged by common subspace. Int J Comput Vis 109(1–2):94–109
Kim DH, Baddar WJ, Ro YM (2016) Micro-expression recognition with expression-state constrained spatio-temporal feature representations. In: Proceedings of the 24th ACM International Conference on multimedia, pp 382–386
Lades M, Vorbruggen JC, Buhmann J, Lange J, Von Der Malsburg C, Wurtz RP, Konen W (1993) Distortion invariant object recognition in the dynamic link architecture. IEEE Trans Comput 42(3):300–311
Li X, Pfister T, Huang X, Zhao G, Pietikainen M (2013) A spontaneous micro-expression database: inducement, collection and baseline. In: IEEE Computer Society, pp 1–6
Li X, Pfister T, Huang X, Zhao G, Pietikäinen M (2013) A spontaneous micro-expression database: Inducement, collection and baseline. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp 1–6
Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210
Pfister T, Li X, Zhao G, Matti Pietikäinen (2011) Recognising spontaneous facial micro-expressions. In: IEEE international conference on computer vision, pp 1449–1456
Schweikert G, Widmer C, B Schölkopf, G Rätsch (2009) An empirical analysis of domain adaptation algorithms. In: Advances in neural information processing systems, pp 1433–1440
Steffens J, Elagin E, Neven H (1998) Personspotter-fast and robust system for human detection, tracking and recognition. In: Proceedings Third IEEE International Conference on automatic face and gesture recognition, pp 516–521.
Sun S, Shi H (2013) Bayesian multi-source domain adaptation. Proc Intl Conf Mach Learn Cybern 1:24–28
Sun Q, Chattopadhyay R, Panchanathan S, Ye J (2012) A two-stage weighting framework for multi-source domain adaptation. In: Advances in neural information processing systems, pp 505–513
Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92
Sun B, Cao S, Li D (2020) dynamic micro-expression recognition using knowledge distillation. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2020.2986962
Tu W, Sun S (2012) Cross-domain representation-learning framework with combination of class-separate and domain-merge objectives. In: International Workshop on cross domain knowledge discovery in web & social network mining, pp 18–25
Wang Y, Chua CS, Ho YK (2002) Facial feature detection and face recognition from 2D and 3D images. Pattern Recogn Lett 23(10):1191–1202
Wang MS, Niu SZ, Gao ZG (2019) A novel scene text recognition method based on deep learning. Comput Mater Contin 60(2):781–794
Wiskott L, Krüger N, Kuiger N, Von Der Malsburg C (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19(7):775–779
Wright J, Ganesh A, Rao S, Ma Y (2009) Robust principal component analysis: exact recovery of corrupted low-rank matrices. In: Advances in neural information processing systems, pp 2080–2088
Xu Z, Sun S (2012) Multi-source transfer learning with multi-view adaboost. In: International conference on neural information processing, pp 332–339
Yan WJ, Li X, Wang SJ, Zhao G, Liu YJ, Chen YH, Fu X (2014) CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE 9(1):e86041
Yan C, Gong B, Wei Y, Gao Y (2020) Deep multi-view enhancement hashing for image retrieval. IEEE Trans Pattern Anal Mach Intell
Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive SVMs. In: Proceedings of the 15th International Conference on MULTIMEDIA, pp 188–197
Yap MH, See J, Hong X, Wang SJ (2018) Facial micro-expressions grand challenge 2018 summary. In: IEEE computer society, pp 675–678
Zhang Z, Saligrama V (2016) Zero-shot learning via joint latent similarity embedding. In: proceedings of the IEEE Conference on computer vision and pattern recognition, pp 6034–6042
Zhang T, Zong Y, Zheng W (2020) Cross-database micro-expression recognition: a benchmark. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2020.2985365
Zhao WY, Chellappa R, Phillips PJ, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458
Zhao G, Wang SJ, Yan WJ, Fu X, Zhou CG (2015) Micro-expression recognition using robust principal component analysis and local spatiotemporal directional features. In: European Conference on computer vision, pp 325–338
Zhao Y, Yue JJ, Song W, Xu XN, Li XL, Wu LC, Ji Q (2019) Tibetan multi-dialect speech and dialect identity recognition. Comput Mater Contin 60(3):1223–1235
Zheng W, Zong Y, Zhou X, Xin M (2016) Cross-domain color facial expression recognition using transductive transfer subspace learning. IEEE Trans Affect Comput 9(1):21–37
Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O (2009) Cross domain distribution adaptation via kernel mapping. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 1027–1036
Acknowledgements
This work was supported by the National Nature Science Foundation of China under grant numbers No. 61502240, No. 61502096, No. 61304205, No. 61773219, the Natural Science Foundation of Jiangsu Province under grant numbers No. BK20191401 and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, X., Xu, T., Sun, W. et al. Multiple source domain adaptation in micro-expression recognition. J Ambient Intell Human Comput 12, 8371–8386 (2021). https://doi.org/10.1007/s12652-020-02569-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-020-02569-9