Skip to main content
SpringerLink
Log in

A hierarchical reasoning graph neural network for the automatic scoring of answer transcriptions in video job interviews

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

We address the task of automatically scoring the competency of candidates based on textual features, from the automatic speech recognition transcriptions in the asynchronous video job interviews. The key challenge is to construct the dependency relations and semantic level interaction over each question–answer (QA) pair. However, most recent studies focus on the representation of questions and answers, but ignore the dependency information and interaction between them, which is critical for QA evaluation. In this work, we propose a hierarchical reasoning graph neural network for the automatic assessment of question–answer pairs. Specifically, we construct a sentence-level relational graph neural network to capture the dependency information of sentences in or between the question and the answer. Based on these graphs, we employ a semantic-level reasoning graph attention network to model the interaction states of the current QA session. Finally, we propose a gated recurrent unit encoder to represent the temporal question–answer pairs for the final prediction. Empirical results on CHNAT (a real-world dataset) validate that our proposed model significantly outperforms matching-based benchmark models. Ablation studies and experimental results with 10 random seeds also show the effectiveness and stability of our models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://www.xfyun.cn/services/lfasr

References

  1. Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, Gulcehre C, Song F, Ballard A, Gilmer J, Dahl G, Vaswani A, Allen K, Nash C, Langston V, Dyer C, Heess N, Wierstra D, Kohli P, Botvinick M, Vinyals O, Li Y, Pascanu R (2018) Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261

  2. Bian S, Xin ZW, Song Y, Zhang T, Wen J-R (2019) Domain adaptation for person-job fit with transferable deep global match network. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 4812–4822

  3. Chen Y, Wu L, Zaki MJ (2020) Toward subgraph guided knowledge graph question generation with graph neural networks. arXiv:2004.06015

  4. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR:1412.3555

  5. L CLAUDIA and MARTIN CC-rater (2003) Automated scoring of short-answer questions. Comput Human 37:92–96

  6. Conneau A, Kiela D, Schwenk H, Barrault L, Bordes A (2017) Supervised learning of universal sentence representations from natural language inference data. EMNLP, pp 670–680

  7. Dai H, Dai B, Song L (2016) Discriminative embeddings of latent variable models for structured data. In: International conference on machine learning, pp 2702–2711

  8. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  9. Richard DA, Paula D, Donald DL (1991) Reproducibility and responsiveness of health status measures statistics and strategies for evaluation. Control Clin Trials 12(4):S142–S158

    Article  Google Scholar 

  10. Ghosal D, Majumder N, Poria S, Chhaya N, Gelbukh A (2019) Dialoguegcn: a graph convolutional neural network for emotion recognition in conversation. CoRR, abs/1908.11540

  11. Goenka P, Piplani M, Sawhney R, Mathur P, Shah RR (2020) Esas: towards practical and explainable short answer scoring (student abstract). In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 13797–13798

  12. Hemamou Léo FG, Martin J-C, Clavel C (2019) Slices of attention in asynchronous video job interviews. In: 2019 8th international conference on affective computing and intelligent interaction (ACII). IEEE, pp 1–7

  13. Hemamou Léo FG, Vandenbussche V, Martin J-C, Clavel C (2019) Hirenet: A hierarchical attention model for the automatic analysis of asynchronous video job interviews. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 573–581

  14. Yunseok J, Yale S, Dongjoo KC, Youngjae Y, Youngjin K, Gunhee K (2019) Video question answering with spatio-temporal reasoning. Int J Comput Vis 127(10):1385–1412

    Article  Google Scholar 

  15. Jiang J-Y, Zhang M, Li C, Bendersky M, Golbandi N, Najork M (2019) Semantic text matching for long-form documents. In: The world wide web conference, pp 795–806

  16. Jiang P, Han Y (2020) Reasoning with heterogeneous graph alignment for video question answering. In: AAAI, pp 11109–11116

  17. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In ICLR, pp 1324–1339

  18. Lawrence I, Lin K (1989) A concordance correlation coefficient to evaluate reproducibility. Biometrics, pp 255–268

  19. Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning, pp 1188–1196

  20. Liu X, Chen Q, Liu Y, Siebert J, Baotian H, Xiangping W, Tang B (2021) Decomposing word embedding with the capsule network. Knowl Based Syst 212:106611

    Article  Google Scholar 

  21. Lun J, Zhu J, Tang Y, Yang M (2020) Multiple data augmentation strategies for improving performance on automatic short answer scoring. In: AAAI, pp 13389–13396

  22. Luo W, Zhang C, Zhang X, Wu H (2019) Improving action recognition with the graph-neural-network-based interaction reasoning. In: 2019 IEEE visual communications and image processing (VCIP). IEEE, pp 1–4

  23. Luo Y, Zhang H, Wen Y, Zhang X (2019) Resumegan: an optimized deep representation learning framework for talent-job fit via adversarial learning. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 1101–1110

  24. Marcheggiani D, Bastings J, Titov I (2018) Exploiting semantics in neural machine translation with graph convolutional networks. In: Proceedings of NAACL, pp 486–492

  25. Mueller J, Thyagarajan A (2016) Siamese recurrent architectures for learning sentence similarity. In: Thirtieth AAAI conference on artificial intelligence

  26. Pan L, Xie Y, Feng Y, Chua T-S, Kan M-Y (2020) Semantic graphs for generating deep questions. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 1463–1475

  27. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  28. Qin C, Zhu H, Xu T, Zhu C, Jiang L, Chen E, Xiong H (2018) Enhancing person-job fit for talent recruitment: An ability-aware neural network approach. In: The 41st international ACM SIGIR conference on research & development in information retrieval, pp 25–34

  29. Riordan B, Horbach A, Cahill A, Zesch T, Lee C (2017) Investigating neural architectures for short answer scoring. In: Proceedings of the 12th workshop on innovative use of NLP for building educational applications, pp 159–168

  30. Saha S, Dhamecha TI, Marvaniya S, Foltz P, Sindhgatta R, Sengupta B (2019) Joint multi-domain learning for automatic short answer grading. arXiv:1902.09183

  31. Schlichtkrull M, Kipf TN, Bloem P, Den Berg RV, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference. Springer, pp 593–607

  32. Shen D, Zhu H, Zhu C, Xu T, Ma C, Xiong H (2018) A joint learning approach to intelligent job interview assessment. In: IJCAI, pp 3542–3548

  33. Suen H-Y, Hung K-E, Lin C-L (2019) Tensorflow-based automatic personality recognition used in asynchronous video interviews. IEEE Access 7:61018–61023

    Article  Google Scholar 

  34. Panagiotis T, George T, Nicolaou MA, Schuller BW, Stefanos Z (2017) End-to-end multimodal emotion recognition using deep neural networks. IEEE J Sel Top Signal Process 11(8):1301–1309

    Article  Google Scholar 

  35. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations

  36. Wang D, Liu P, Zheng Y, Qiu X, Huang X (2020) Heterogeneous graph neural networks for extractive document summarization. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 6209–6219

  37. Xu T, Zhu H, Zhu C, Li P, Xiong H (2018) Measuring the popularity of job skills in recruitment market: a multi-criteria approach. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  38. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the AAAI conference on artificial intelligence, pp 7370–7377

  39. Zhang Y, Yu X, Cui Z, Wu S, Wen Z, Wang L (2020) Every document owns its structure: inductive text classification via graph neural networks. In: Association for computational linguistics, pp 334–339

  40. Zhang Y, Chen X, Yang Y, Ramamurthy A, Li B, Qi Y, Song L (2020) Efficient probabilistic logic reasoning with graph neural networks. ICLR

  41. Zhao S, Zhang Y, Xiong X, Botelho A, Heffernan N (2017) A memory-augmented neural model for automated grading. In: Proceedings of the fourth (2017) ACM conference on learning@ scale, pp 189–192

  42. Zhou J, Han X, Yang C, Liu Z, Wang L, Li C, Sun M (2019) Gear: graph-based evidence aggregating and reasoning for fact verification. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 892–901

Download references

Acknowledgements

This work is supported by Natural Science Foundation of China (Grant No. 61872113, 61573118, U1813215, 61876052), Special Foundation for Technology Research Program of Guangdong Province (Grant No. 2015B010131010), Strategic Emerging Industry Development Special Funds of Shenzhen (Grant No. JCYJ20170307150528934, JCYJ2017 0811153836555, JCYJ20180306172232154), Innovation Fund of Harbin Institute of Technology (Grant No. HIT. NSRIF.2017052).

Author information

Authors and Affiliations

  1. Intelligent Computing Research Center, Harbin Institute of Technology (Shenzhen), Building L, HIT Campus, Shenzhen University Town, Shenzhen, 518055, Guangdong, China

    Kai Chen, Meng Niu & Qingcai Chen

Authors
  1. Kai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingcai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingcai Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, K., Niu, M. & Chen, Q. A hierarchical reasoning graph neural network for the automatic scoring of answer transcriptions in video job interviews. Int. J. Mach. Learn. & Cyber. 13, 2507–2517 (2022). https://doi.org/10.1007/s13042-022-01540-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01540-8

Keywords

Search

Navigation