Advertisement

Leveraging Conversational Systems to Assists New Hires During Onboarding

  • Praveen Chandar
  • Yasaman Khazaeni
  • Matthew Davis
  • Michael Muller
  • Marco Crasso
  • Q. Vera Liao
  • N. Sadat Shami
  • Werner Geyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10514)

Abstract

The task of onboarding a new hire consumes great amounts of resources from organizations. The faster a “newbie” becomes an “insider”, the higher the chances of job satisfaction, retention, and advancement in their position. Conversational agents (AI agents) have the potential to effectively transform productivity in many enterprise workplace scenarios so applying them to the onboarding process can prove to be a very solid use case for such agents. In this work, we present a conversational system to aid new hires through their onboarding process. Users interact with the system via an instant messaging platform, to fulfill their work related information needs as if it were a human assistant. We describe the end-to-end process involved in building a domain specific conversational system and share our experiences in deploying it to 344 new hires in a month-long study. The feasibility of our approach is evaluated by analyzing message logs and questionnaires. Through three different measures, we observed an accuracy of about 60% at the message level and a higher than average retention rate for the agent. Our results suggest that this agent-based approach can very well compete with the existing tools for new hires.

Keywords

Artificial intelligence Conversational agents Personal assistants Evaluation NLP 

References

  1. 1.
    AlchemyAPI (2010). http://www.alchemyapi.com
  2. 2.
    Bickmore, T., Cassell, J.: Relational agents: a model and implementation of building user trust. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 396–403. ACM (2001)Google Scholar
  3. 3.
    Bickmore, T., Pfeifer, L., Schulman, D.: Relational agents improve engagement and learning in science museum visitors. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 55–67. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23974-8_7 CrossRefGoogle Scholar
  4. 4.
    Bickmore, T.W., Pfeifer, L.M., Jack, B.W.: Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1265–1274. ACM (2009)Google Scholar
  5. 5.
    Bickmore, W.T., Utami, D., Matsuyama, R., Paasche-Orlow, K.M.: Improving access to online health information with conversational agents: a randomized controlled experiment. J. Med. Internet Res. 18(1), e1 (2016)CrossRefGoogle Scholar
  6. 6.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(Jan), 993–1022 (2003)zbMATHGoogle Scholar
  7. 7.
    Cassell, J.: Embodied conversational interface agents. Commun. ACM 43(4), 70–78 (2000)CrossRefGoogle Scholar
  8. 8.
    Ji, Z., Lu, Z., Li, H.: An information retrieval approach to short text conversation. arXiv preprint arXiv:1408.6988 (2014)
  9. 9.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, pp. 1746–1751 (2014)Google Scholar
  10. 10.
    Kiseleva, J., Williams, K., Jiang, J., Hassan Awadallah, A., Crook, A.C., Zitouni, I., Anastasakos, T.: Understanding user satisfaction with intelligent assistants. In: Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, pp. 121–130. ACM (2016)Google Scholar
  11. 11.
    Lave, J., Wenger, E.: Situated Learning. The Language of Science Education. Cambridge University Press, New York (1991)CrossRefGoogle Scholar
  12. 12.
    Liao, Q.V., Davis, M., Geyer, W., Muller, M., Shami, N.S.: What can you do?: Studying social-agent orientation and agent proactive interactions with an agent for employees. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, pp. 264–275. ACM (2016)Google Scholar
  13. 13.
    Metzler, D., Croft, W.B.: A Markov random field model for term dependencies. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development In Information Retrieval, pp. 472–479. ACM (2005)Google Scholar
  14. 14.
    Miller, V.D., Jablin, F.M.: Information seeking during organizational entry: influences, tactics, and a model of the process. Acad. Manag. Rev. 16(1), 92–120 (1991)CrossRefGoogle Scholar
  15. 15.
    Pickard, M.D., Burns, M.B., Moffitt, K.C.: A theoretical justification for using embodied conversational agents (ECAs) to augment accounting-related interviews. J. Inf. Syst. 27(2), 159–176 (2013)Google Scholar
  16. 16.
    Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social media. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, pp. 583–593. Association for Computational Linguistics, Stroudsburg, PA, USA (2011). http://dl.acm.org/citation.cfm?id=2145432.2145500
  17. 17.
    Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, ACL 2015, pp. 1577–1586 (2015)Google Scholar
  18. 18.
    Shechtman, N., Horowitz, L.M.: Media inequality in conversation: how people behave differently when interacting with computers and people. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 281–288. ACM (2003)Google Scholar
  19. 19.
    Silvervarg, A., Jönsson, A.: Subjective and objective evaluation of conversational agents in learning environments for young teenagers. In: Proceedings of the 7th IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems (2011)Google Scholar
  20. 20.
    Silvervarg, A., JÃűnsson, A.: Subjective and objective evaluation of conversational agents in learning environments for young teenagers. In: 7th IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems (2011)Google Scholar
  21. 21.
    Strohman, T., Metzler, D., Turtle, H., Croft, W.B.: Indri: a language model-based search engine for complex queries. In: Proceedings of the International Conference on Intelligent Analysis, vol. 2, pp. 2–6. Citeseer (2005)Google Scholar
  22. 22.
    Tegos, S., Demetriadis, S.: Conversational agents improve peer learning through building on prior knowledge. J. Educ. Technol. Soc. 20(1), 99–111 (2017)Google Scholar
  23. 23.
    Trappl, R. (ed.): Your Virtual Butler. LNCS (LNAI), vol. 7407. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-37346-6 Google Scholar
  24. 24.
    Traum, D., DeVault, D., Lee, J., Wang, Z., Marsella, S.: Incremental dialogue understanding and feedback for multiparty, multimodal conversation. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 275–288. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33197-8_29 CrossRefGoogle Scholar
  25. 25.
    Turing, A.M.: Computing machinery and intelligence. Mind 59(236), 433–460 (1950)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Vinyals, O., Le, Q.: A neural conversational model. arXiv preprint arXiv:1506.05869 (2015)
  27. 27.
    Walker, M.A., Litman, D.J., Kamm, C.A., Abella, A.: Paradise: a framework for evaluating spoken dialogue agents. In: Proceedings of the Eighth Conference on European Chapter of the Association for Computational Linguistics, pp. 271–280. Association for Computational Linguistics (1997)Google Scholar
  28. 28.
    Wang, Z., Lee, J., Marsella, S.: Towards more comprehensive listening behavior: beyond the bobble head. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 216–227. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23974-8_24 CrossRefGoogle Scholar
  29. 29.
    Williams, J., Raux, A., Ramachandran, D., Black, A.: The dialog state tracking challenge. In: Proceedings of the SIGDIAL 2013 Conference, pp. 404–413 (2013)Google Scholar
  30. 30.
    Xiao, J., Stasko, J., Catrambone, R.: An empirical study of the effect of agent competence on user performance and perception. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 1, pp. 178–185. IEEE Computer Society (2004)Google Scholar
  31. 31.
    Yao, M.: 4 critical steps to maximize chatbot retention & engagement (December 2016). http://www.topbots.com/4-critical-steps-to-maximize-chatbot-retention-engagement/

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  • Praveen Chandar
    • 1
  • Yasaman Khazaeni
    • 1
  • Matthew Davis
    • 1
  • Michael Muller
    • 1
  • Marco Crasso
    • 2
  • Q. Vera Liao
    • 3
  • N. Sadat Shami
    • 4
  • Werner Geyer
    • 1
  1. 1.IBM ResearchCambridgeUSA
  2. 2.IBM Research - SilverGateBuenos AiresArgentina
  3. 3.IBM ResearchYorktownUSA
  4. 4.IBMArmonkUSA

Personalised recommendations