Skip to main content
SpringerLink
Log in

The Impact of Chatbot Linguistic Register on User Perceptions: A Replication Study

  • Conference paper
  • First Online:
Chatbot Research and Design (CONVERSATIONS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13171))

Included in the following conference series:

Abstract

Chatbots often perform social roles associated with human interlocutors; hence, designing chatbot language to conform with the stereotypes of its social category is critical to the success of this technology. In a previous study, Chaves et al. performed a corpus analysis to evaluate how language variation in an interactional situation, namely the linguistic register, influences the user’s perceptions of a chatbot. In this paper, we present a replication study with a different corpus to understand the effect of corpus selection in the original study’s findings. Our results confirm the findings in the previous study and demonstrate the reproducibility of the research methodology; we also reveal new insights about language design for tourist assistant chatbots.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available at http://m.me/praguevisitor. Last accessed: November, 2021.

  2. 2.

    https://www.praguevisitor.eu.

  3. 3.

    Download and more details at https://www.microsoft.com/en-us/research/project/frames-dataset/.

  4. 4.

    https://www.prolific.co.

  5. 5.

    The R code and datasets are available on GitHub [10].

References

  1. Araujo, T.: Living up to the chatbot hype: the influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Comput. Hum. Behav. 85, 183–189 (2018)

    Article  Google Scholar 

  2. Argamon, S.: Register in computational language research. Regist. Stud. 1(1), 100–135 (2019)

    Article  Google Scholar 

  3. Asri, L.E., et al.: Frames: a corpus for adding memory to goal-oriented dialogue systems. In: Proceedings of the SIGDIAL 2017 Conference, pp. 207–219. Association for Computational Linguistics, Saarbrucken(2017)

    Google Scholar 

  4. Biber, D.: Variation Across Speech and Writing. Cambridge University Press, Cambridge, UK (1988)

    Book  Google Scholar 

  5. Biber, D.: Register as a predictor of linguistic variation. Corpus Linguist. Ling. Theory 8(1), 9–37 (2012)

    Article  Google Scholar 

  6. Biber, D.: Mat-multidimensional analysis tagger (2017). https://goo.gl/u7h9gb

  7. Biber, D.: Text-linguistic approaches to register variation. Regist. Stud. 1(1), 42–75 (2019)

    Article  Google Scholar 

  8. Biber, D., Conrad, S.: Register, Genre, and Style, 2nd edn. Cambridge University Press, New York (2019)

    Book  Google Scholar 

  9. Biber, D., Johansson, S., Leech, G., Conrad, S., Finegan, E., Quirk, R.: Longman Grammar of Spoken and Written English, vol. 2. Pearson Longman, London (1999)

    Google Scholar 

  10. Chaves, A.P.: Github Repository (2020). https://github.com/chavesana/chatbots-register

  11. Chaves, A.P., Doerry, E., Egbert, J., Gerosa, M.: It’s how you say it: identifying appropriate register for chatbot language design. In: Proceedings of the 7th International Conference on Human-Agent Interaction (HAI 2019). p. 8. ACM, New York, October 2019. https://doi.org/10.1145/3349537.3351901

  12. Chaves, A.P., Egbert, J., Gerosa, M.A.: Chatting like a robot: the relationship between linguistic choices and users’ experiences. In: ACM CHI 2019 Workshop on Conversational Agents: Acting on the Wave of Research and Development. p. 8. Glasgow (2019). https://convagents.org/

  13. Chaves, A.P., Egbert, J., Hocking, T., Doerry, E., Gerosa, M.A.: Chatbots language design: the influence of language variation on user experience. ACM Trans. Comput.-Hum. Inter. (to appear Author’s version at arXiv:210111089)

  14. Chaves, A.P., Gerosa, M.A.: How should my chatbot interact? A survey on social characteristics in human-chatbot interaction design. Int. J. Hum.-Comput. Interact. 37, 1–30 (2020)

    Google Scholar 

  15. Conrad, S., Biber, D.: Multi-dimensional Studies of Register Variation in English. Routledge, New York (2014)

    Google Scholar 

  16. Dennis, A.R., Valacich, J.S.: A replication manifesto. AIS Trans. Replic. Res. 1(1), 1 (2015)

    Google Scholar 

  17. Elsholz, E., Chamberlain, J., Kruschwitz, U.: Exploring language style in chatbots to increase perceived product value and user engagement. In: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, pp. 301–305. ACM, New York (2019)

    Google Scholar 

  18. Fogg, B.: Computers as persuasive social actors. In: Fogg, B. (ed.) Persuasive Technology, Chap. 5, pp. 89–120. Interactive Technologies, Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  19. Følstad, A., Brandtzaeg, P.B.: Users’ experiences with chatbots: findings from a questionnaire study. Qual. User Exp. 5(1), 1–14 (2020). https://doi.org/10.1007/s41233-020-00033-2

    Article  Google Scholar 

  20. Følstad, A., Nordheim, C.B., Bjørkli, C.A.: What makes users trust a chatbot for customer service? An exploratory interview study. In: Bodrunova, S.S. (ed.) INSCI 2018. LNCS, vol. 11193, pp. 194–208. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01437-7_16

    Chapter  Google Scholar 

  21. Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. Articles 33(1), 1–22 (2010). https://doi.org/10.18637/jss.v033.i01, https://www.jstatsoft.org/v033/i01

  22. Gnewuch, U., Morana, S., Maedche, A.: Towards designing cooperative and social conversational agents for customer service. In: Proceedings of the International Conference on Information Systems 2017, vol. 1. Association for Information Systems, South Korea (2017)

    Google Scholar 

  23. Go, E., Sundar, S.S.: Humanizing chatbots: the effects of visual, identity and conversational cues on humanness perceptions. Comput. Hum. Behav. 97, 304–316 (2019)

    Article  Google Scholar 

  24. Google Developers: Conversation design by google (2021). https://developers.google.com/assistant/conversation-design/what-is-conversation-design.https://developers.google.com/assistant/conversation-design/what-is-conversation-design. Accessed 3 Mar 2021

  25. Grudin, J., Jacques, R.: Chatbots, humbots, and the quest for artificial general intelligence. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–11. ACM, New York (2019)

    Google Scholar 

  26. Hill, J., Ford, W.R., Farreras, I.G.: Real conversations with artificial intelligence: a comparison between human-human online conversations and human-chatbot conversations. Comput. Hum. Behav. 49, 245–250 (2015)

    Article  Google Scholar 

  27. Hwang, S., Kim, B., Lee, K.: A data-driven design framework for customer service chatbot. In: Marcus, A., Wang, W. (eds.) HCII 2019. LNCS, vol. 11583, pp. 222–236. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23570-3_17

    Chapter  Google Scholar 

  28. Jacquet, B., Hullin, A., Baratgin, J., Jamet, F.: The impact of the Gricean maxims of quality, quantity and manner in chatbots. In: 2019 International Conference on Information and Digital Technologies (IDT), pp. 180–189. IEEE (2019)

    Google Scholar 

  29. Jakic, A., Wagner, M.O., Meyer, A.: The impact of language style accommodation during social media interactions on brand trust. J. Serv. Manag. 28(3), 418–441 (2017)

    Article  Google Scholar 

  30. Jenkins, M.-C., Churchill, R., Cox, S., Smith, D.: Analysis of user interaction with service oriented chatbot systems. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 76–83. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73110-8_9

    Chapter  Google Scholar 

  31. Jiang, R.E., Banchs, R.: Towards improving the performance of chat oriented dialogue system. In: 2017 International Conference on Asian Language Processing (IALP), pp. 23–26. IEEE, New York (2017)

    Google Scholar 

  32. Kessler, B.: Computational dialectology in Irish Gaelic. In: Proceedings of the Seventh Conference on European Chapter of the Association for Computational Linguistics, EACL 1995, pp. 60–66. Morgan Kaufmann Publishers Inc., San Francisco (1995). https://doi.org/10.3115/976973.976983, https://doi.org/10.3115/976973.976983

  33. Kirakowski, J., et al.: Establishing the hallmarks of a convincing chatbot-human dialogue. In: Human-Computer Interaction. InTech, London (2009)

    Google Scholar 

  34. Laranjo, I., et al.: Conversational agents in healthcare: a systematic review. J. Am. Med. Inf. Assoc. 25(9), 1248–1258 (2018)

    Article  Google Scholar 

  35. Li, J., Galley, M., Brockett, C., Spithourakis, G.P., Gao, J., Dolan, B.: A persona-based neural conversation model. arXiv preprint arXiv:1603.06155 (2016)

  36. Li, Y., Su, H., Shen, X., Li, W., Cao, Z., Niu, S.: Dailydialog: a manually labelled multi-turn dialogue dataset. In: International Joint Conference on Natural Language Processing (IJCNLP), pp. 986–995. Asian Federation of Natural Language Processing, Taipei, Taiwan (2017)

    Google Scholar 

  37. Liebrecht, C., Sander, L., van Hooijdonk, C.: Too informal? how a chatbot’s communication style affects brand attitude and quality of interaction. In: Conversations 2020: 4th International Workshop on Chatbot Research (2020)

    Google Scholar 

  38. Maslowski, I., Lagarde, D., Clavel, C.: In-the-wild chatbot corpus: from opinion analysis to interaction problem detection. In: International Conference on Natural Language. Signal and Speech Processing, pp. 115–120. International Science and General Applications, Marocco (2017)

    Google Scholar 

  39. Morrissey, K., Kirakowski, J.: ‘Realness’ in chatbots: establishing quantifiable criteria. In: Kurosu, M. (ed.) HCI 2013. LNCS, vol. 8007, pp. 87–96. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39330-3_10

    Chapter  Google Scholar 

  40. Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 72–78. ACM, New York (1994)

    Google Scholar 

  41. Nevill, C., Bell, T.: Compression of parallel texts. Inf. Process. Manag. 28(6), 781–793 (1992)

    Article  Google Scholar 

  42. Pillai, R., Sivathanu, B.: Adoption of AI-based chatbots for hospitality and tourism. Int. J. Contemp. Hosp. Manag. 32(10), 3199–3226 (2020)

    Google Scholar 

  43. Resendez, V.: A very formal agent: how culture, mode of dressing and linguistic style influence the perceptions toward an Embodied Conversational Agent? Master’s thesis, University of Twente (2020)

    Google Scholar 

  44. Svenningsson, N., Faraon, M.: Artificial intelligence in conversational agents: a study of factors related to perceived humanness in chatbots. In: Proceedings of the 2019 2nd Artificial Intelligence and Cloud Computing Conference on ZZZ, pp. 151–161 (2019)

    Google Scholar 

  45. Tariverdiyeva, G.: Chatbots’ perceived usability in information retrieval tasks: an exploratory analysis. Master’s thesis, University of Twente (2019)

    Google Scholar 

  46. Tegos, S., Demetriadis, S., Tsiatsos, T.: An investigation of conversational agent interventions supporting historical reasoning in primary education. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) International Conference on Intelligent Tutoring Systems, pp. 260–266. Springer International Publishing, Cham (2016)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Federal University of Technology - Parana, Campo Mourão, PR, Brazil

    Ana Paula Chaves

  2. Northern Arizona University, Flagstaff, AZ, USA

    Marco Aurelio Gerosa

Authors
  1. Ana Paula Chaves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Aurelio Gerosa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Paula Chaves .

Editor information

Editors and Affiliations

  1. SINTEF, Oslo, Norway

    Asbjørn Følstad

  2. VU Amsterdam, Amsterdam, The Netherlands

    Theo Araujo

  3. CERTH-ITI, Thessaloniki, Greece

    Symeon Papadopoulos

  4. Durham University, Durham, UK

    Effie L.-C. Law

  5. University of Edinburgh, Edinburgh, UK

    Ewa Luger

  6. Center for Artificial Intelligence Research, University of Agder, Kristiansand, Norway

    Morten Goodwin

  7. University of Oslo, Oslo, Norway

    Petter Bae Brandtzaeg

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chaves, A.P., Gerosa, M.A. (2022). The Impact of Chatbot Linguistic Register on User Perceptions: A Replication Study. In: Følstad, A., et al. Chatbot Research and Design. CONVERSATIONS 2021. Lecture Notes in Computer Science(), vol 13171. Springer, Cham. https://doi.org/10.1007/978-3-030-94890-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-94890-0_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-94889-4

  • Online ISBN: 978-3-030-94890-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation