Design of Conversational Agents for CSCL: Comparing Two Types of Agent Intervention Strategies in a University Classroom
- 1k Downloads
The use of Conversational Agents (CAs) in computer-supported collaborative learning (CSCL) has shown promising results regarding students’ productive dialogue and learning. Yet, limited work has explored the connection between the configuration of the CA behavior, the nature of the learning task, and the student behavior in authentic educational settings. In this work, we describe a pedagogical design space of CAs for collaborative learning composed of three dimensions: task design, domain model, and agent intervention strategies. We conduct an initial field study in a university classroom comparing two types of agent intervention strategies based on student participation, dialogue, and satisfaction. 54 university students worked in pairs in the same collaborative brainstorming task with a CA tool and were randomly assigned in two CA conditions with a) knowledge-based prompts to connect two domain concepts, b) social prompts to link their partners’ contributions. The results show that students who received knowledge-based prompts significantly exchanged more messages with evidence of explicit reasoning and were more satisfied with the agent and their discussion during the task. Students from both conditions reported problems like the lack of context-awareness and timely interventions by the agent. We discuss the relation between the agent intervention strategies and the task design towards seeking design recommendations for CAs in CSCL.
KeywordsConversational agents CSCL Task design Agent intervention strategies Dialogue
Authors would like to thank Alejandra Martínez-Monés who assisted in the study and Stavros Demetriadis, Tasos Karakostas, and Stathis Nikolaidis who provided the CA technology, in the context of project grant 588438-EPP-1-2017-1-EL-EPPKA2-KA funded by the European Commission, which supported this research. Further support was provided by project grant TIN2017-85179-C3-2-R funded by the Spanish State Research Agency (AEI) and the European Regional Development Fund and project grant VA257P18 funded by the Regional Government of Castilla y León and the European Regional Development Fund.
- 2.Amarasinghe, I., Hernández Leo, D., Manathunga, K., Jonsson, A.: Sustaining continuous collaborative learning flows in MOOCs: orchestration agent approach. J. Univ. Comput. Sci. 24(8), 1034–1051 (2018)Google Scholar
- 3.Bittner, E., Oeste-Reiß, S., Leimeister, J.M.: Where is the bot in our team? toward a taxonomy of design option combinations for conversational agents in collaborative work. In: Proceedings of the 52nd Hawaii International Conference on System Sciences, pp. 284–293 (2019)Google Scholar
- 4.Bittner, E.A., Küstermann, G.C., Tratzky, C.: The facilitator is a Bot: towards a converstational agent for facilitating idea elaboration on idea platforms. In: Proceedings of the 27th European Conference on Information Systems (ECIS) (2019)Google Scholar
- 7.Goodyear, P., Carvalho, L.: The Architecture of Productive Learning Networks. Routledge, Abingdon (2014)Google Scholar
- 9.Huang, J.X., Lee, K.S., Kwon, O.W., Kim, Y.K.: A chatbot for a dialogue-based second language learning system. In: CALL In a Climate of Change: Adapting to Turbulent Global Conditions, Short Papers from EUROCALL, vol. 2017, pp. 151–157 (2017)Google Scholar
- 12.Kumar, R., Rosé, C.P., Wang, Y.C., Joshi, M., Robinson, A.: Tutorial dialogue as adaptive collaborative learning support. Front. Artif. Intell. Appl. 158, 383 (2007)Google Scholar
- 14.Murray, T.: An overview of intelligent tutoring system authoring tools: Updated analysis of the state of the art. In: Murray, T., Blessing, S.B., Ainsworth, S. (eds.) Authoring Tools for Advanced Technology Learning Environments, pp. 491–544. Springer, Dordrecht (2003). https://doi.org/10.1007/978-94-017-0819-7_17dCrossRefGoogle Scholar
- 15.OECD: Draft collaborative problem solving framework (2015). http://www.oecd.org/pisa/pisaproducts/Draft. PISA 2015 Collaborative Problem Solving Framework.pdf
- 16.Saldaña, J.: Thinking Qualitatively: Methods of Mind. Sage Publications, Thousand Oaks (2014)Google Scholar
- 20.Winkler, R., Söllner, M.: Unleashing the potential of chatbots in education: a state-of-the-art analysis. In: Academy of Management Annual Meeting (AOM) (2018)Google Scholar