Advertisement

Conversational agents for academically productive talk: a comparison of directed and undirected agent interventions

  • Stergios Tegos
  • Stavros Demetriadis
  • Pantelis M. Papadopoulos
  • Armin Weinberger
Article
First Online:

Abstract

Conversational agents that draw on the framework of academically productive talk (APT) have been lately shown to be effective in helping learners sustain productive forms of peer dialogue in diverse learning settings. Yet, literature suggests that more research is required on how learners respond to and benefit from such flexible agents in order to fine-tune the design of automated APT intervention modes and, thus, enhance agent pedagogical efficacy. Building on this line of research, this work explores the impact of a configurable APT agent that prompts peers to build on prior knowledge and logically connect their contributions to important domain concepts discussed in class. A total of 96 computer science students engaged in a dialogue-based activity in the context of a Human-Computer Interaction (HCI) university course. During the activity, students worked online in dyads to accomplish a learning task. The study compares three conditions: students who collaborated without any agent interference (control), students who received undirected agent interventions that addressed both peers in the dyad (U treatment), and students who received directed agent interventions addressing a particular learner instead of the dyad (D treatment). The results suggest that although both agent intervention methods can improve students’ learning outcomes and dyad in-task performance, the directed one is more effective than the undirected one in enhancing individual domain knowledge acquisition and explicit reasoning. Furthermore, findings show that the positive effect of the agent on dyad performance is mediated by the frequency of students’ contributions displaying explicit reasoning, while most students perceive agent involvement favorably.

Keywords

Conversational agent Academically productive talk Computer-supported collaborative learning Peer dialogue 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

We are appreciative of Fotini Bourotzoglou’s contribution to this work.

References

  1. Adamson, D., & Rosé, C. P. (2013). Academically Productive Talk: One Size Does Not Fit All. In Artificial Intelligence in Education (AIED) 2013 Workshops Proceedings (p. 51–60).Google Scholar
  2. Adamson, D., Ashe, C., Jang, H., Yaron, D., & Rosé, C. P, (2013). Intensification of group knowledge exchange with academically productive talk agents. In N. Rummel, M. Kapur, M. Nathan, & S. Puntambekar (Eds.), To See the World and a Grain of Sand: Learning across Levels of Space, Time, and Scale: CSCL 2013 Conference Proceedings (vol. 1, pp. 10–17).Google Scholar
  3. Adamson, D., Dyke, G., Jang, H., & Rosé, C. P. (2014). Towards an agile approach to adapting dynamic collaboration support to student needs. International Journal of Artificial Intelligence in Education, 24(1), 92–124.CrossRefGoogle Scholar
  4. Asterhan, C. S., & Schwarz, B. B. (2016). Argumentation for learning: well-trodden paths and unexplored territories. Educational Psychologist, 51(2), 164–187.CrossRefGoogle Scholar
  5. Boeije, H. (2002). A purposeful approach to the constant comparative method in the analysis of qualitative interviews. Quality and Quantity, 36(4), 391–409.CrossRefGoogle Scholar
  6. Brandom, R. (1998). Making it explicit: reasoning, representing, and discursive commitment. Harvard University Press.Google Scholar
  7. Cafaro, A., Glas, N., & Pelachaud, C. (2016). The Effects of Interrupting Behavior on Interpersonal Attitude and Engagement in Dyadic Interactions. In J. Thangarajah, K. Tuyls, C. Jonker, & S. Marsella (Eds.), 15th International Conference on Autonomous Agents and Multiagent Systems (pp. 911–920).Google Scholar
  8. Chi, M. T. (2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105.CrossRefGoogle Scholar
  9. Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for computer-supported collaborative learning. Journal of Computer Assisted Learning, 23(1), 1–13.CrossRefGoogle Scholar
  10. Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6, 241–252.CrossRefGoogle Scholar
  11. Dyke, G., Adamson, D., Howley, I., & Rosé, C. P. (2013). Enhancing scientific reasoning and discussion with conversational agents. Learning Technologies, IEEE Transactions on, 6(3), 240–247.CrossRefGoogle Scholar
  12. Fischer, F., Kollar, I., Mandl, H., & Haake, J. (Eds.) (2007). Scripting computer-supported communication of knowledge. Cognitive, computational and educational perspectives. New York: Springer.Google Scholar
  13. Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56–66.CrossRefGoogle Scholar
  14. Goodman, B. A., Linton, F. N., Gaimari, R. D., Hitzeman, J. M., Ross, H. J., & Zarrella, G. (2005). Using dialogue features to predict trouble during collaborative learning. User Modeling and User-Adapted Interaction, 15(1–2), 85–134.CrossRefGoogle Scholar
  15. Gulz, A., Haake, M., Silvervarg, A., Sjödén, B., & Veletsianos, G. (2011). Building a social conversational pedagogical agent: design challenges and methodological approaches. In D. Perez-Marin & I. Pascual-Nieto (Eds.), Conversational Agents and Natural Language Interaction: Techniques and Effective Practices (pp. 128–155). IGI Global.Google Scholar
  16. Harrer, A., McLaren, B. M., Walker, E., Bollen, L., & Sewall, J. (2006). Creating cognitive tutors for collaborative learning: steps toward realization. User Modeling and User-Adapted Interaction, 16(3–4), 175–209.CrossRefGoogle Scholar
  17. Hayes, A. F. (2013). An introduction to mediation, moderation, and conditional process analysis: a regression-based approach. New York, NY: Guilford Press.Google Scholar
  18. Hmelo-Silver, C. E. (2013). Multivocality as a tool for design-based research. In D. D. Suthers, K. Lund, C. P. Rosé, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 561–573). New York: Springer.CrossRefGoogle Scholar
  19. Howley, I., Kumar, R., Mayfield, E., Dyke, G., & Rosé, C. P. (2013). Gaining insights from sociolinguistic style analysis for redesign of conversational agent based support for collaborative learning. In D. Suthers, K. Lund, C. P. Rosé, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 477–494) .Springer USGoogle Scholar
  20. Huitt, W. (2011). Bloom et al.’s taxonomy of the cognitive domain. In Educational Psychology Interactive. Valdosta, GA: Valdosta State University.Google Scholar
  21. Kumar, R., & Rosé, C. P. (2011). Architecture for building conversational agents that support collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 21–34.CrossRefGoogle Scholar
  22. Liu, C. C., & Tsai, C. C. (2008). An analysis of peer interaction patterns as discoursed by on-line small group problem-solving activity. Computers & Education, 50(3), 627–639.CrossRefGoogle Scholar
  23. Ludvigsen, S., & Mørch, A. (2010). Computer-supported collaborative learning: basic concepts, multiple perspectives, and emerging trends. In B. McGaw, P. Peterson, & E. Baker (Eds.), The international encyclopedia of education 3rd edition, volume 5 (pp. 290–296). Oxford, UK: Elsevier.CrossRefGoogle Scholar
  24. Magnisalis, I., Demetriadis, S., & Karakostas, A. (2011). Adaptive and intelligent systems for collaborative learning support: a review of the field. IEEE Transactions on Learning Technologies, 4(1), 5–20.CrossRefGoogle Scholar
  25. Michaels, S., & O’Connor, C. (2013). Conceptualizing talk moves as tools: professional development approaches for academically productive discussion. In L. B. Resnick, C. Asterhan, & S. N. Clarke (Eds.), Socializing intelligence through talk and dialogue. Washington DC: American Educational Research Association.Google Scholar
  26. Michaels, S., O’Connor, C., & Resnick, L. B. (2008). Deliberative discourse idealized and realized: accountable talk in the classroom and in civic life. Studies in Philosophy and Education, 27(4), 283–297.CrossRefGoogle Scholar
  27. Michaels, S., O’Connor, M. C., Hall, M. W., & Resnick L. B. (2010). Accountable Talk Sourcebook: For Classroom That Works. University of Pittsburgh Institute for Learning. Retrieved on May 1, 2016, from http://ifl.pitt.edu/index.php/download/index/ats.
  28. Noroozi, O., Teasley, S., Biemans, H. A., Weinberger, A., & Mulder, M. (2013). Facilitating learning in multidisciplinary groups with transactive CSCL scripts. International Journal of Computer-Supported Collaborative Learning, 8(2), 189–223.CrossRefGoogle Scholar
  29. Oehl, M., & Pfister, H. R. (2010). E-collaborative knowledge construction in chat environments. E-Collaborative Knowledge Construction: Learning from Computer-Supported and Virtual Environments, 54–72.Google Scholar
  30. Papadopoulos, P. M., Demetriadis, S., & Weinberger, A. (2013). ‘Make it explicit!’: improving collaboration through increase of script coercion. Journal of Computer Assisted Learning, 29(4), 383–398.CrossRefGoogle Scholar
  31. Preece, J., Sharp, H., & Rogers, Y. (2015). Interaction design: beyond human-computer interaction. John Wiley & Sons.Google Scholar
  32. Resnick, L. B., Michaels, S., & O’Connor, C. (2010). How (well structured) talk builds the mind. In R. Sternberg & D. Preiss (Eds.), From genes to context: new discoveries about learning from educational research and their applications (pp. 163–194). New York: Springer.Google Scholar
  33. Rus, V., D’Mello, S., Hu, X., & Graesser, A. (2013). Recent advances in conversational intelligent tutoring systems. AI Magazine, 34(3), 42–54.Google Scholar
  34. Sionti, M., Ai, H., Rosé, C. P., & Resnick, L. (2012). A framework for analyzing development of argumentation through classroom discussions. In N. Pinkwart & B. McLaren (Eds.), Educational Technologies for Teaching Argumentation Skills (pp. 28–55). Bentham Science Publishers.Google Scholar
  35. Slavin, R. E. (1992). When and why does cooperative learning increase achievement? Theoretical and empirical perspectives. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups. The theoretical anatomy of group learning (pp. 145–173). Cambridge: Cambridge University Press.Google Scholar
  36. Sohmer, R., Michaels, S., O’Connor, M. C., & Resnick, L. (2009). Guided construction of knowledge in the classroom. In B. Schwarz, T. Dreyfus, & R. Hershkowitz (Eds.), Transformation of knowledge through classroom interaction (pp. 105–129). New York: Taylor and Francis.Google Scholar
  37. Stahl, G. (2015). Computer-supported academically productive discourse. Socializing intelligence through academic talk and dialogue, 213–224. Retrieved on May 1, 2016 from http://gerrystahl.net/pub/lrdc2015.pdf.
  38. Stahl, G., & Rosé, C. P. (2011). Group cognition in online teams. In E. Salas & S. M. Fiore (Eds.), Theories of team cognition: cross-disciplinary perspectives. New York, NY: Taylor & Francis.Google Scholar
  39. Stahl, G., Cress, U., Ludvigsen, S., & Law, N. (2014). Dialogic foundations of CSCL. International Journal of Computer-Supported Collaborative Learning, 9(2), 117.CrossRefGoogle Scholar
  40. Tegos, S. (2016). Web-based conversational agents for collaborative learning support (Unpublished doctoral dissertation). Aristotle University of Thessaloniki, Thessaloniki, Greece.Google Scholar
  41. Tegos, S., & Demetriadis, S. (in press). Conversational agents improve peer learning through building on prior knowledge. Educational Technology & Society.Google Scholar
  42. Tegos, S., Demetriadis, S., & Karakostas, A. (2014). Conversational Agent to Promote Students’ Productive Talk: The Effect of Solicited vs. Unsolicited Agent Intervention. In IEEE 14th International Conference on Advanced Learning Technologies (pp. 72–76).Google Scholar
  43. Tegos, S., Demetriadis, S., & Karakostas, A. (2015). Promoting academically productive talk with conversational agent interventions in collaborative learning settings. Computers & Education, 87, 309–325.CrossRefGoogle Scholar
  44. Vogel, F., Wecker, C., Kollar, I., & Fischer, F. (2016). Socio-Cognitive Scaffolding with Computer-Supported Collaboration Scripts: a Meta-Analysis. Educational Psychology Review, 1–35.Google Scholar
  45. Webb, N. M. (2009). The teacher’s role in promoting collaborative dialogue in the classroom. British Journal of Educational Psychology, 79(1), 1–28.CrossRefGoogle Scholar
  46. Weinberger, A. (2011). Principles of transactive computer-supported collaboration scripts. Nordic Journal of Digital Literacy, 6(3), 189–202.Google Scholar
  47. Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46(1), 71–95.CrossRefGoogle Scholar
  48. Weinberger, A., Stegmann, K., & Fischer, F. (2007). Knowledge convergence in collaborative learning: concepts and assessment. Learning and Instruction, 17(4), 416–426.CrossRefGoogle Scholar
  49. Wolf, M. K., Crosson, A. C., & Resnick, L. B. (2005). Accountable talk in reading comprehension instruction. Regents of the University of California.Google Scholar

Copyright information

© International Society of the Learning Sciences, Inc. 2016

Authors and Affiliations

  • Stergios Tegos
    • 1
  • Stavros Demetriadis
    • 1
  • Pantelis M. Papadopoulos
    • 2
  • Armin Weinberger
    • 3
  1. 1.School of InformaticsAristotle University of ThessalonikiThessalonikiGreece
  2. 2.Centre for Teaching Development and Digital MediaAarhus UniversityAarhusDenmark
  3. 3.Department of Educational TechnologySaarland UniversitySaarbrückenGermany

Personalised recommendations

Cite article

Buy options