Abstract
A web-based, collaborative distance-learning system that will allow groups of students to interact with each other remotely and with an intelligent electronic agent that will aid them in their learning has the potential for improving on-line learning. The agent would follow the discussion and interact with the participants when it detects learning trouble of some sort, such as confusion about the problem they are working on or a participant who is dominating the discussion or not interacting with the other participants. In order to recognize problems in the dialogue, we investigated conversational elements that can be utilized as predictors for effective and ineffective interaction between human students. These elements can serve as the basis for student and group models. In this paper, we discuss group interaction during collaborative learning, our representation of participant dialogue, and the statistical models we are using to determine the role being played by a participant at any point in the dialogue and the effectiveness of the group. We also describe student and group models that can be built using conversational elements and discuss one set that we built to illustrate their potential value in collaborative learning.
Similar content being viewed by others
References
J.R. Anderson (1990) The Adaptive Character of Thought Erlbaum Mahwah, NJ
J.R. Anderson C.F. Boyle A. Corbett M. Lewis (1990) ArticleTitleCognitive modeling and intelligent tutoring Artificial Intelligence 42 7–49 Occurrence Handle10.1016/0004-3702(90)90093-F
Ang J., Dhillon R., Krupski A., Shriberg E., Stolcke A. (2002). Prosody-based automatic detection of annoyance and frustration in human–computer dialog. In: Proceedings of ICSLP-2002, Denver, CO, pp. 2037–2040
Batliner A., Noth E., Buckow J., Huber R., Warnke V., Niemann H. (2001). Whence and whither prosody in automatic speech understanding: A case study. In: Proceedings ISCA Tutorial and Research Workshop on Prosody and Speech Recognition and Understanding, Red Bank, NJ
R.M. Belbin (2004) Management Teams EditionNumber2 Butterworth-Heinemann Oxford
K. Benne P. Sheats (1948) ArticleTitleFunctional roles of group members Journal of Social Issues 4 41–49
S. Bloom B. (1984) ArticleTitleThe two sigma problem: The search for methods of group instruction as effective as one-to-one tutoring Educational Researcher. 13 IssueID6 4–16 Occurrence Handle10.2307/1175554
Bosma W., André E. (2004). Exploiting emotions to disambiguate dialogue acts. In: Proceedings of the Conference on Intelligent User Interfaces, Portugal, pp. 85–92
A. Brown A. Palincsar (1989) Guided, cooperative learning and individual knowledge acquisition Resnick. Lauren B. (Eds) Knowledge, Learning and Instruction. Lawrence Erlbaum Hillsdale, NJ 393–451
S. Bull P. Brna H. Pain (1995) ArticleTitleExtending the scope of the student model User Modeling and User-Adapted Interaction 5 45–65 Occurrence Handle10.1007/BF01101801
M. Burton P. Brna R. Pilkington Clarissa. (2000) ArticleTitleA Laboratory for the Modelling of Collaboration International Journal of Artificial Intelligence in Education 11 79–105
J. Carletta (1996) ArticleTitleAssessing agreement on classification tasks: The Kappa statistic Computational Linguistics. 22 IssueID2 249–254
Chan T., Baskin A. (1988). Studying with the prince. The computer as a learning companion. In: Proceedings of the ITS-88 Conference, (Montréal, Canada), pp. 194–200
M. Chi (1996) ArticleTitleConstructing self-explanations and scaffolded explanations in tutoring Applied Cognitive Psychology, 10 S33–S49 Occurrence Handle1431973 Occurrence Handle10.1002/(SICI)1099-0720(199611)10:7<33::AID-ACP436>3.0.CO;2-E
A. Collins J.S. Brown S. Newman (1989) Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics L.B. Resnick (Eds) Knowing, learning and Instruction: Essays in Honor of Robert Glaser. Lawrence Erlbaum Hillsdale, NJ 453–494
A. Collins (1991) Cognitive Apprenticeship and Instructional Technology L. Idol B.F. Jones (Eds) Educational Values and Cognitive Instruction. Lawrence Erlbaum Hillsdale, NJ 121–138
Corbett A.T., Anderson J.R. (1989). ‘Feedback timing and student control in the LISP Intelligent Tutoring System’. In: Proceedings of the Fourth International Conference on AI and Education, pp. 64–72
M. Czarkowski J. Kay (2000) Bringing scrutability to adaptive hypertext teaching Gauthier. Frasson. Lehn. Van (Eds) Intelligent Tutoring Systems. Springer Berlin 423–432
Dietterich T.G. (2002). Machine learning for sequential data: A review. In: Caelli T. (ed). Lecture Notes in Computer Science. 2396, 15–30
R.M. Felder G.N. Felder E.J. Dietz (1998) ArticleTitleA longitudinal study of engineering student performance and retention. V. comparisons with traditionally-taught students Journal of Engineering Education. 87 IssueID4 469–480
F. Flores M. Graves B. Hartfield T. Winograd (1988) ArticleTitleComputer systems and the design of organizational interaction ACM Transactions on Office Information Systems. 6 IssueID2 153–172 Occurrence Handle10.1145/45941.45943
Forbes-Riley K., Litman D.J. (2004). Predicting emotion in spoken dialogue from multiple knowledge sources. In: Proceedings of HLT-NAACL 2004, Boston, MA, pp. 201–208
Goodman B., Iorizzo L. (2000). Learning with reflection: Project PRAXIS. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL
Gaimari R., Soller A.L. (1996). Collaborative learning in an intelligent tutoring system. In: Proceedings of the 1996 Conference on Computer-Supported Cooperative Work Workshop on Approaches for Distributed Learning through Computer-Supported Collaborative Learning, Cambridge, MA
Goodman B., Soller A., Linton F., Gaimari R. (1996). [Videotaped study: 3 groups of 4–5 students each solving software system design problems using Object Modeling Technique during a one week course at The MITRE Institute]. Unpublished raw data
Goodman B., Soller A., Linton F., Gaimari R. (1997). Encouraging student reflection and articulation using a learning companion. In: Proceedings of the AI-ED 97 World Conference on Artificial Intelligence in Education, Kobe, Japan, pp. 151–158
B. Goodman A. Soller F. Linton R. Gaimari (1998) ArticleTitleEncouraging student reflection and articulation using a learning companion International Journal of Artificial Intelligence in Education. 9 IssueID3–4 237–255
B. Goodman M. Geier L. Haverty F. Linton R. McCready (2001) A framework for asynchronous collaborative learning and problem-solving J. Moore C.L. Redfield W.L. Johnson (Eds) Artificial Intelligence in Education. IOS Press Amsterdam 188–199
Goodman B., Hitzeman J., Linton F., Ross H. (2003a). Towards intelligent agents for collaborative learning: recognizing the role of dialogue participants. In: Proceedings of the International Conference on User Modeling, Johnstown, PA, pp. 363–367
B. Goodman R. Gaimari J. Zarrella F. Linton et al. (2003) An empirical analysis of learner discourse H.U. Hoppe (Eds) Proceedings of the 10th International Conference on Artificial Intelligence in Education: Artificial Intelligence in Education. IOS Press Amsterdam 416–418
Hmelo-Silver C.E. (2002). Collaborative ways of knowing: Issues in facilitation. In: Stahl G. (ed). Proceedings of CSCL 2002, Boulder, CO, pp. 199–208
Hillard D., Ostendorf M., Shriberg E. (2003). Detection of agreement vs. disagreement in meetings: Training with unlabeled data. In: Proceedings of HLT-NAACL Conference, Edmonton, Canada, pp. 34–36
Hirschberg J., Litman D.J., Swerts M. (2000). Generalizing prosodic prediction of speech recognition errors. In: Proceedings of the 6th International Conference of Spoken Language Processing (ICSLP-2000), Beijing, China, pp. 615–618
S. Jarboe (1996) Procedures for enhancing group decision making B. Hirokawa M. Poole (Eds) Communication and Group Decision Making. Sage Publications Thousand Oaks, CA 345–383
D. Jonassen S. Land (2000) Theoretical Foundations of Learning Environments Erlbaum Mahwah, NJ
D. Johnson R. Johnson E.J. Holubec (1990) Circles of Learning: Cooperation in the Classroom Interaction Book Company Edina, MN
Jokinen K., Hurtig T., Hynnä K., Kanto K., Kaipainen M., Kerminen A. (2001). Self-organizing dialogue management. In: Proceedings of the 2nd Workshop on Neural Networks and Natural Language Processing, Natural Language Pacific Rim Symposium (NLPRS), Tokyo, Japan, pp. 78–85
S. Katz J. Aronis C. Creitz (1999) Modeling pedagogical interactions with machine learning S.P. Lajoie M. Vivet (Eds) Artificial Intelligence in Education. IOS Press Amsterdam 543–550
S. Katz G. O’Donnell H. Kay (2000) ArticleTitleapproach to analyzing the role and structure of reflective dialogue International Journal of Artificial Intelligence in Education 11 320–343
Kay J. (1998). Scrutable User Modeling Shell for User-adapted Interaction. Ph.D. Thesis Basser Department of Computer Science. University of Sydney, Australia
C. Kneser R. Pilkington T. Treasure-Jones (2001) ArticleTitleThe tutor’s role: An investigation of the power of exchange structure analysis to identify different roles in CMC seminars International Journal of Artificial Intelligence in Education 12 63–84
A. Lesgold S. Katz L. Greenberg E. Hughes G. Eggan (1992) Extensions of intelligent tutoring paradigms to support collaborative learning S. Dijkstra H. Krammer J. Merrienboer Particlevan (Eds) Instructional Models in Computer-Based Learning Environments. Springer Berlin 291–311
Linton F., Goodman B., Gaimari R., Zarrella J., Ross H. (2003). Student modeling for an intelligent agent in a collaborative learning environment. In: Proceedings of the International Conference on User Modeling, Johnstown, PA, pp. 342–352
Litman D.J., Hirschberg J., Swerts M. (2000). Predicting automatic speech recognition performance using prosodic cues. In: Proceedings of the First Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL’00), Seattle, WA, pp. 218–225
Lund K., Baker M., Baron M. (1996). Modeling dialogue and beliefs as a basis for generating guidance in a CSCL environment. In: Proceedings of the ITS-96 Conference, Montreal, pp. 206–614
M. McManus R. Aiken (1995) ArticleTitleMonitoring computer-based problem-solving Journal of Artificial Intelligence in Education. 6 IssueID4 307–336
R. Morales H. Pain T. Conlon (2000) Understandable learner models for a sensorimotor control task Gauthier. Frasson. Lehn. Van (Eds) Intelligent Tutoring Systems. Springer Berlin 222–231
M. Mühlenbrock F. Tewissen H.U. Hoppe (1998) ArticleTitleA framework system for intelligent support in open distributed learning environments International Journal of Artificial Intelligence in Education. 9 256–274
Rabiner L.R. (1989). A tutorial on hidden Markov models. In: Proceedings of the IEEE, Vol. 77, pp. 257–286
Rehg W., McBurney P., Parsons S. (2001). Computer decision support systems for public argumentation: criteria for assessment. In: Hansen H.V., Tindale C.W., Blair J.A., Johnson R.H. (ed). Argumentation and its Applications. Proceedings of the Fourth Biennial Conference of the Ontario Society for the Study of Argumentation, OSSA 2001
J. Rumbaugh M. Blaha W. Premerlani F. Eddy W. Lorensen (1991) Object-Oriented modeling and design Prentice Hall Englewood Cliffs, NJ
Samuel K., Carberry S., Vijay-Shanker K. (1998). Dialogue act tagging with transformation-based learning. In: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1150–1156
J. Searle (1969) DIALOGUE ACTS: An Essay in the Philosophy of Language Cambridge University Press London
J. Searle (1996) A taxonomy of illocutionary acts A. Martinich (Eds) The Philosophy of Language EditionNumber3 Oxford University Press New York
E. Shriberg R. Bates A. Stolcke P. Taylor D. Jurafsky K. Ries N. Coccaro R. Martin M. Meteer C. Van Ess-Dykema (1998) ArticleTitleCan prosody aid the automatic classification of dialog acts in conversational speech Language and Speech. 41 IssueID3–4 439–487
Shriberg E., Stolcke A., Baron D. (2001). Can prosidy aid the automatic processing of multi-party meetings? Evidence from predicting punctuation, disfluencies, and overlapping speech. In: Proceedings of the ISCA Tutorial and Research Workshop on Prosody in Speech Recognition and Understanding, Red Bank, NJ, pp. 139–146
V.J. Shute (1990) Rose Garden Promises of Intelligent Tutoring Systems: Blossom or Thorn? Presented at the Space Operations and Research (SOAR) Symposium Albuquerque NM
Singley M.K., Singh M., Fairweather P., Farrell R., Swerling S. (2000). Algebra jam: Supporting teamwork and managing roles in a collaborative learning environment. In: Proceedings of Computer Supported Collaborative Work 2000, Philadelphia, PA, pp. 145–154
Soller A. (2002). Computational Analysis of Knowledge Sharing in Collaborative Distance Learning. Doctoral Dissertation. Department of Computer Science, University of Pittsburgh
Soller A. (2003). Personal communication
A. Soller (2004) ArticleTitleComputational modeling and analysis of knowledge sharing in collaborative distance learning User Modeling and User-Adapted Interaction. 14 IssueID4 351–381 Occurrence Handle10.1023/B:USER.0000043436.49168.3b
Soller A.L. (1997a). An intelligent CSCL communication interface. In: Proceedings of AI-ED 97 Workshop IV: Collaborative Learning/Working Support System with Networking, Kobe, Japan, pp. 94–95
Soller A.L. (1997b). Back to the drawing board: Explaining causal relationships in an argumentation-based ITS. In: Proceedings of the AI-ED 97 World Conference on Artificial Intelligence in Education, Kobe, Japan, pp. 231–238
Soller A., Goodman B., Linton F., Gaimari R. (1998). Promoting effective peer interaction in an intelligent collaborative learning environment. In: Proceedings of the Fourth International Conference on Intelligent Tutoring Systems (ITS 98), San Antonio, TX, pp. 186–195
A. Soller A. Lesgold (2000) Modeling the process of collaborative learning. International workshop on new technologies in Collaborative Learning Awaji-Yumebutai Japan
Soller A., Lesgold A., Linton F., Goodman B. (1999a). What makes peer interaction effective? Modeling effective communication in an intelligent CSCL. In: Proceedings of the 1999 AAAI Fall Symposium: Psychological Models of Communication in Collaborative Systems, Cape Cod, MA, pp. 116–123
Soller A., Lesgold A. (1999). Analyzing peer dialogue from an active learning perspective. Proceedings of the AI-ED 99 Workshop: Analysing Educational Dialogue Interaction: Towards Models that Support Learning, LeMans, France, pp. 63–71
Soller A., Linton F., Goodman B., Lesgold A. (1999b). Toward intelligent analysis and support of collaborative learning interaction. In: Proceedings of the Ninth International Conference on Artificial Intelligence in Education, LeMans, France, pp. 75–82
A. Soller J. Wiebe A. Lesgold (2002) A machine learning approach to assessing knowledge sharing during collaborative learning activities G. Stahl (Eds) Proceedings of CSCL 2002. Boulder CO 128–137
A.L. Soller (2001) ArticleTitleSupporting social interaction in an intelligent collaborative learning system International Journal of Artificial Intelligence in Education. 12 IssueID1 40–62
Stevens R., Ikeba J., Casillas A., Palacio-Cayetano J., Clyman S. (1999). Artificial neural network-based performance assessments, Computers in Human Behavior, 15, pp. 295–313
P. Tesluk J.E. Mathieu S.J. Zaccaro M. Marks (1997) Task and aggregation issues in the analysis and assessment of team performance M.T. Brannick E. Salas C. Prince (Eds) Team Performance Assessment and Measurement. Lawrence Erlbaum Mahwah, NJ 197–224
de Vicente A., Bouwer A., Pain H. (1999). Initial impressions on using the DISCOUNT scheme. In: Proceedings of the Workshop on Analysing Educational Dialogue Interaction (AIED 99 Workshop), LeMans, France, pp. 87–94
A. Waibel M. Bett M. Finke R. Stiefelhagen (1998) Meeting browser: Tracking and summarizing meetings D.E.M. Penrose (Eds) Proceedings of the Broadcast News Transcription and Understanding Workshop. Morgan Kaufmann Lansdowne, VA 281–286
Walker M.A., Wright J., Langkilde I. (2000). Using natural language processing and discourse features to identify understanding errors in a spoken dialogue system. In: Proceedings of the International Conference on Machine Learning, Stanford, CA, pp. 1111–1118
Winter M., McCalla G. (2003). An analysis of group performance in terms of the functional knowledge and teamwork skills of group members. In: Proceedings of the UM2003 Workshop on User and Group Models for Web-based Adaptive Collaborative Environments, Johnstown, PA, pp. 35–44
A. Wu R. Farrell M. Singley (2002) Scaffolding Group Learning in a Collaborative Networked Environment G. Stahl (Eds) Proceedings of CSCL 2002. Boulder CO 245–254
Author information
Authors and Affiliations
Corresponding author
Additional information
An erratum to this article is available at http://dx.doi.org/10.1007/s11257-006-9001-x.
Rights and permissions
About this article
Cite this article
Goodman, B.A., Linton, F.N., Gaimari, R.D. et al. Using Dialogue Features to Predict Trouble During Collaborative Learning. User Model User-Adap Inter 15, 85–134 (2005). https://doi.org/10.1007/s11257-004-5269-x
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11257-004-5269-x