Skip to main content

Identifying Effective Moves in Tutoring: On the Refinement of Dialogue Act Annotation Schemes

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 8474)

Abstract

The rich natural language dialogue that is exchanged between tutors and students has inspired many successful lines of research on tutorial dialogue systems. Yet, today’s tutorial dialogue systems do not regularly achieve the same level of student learning gain as has been observed with expert human tutors. Implementing models directly informed by, and even machine-learned from, human-human tutorial dialogue is highly promising. With this goal in mind, this paper makes two contributions to tutorial dialogue systems research. First, it presents a dialogue act annotation scheme that is designed specifically to address a common weakness within dialogue act tag sets, namely, their dominance by a single large majority dialogue act class. Second, using this new fine-grained annotation scheme, the paper describes important correlations uncovered between tutor dialogue acts and student learning gain within a corpus of tutorial dialogue for introductory computer science. These findings can inform the design of future tutorial dialogue systems by suggesting ways in which systems can adapt at a fine-grained level to student actions.

Keywords

  • Learning Gain
  • Annotation Scheme
  • Tutorial Dialogue
  • Evaluative Question
  • Dialogue Event

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-07221-0_24
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-07221-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. VanLehn, K., et al.: When Are Tutorial Dialogues More Effective Than Reading? Cog. Sci. 31(1), 3–62 (2007)

    CrossRef  MathSciNet  Google Scholar 

  2. Bloom, B.S.: The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educ. Res. 13(6), 4–16 (1984)

    CrossRef  Google Scholar 

  3. Chi, M.T., et al.: Learning from human tutoring. Cog. Sci. 25(4), 471–533 (2001)

    CrossRef  Google Scholar 

  4. Lepper, M.R., et al.: Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. Computers as Cognitive Tools 1993, 75–105 (1999)

    Google Scholar 

  5. Graesser, A.C., et al.: Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cog. Psy. 9(6), 495–522 (1995)

    CrossRef  Google Scholar 

  6. Chi, M., VanLehn, K., Litman, D.: Do Micro-Level Tutorial Decisions Matter: Applying Reinforcement Learning to Induce Pedagogical Tutorial Tactics. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part I. LNCS, vol. 6094, pp. 224–234. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  7. Dzikovska, M.O., Steinhauser, N.B., Moore, J.D., Campbell, G.E., Harrison, K.M., Taylor, L.S.: Content, social, and metacognitive statements: An empirical study comparing human-human and human-computer tutorial dialogue. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 93–108. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  8. Kumar, R., Ai, H., Beuth, J.L., Rosé, C.P.: Socially Capable Conversational Tutors Can Be Effective in Collaborative Learning Situations. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part I. LNCS, vol. 6094, pp. 156–164. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  9. D’Mello, S.K., et al.: A Motivationally Supportive Affect-Sensitive AutoTutor. New Perspectives on Affect and Learning Tech. 3, 113–126 (2011)

    CrossRef  Google Scholar 

  10. Chen, L., et al.: Exploring Effective Dialogue Act Sequences in One-on-one Computer Science Tutoring Dialogues. In: Tetreault, J., et al. (eds.) Proc. 6th BEA Work., Portland, USA, pp. 65–75. Assoc. for Comp. Ling (2011)

    Google Scholar 

  11. Stellan, Ohlsson, o.: Beyond the Code-and-count Analysis of Tutoring Dialogues. In: R, Luckin, o. (eds.) Proc. 13th Int. Conf. AIED, Los Angeles, USA, vol. 158, pp. 349–356. IOS (2007)

    Google Scholar 

  12. Forbes-Riley, K., Litman, D.J.: Adapting to Student Uncertainty Improves Tutoring Dialogues. In: Vania, Dimitrova, o. (eds.) Proc. 14th Int. Conf. AIED, Brighton, United Kingdom, pp. 33–40. IOS (2009)

    Google Scholar 

  13. Cohen, P.A., et al.: Educational Outcomes of Tutoring: A Meta-analysis of Findings. Am. Educ. Res. J. 19(2), 237–248 (1982)

    CrossRef  Google Scholar 

  14. D’Mello, S.K., et al.: Mining Collaborative Patterns in Tutorial Dialogues. J. EDM 2(1), 1–37 (2010)

    Google Scholar 

  15. Chu-Carroll, J.: A Statistical Model for Discourse Act Recognition in Dialogue Interactions. In: Chu-Carroll, J., Green, N. (eds.) AAAI Spring Symp.: Applying Machine Learning to Discourse Processing, Pan Alto, USA, vol. 1996, pp. 12–17. AAAI Press (1998)

    Google Scholar 

  16. Litman, D.J., Forbes-Riley, K.: Correlations between dialogue acts and learning in spoken tutoring dialogues. Nat. Lang. Eng. 12(2), 161–176 (2006)

    CrossRef  Google Scholar 

  17. Mitchell, C.M., et al.: Recognizing Effective and Student-Adaptive Tutor Moves in Task-Oriented Tutorial Dialogue. In: Youngblood, M.G., McCarthy, P.M. (eds.) Proc. 25th Int. FLAIRS Conf., Marco Island, Florida, pp. 450–455. AAAI Press (2009)

    Google Scholar 

  18. Person, N.K., et al.: The Dialog Advancer Network: A Conversation Manager for AutoTutor. In: Gauthier, G., et al. (eds.) Proc. ITS Work. Modeling Human Teaching Tactics and Strategies, Montreal, Canada, pp. 86–92. Springer (2000)

    Google Scholar 

  19. Core, M.G., Allen, J.F.: Coding Dialogs with the DAMSL Annotation Scheme. In: Proc. 1997 AAAI Fall Symp.: Communicative Action in Humans and Machines, Providence, USA, pp. 28–35. AAAI (1997)

    Google Scholar 

  20. Landis, J.R., Koch, G.G.: The Measurement of Observer Agreement for Categorical Data Data for Categorical of Observer Agreement The Measurement. Biometrics 33(1), 159–174 (1977)

    CrossRef  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Vail, A.K., Boyer, K.E. (2014). Identifying Effective Moves in Tutoring: On the Refinement of Dialogue Act Annotation Schemes. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds) Intelligent Tutoring Systems. ITS 2014. Lecture Notes in Computer Science, vol 8474. Springer, Cham. https://doi.org/10.1007/978-3-319-07221-0_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-07221-0_24

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-07220-3

  • Online ISBN: 978-3-319-07221-0

  • eBook Packages: Computer ScienceComputer Science (R0)