When Does Disengagement Correlate with Learning in Spoken Dialog Computer Tutoring?
We investigate whether an overall student disengagement label and six different labels of disengagement type are predictive of learning in a spoken dialog computer tutoring corpus. Our results show first that although students’ percentage of overall disengaged turns negatively correlates with the amount they learn, the individual types of disengagement correlate differently with learning: some negatively correlate with learning, while others don’t correlate with learning at all. Second, we show that these relationships change somewhat depending on student prerequisite knowledge level. Third, we show that using multiple disengagement types to predict learning improves predictive power. Overall, our results suggest that although adapting to disengagement should improve learning, maximizing learning requires different system interventions depending on disengagement type.
Keywordstypes of disengagement learning correlations spoken dialog computer tutors manual annotation natural language processing
Unable to display preview. Download preview PDF.
- 2.Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Merheranian, H., Fisher, D., Barto, A., Mahadevan, S., Woolf, B.: Repairing disengagement with non-invasive interventions. In: Proc. Artificial Intelligence in Education (AIED), pp. 195–202 (2007)Google Scholar
- 4.Beck, J.: Engagement tracking: using response times to model student disengagement. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED), Amsterdam, pp. 88–95 (2005)Google Scholar
- 6.Forbes-Riley, K., Litman, D.: Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor. Speech Communication (2011) (in press)Google Scholar
- 7.Forbes-Riley, K., Litman, D.: Annotating disengagement for spoken dialogue computer tutoring. In: D’Mello, S., Calvo, R. (eds.) Affect and Learning Technologies (to appear, 2011)Google Scholar
- 8.Jordan, P., Hall, B., Ringenberg, M., Cui, Y., Rose, C.: Tools for authoring a dialogue agent that participates in learning studies. In: Proc. Artificial Intelligence in Education (2007)Google Scholar
- 10.Murray, R.C.: vanLehn, K.: Effects of dissuading unnecessary help requests while providing proactive help. In: Proc. of the International Conference on Artificial Intelligence in Education, pp. 887–889 (2005)Google Scholar