An Evaluation of the Effectiveness of Just-In-Time Hints

Abstract

The present study evaluates the effectiveness of Just-In-Time Hints (JITs) by testing two competing hypotheses about learning from errors. The tutor-remediation hypothesis predicts that students learn best when a tutoring system immediately explains why an entry is incorrect. The self-remediation hypothesis predicts that learning is maximized when learners attempt to correct their own errors. The Cognitive Tutor was used to test these hypotheses because it offers both JITs, which map onto the tutor-remediation hypothesis, and flag feedback, which maps onto the self-remediation hypothesis. To evaluate the effectiveness of JITs, we conducted a naturalistic experiment where learning from older versions of the software, which did not include specific JITs, was contrasted with a later version that included the JITs. The results suggest JITs reduced the frequency of errors; however, this observation was qualified by an aptitude-treatment interaction whereby high- and low-prior knowledge students differentially benefited from JIT availability.