Estimating the Effect of Web-Based Homework
Traditional studies of intelligent tutoring systems have focused on their use in the classroom. Few have explored the advantage of using ITS as a web-based homework (WBH) system, providing correctness-only feedback to students. A second underappreciated aspect of WBH is that teachers can use the data to more efficiently review homework. Universities across the world are employing these WBH systems but there are no known comparisons of this in K12. In this work we randomly assigned 63 thirteen and fourteen year olds to either a traditional homework condition (TH) involving practice without feedback or a WBH condition that added correctness feedback at the end of a problem and the ability to try again. All students used ASSISTments, an ITS, to do their homework but we ablated all of the intelligent tutoring aspects of hints, feedback messages and mastery learning as appropriate to the two practice conditions. We found that students learned reliably more in the web-based homework condition and with an effect size of 0.56. Additionally, teacher use of the homework data lead to a more robust and systematic review of the homework. Future work will further examine modifications to WBH to further improve learning from homework and the role of WBH in formative assessment.
Keywordsintelligent tutoring systems immediate feedback homework effect size formative assessment
Unable to display preview. Download preview PDF.
- 1.Black, P., Wiliam, D.: Inside the black box: Raising standards through classroom assessment. Granada Learning (2006)Google Scholar
- 2.CEM (2013), http://www.cemcentre.org/evidence-based-education/effect-size-calculator (accessed January 28, 2013)
- 3.Kelly, K.: (2012), http://www.webcitation.org/6E03PhjrP contains the materials Browse, http://web.cs.wpi.edu/~nth/PublicScienceArchive/Kelly.htm to view the materials
- 5.Rochelle: The IES Grant (2013), http://ies.ed.gov/funding/grantsearch/details.asp?ID=1273 (accessed January 28, 2013)
- 6.Shute, V.: Focus on Formative Feedback. Review of Educational Research 78(1), 153–189 (2008), http://www.ets.org/Media/Research/pdf/RR-07-11.pdf CrossRefGoogle Scholar
- 7.Singh, R., Saleem, M., Pradhan, P., Heffernan, C., Heffernan, N.T., Razzaq, L., Dailey, M.D., O’Connor, C., Mulcahy, C.: Feedback during Web-Based Homework: The Role of Hints. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS, vol. 6738, pp. 328–336. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 9.VanLehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., Wintersgill, M.: The Andes physics tutoring system: Lessons Learned. International Journal of Artificial Intelligence and Education 15(3), 1–47 (2005)Google Scholar