Abstract
Encouraging teachers to reflect on their instructional practices and course design has been shown to be an effective means of improving instruction and student learning. However, the process of encouraging reflection is difficult; reflection requires quality data, thoughtful analysis, and contextualized interpretation. Because of this, research on and the practice of reflection is often limited to pre-service training or short professional development cycles. This study explores how natural language processing, deep-learning methods can be used to support continuous teacher reflection by facilitating data collection and analysis in any instructional setting that includes ample linguistic and assessment material. Data was collected from an existing introductory undergraduate biology course. A Bidirectional Long-Short Term Memory network was trained to predict assessment item difficulty and tasked with assigning difficulty to recorded lectures. Comparison with the instructor’s perceptions of lecture material difficulty suggested the model was highly reliable at predicting difficult lecture material. We discuss how this model could be expanded into an AI toolkit meant to aid in teacher reflection on their practices and curriculum.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
Data from this project is available by request to the corresponding author.
Code Availability
All R code used in this study is available upon request to the corresponding author.
References
Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education, 26, 600–614. https://doi.org/10.1007/s40593-016-0105-0
Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35(5), 463–482. https://doi.org/10.1016/S0883-0355(02)00004-6
Bokhove, C., & Downey, C. (2018). Automated generation of ‘good enough’ transcripts as a first step to transcription of audio-recorded data. Methodological Innovations, 11(2), 205979911879074. https://doi.org/10.1177/2059799118790743
Charpiat, G., Girard, N., Felardos, L., & Tarabalka, Y. (2019). Input similarity from the neural network perspective. Advances in Neural Information Processing Systems, 32(NeurIPS), 1–10
Chau, H., Labutov, I., Thaker, K., He, D., & Brusilovsky, P. (2021). Automatic Concept Extraction for Domain and Student Modeling in Adaptive Textbooks. International Journal of Artificial Intelligence in Education, 31(4), https://doi.org/10.1007/s40593-020-00207-1
Chollet, F., & Allaire, J. (2017). R Interface to Keras. GitHub. https://github.com/rstudio/keras
Cornford, I. R. (2002). Reflective teaching: Empirical research findings and some implications for teacher education. Journal of Vocational Education and Training, 54(2), 235. https://doi.org/10.1080/13636820200200196
Council, N. R. (2001). Knowing What Students Know (J. W. Pellegrino, N. Chudowsky, & R. Glaser (eds.); 1st ed.). The National Academies Press. https://doi.org/10.17226/10019
Escamilla, I. M., & Meier, D. (2018). The Promise of Teacher Inquiry and Reflection: Early Childhood Teachers as Change Agents. Studying Teacher Education, 14(1), 3–21. https://doi.org/10.1080/17425964.2017.1408463
Feng, M., & Heffernan, N. T. (2005). Informing Teachers Live about Student Learning: Reporting in the Assistment System. Tech Inst Cognition and Learning, 3(508), 1–14
Feng, S., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., & Hovy, E. (2021). A Survey of Data Augmentation Approaches for NLP. 968–988. https://doi.org/10.18653/v1/2021.findings-acl.84
Geden, M., Emerson, A., Carpenter, D., Rowe, J., Azevedo, R., & Lester, J. (2020). Predictive Student Modeling in Game-Based Learning Environments with Word Embedding Representations of Reflection. International Journal of Artificial Intelligence in Education, 1–23. https://doi.org/10.1007/s40593-020-00220-4
Hatton, N., & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching and Teacher Education, 11(1), 33–49. https://doi.org/10.1016/0742-051x(94)00012-U
Jensen, J., Holt, E. A., Sowards, J. B., Ogden, H., T., & West, R. E. (2018). Investigating Strategies for Pre-Class Content Learning in a Flipped Classroom. Journal of Science Education and Technology, 27(6), 523–535. https://doi.org/10.1007/s10956-018-9740-6
Jensen, J., Kummer, T., & Banjoko, A. (2013). Assessing the Effects of Prior Conceptions on Learning Gene Expression. Journal of College Science Teaching, 42(4), 82–91
Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the Test… or Testing to Teach: Exams Requiring Higher Order Thinking Skills Encourage Greater Conceptual Understanding. Educational Psychology Review, 26(2), 307–329. https://doi.org/10.1007/s10648-013-9248-9
Jensen, J., & Lawson, A. (2011). Effects of collaborative group composition and Inquiry instruction on reasoning gains and Achievement in undergraduate biology. CBE Life Sciences Education, 10(1), 64–73. https://doi.org/10.1187/cbe.10-07-0089
Khoshsima, K., & Nosratinia, M. (2019). Inspecting the Prospect of Augmenting Classroom Management by Reflective Teaching and Use of Motivational Strategies. International Journal of Applied Linguistics and English Literature, 8(1), 93–103. https://www.journals.aiac.org.au/index.php/IJALEL/article/view/5250
Knight, P., Tait, J., & Yorke, M. (2006). The professional learning of teachers in higher education. Studies in Higher Education, 31(3), 319–339. https://doi.org/10.1080/03075070600680786
Kummer, T. A., Whipple, C. J., & Jensen, J. L. (2016). Prevalence and Persistence of Misconceptions in Tree Thinking †. Journal of Microbiology & Biology Education, 17(3), 389–398. https://doi.org/10.1128/jmbe.v17i3.1156
Lee, J. H., & Cha, K. W. (2020). An Analysis of the Errors in the Auto-Generated Captions of University Commencement Speeches on YouTube Jeong-Hwa. Journal of Asia TEFL, 17(1), 143–159. https://doi.org/10.18823/asiatefl.2020.17.2.10.463
Leitner, P., Khalil, M., & Ebner, M. (2017). Teaching and Learning Analytics to support Teacher Inquiry: A Systematic Literature Review. In Learning Analytics: Fundaments, Applications, and Trends, Studies in Systems, Decision and Control (Vol. 94, Issue January). https://doi.org/10.1007/978-3-319-52977-6
Liaqat, N. (2017). Reflective Practices: a Means To Teacher Development. Asia Pacific Journal of Contemporary Education and Communication Technology, ISSN(3), 2205–6181
Loughran, J. J. (2002). Effective reflective practice in search of meaning in learning about teaching. Journal of Teacher Education, 53(1), 33–43. https://doi.org/10.1177/0022487102053001004
Messick, S. (1995). Validity of Psychological Assessment. American Psychologist, 50(9), 741–749. http://psycnet.apa.org/journals/amp/50/9/741.pdf &uid=1996-10004-001&db=PA
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. 1st International Conference on Learning Representations, ICLR 2013 - Workshop Track Proceedings, 1–12
Mislevy, R. (2005). Evidence-Centered Assessment Design: Layers, Structures, and Terminology. Principled Assessment Designs for Inquiry Technical Report 9, 9(July), 46. http://padi.sri.com/downloads/TR9_ECD.pdf
Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229. https://doi.org/10.1111/bjet.12273
Oymak, S., & Soltanolkotabi, M. (2020). Toward Moderate Overparameterization: Global Convergence Guarantees for Training Shallow Neural Networks. IEEE Journal on Selected Areas in Information Theory, 1(1), 84–105. https://doi.org/10.1109/jsait.2020.2991332
Ozogul, G., Karlin, M., & Ottenbreit-Leftwich, A. (2018). Preservice Teacher Computer Science Preparation: A Case Study of an Undergraduate Computer Education Licensure Program. Jl of Technology and Teacher Education, 26(3), 375–409
Peercy, M. M., Sharkey, J., Baecher, L., Motha, S., & Varghese, M. (2019). Exploring TESOL teacher educators as learners and reflective scholars: A shared narrative inquiry. TESOL Journal, 10(4), 1–16. https://doi.org/10.1002/tesj.482
Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248. https://doi.org/10.1111/bjet.12207
Saye, J., & Brush, T. (2002). The use of embedded scaffolds with hypermedia-supported student-centered learning. Journal of Educational Multimedia and Hypermedia, 1(2), 1–12. http://www.editlib.org/p/8439
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021). Text Data Augmentation for Deep Learning. In Journal of Big Data (Vol. 8, Issue 1). Springer International Publishing. https://doi.org/10.1186/s40537-021-00492-0
van Leeuwen, A., Knoop-Van Campen, C. A. N., Molenaar, I., & Rummel, N. (2021). How teacher characteristics relate to how teachers use dashboards: Results from two case studies in k–12. Journal of Learning Analytics, 8(2), 6–21. https://doi.org/10.18608/JLA.2021.7325
Wallach, H. M. (2006). Topic Modeling: Beyond Bag-of-Words. ICML ’06: Proceedings of the 23rd International Conference on Machine Learning, 977–984. https://doi.org/10.1145/1143844.1143967
Warr, M., & Mishra, P. (2021). Integrating the discourse on teachers and design: An analysis of ten years of scholarship. Teaching and Teacher Education, 99, 103274. https://doi.org/10.1016/j.tate.2020.103274
Xie, Z., Wang, S. I., Li, J., Daniel, L., Nie, A., Jurafsky, D., & Ng, A. Y. (2017). Data noising as smoothing in neural network language models. ICLR 2017, 1–12
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 107–115. https://doi.org/10.1145/3446776
Funding
This study was support by the Indiana University Instructional Systems Technology Kemp Research Grant.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts
All authors declare that they have no competing or conflicting interest in regards to any research presented within this publication.
Ethics approval
This research was conducted under supervision by and with approval from the Institutional Review Board of Brigham Young University.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Phillips, T.M., Saleh, A. & Ozogul, G. An AI toolkit to support teacher reflection. Int J Artif Intell Educ 33, 635–658 (2023). https://doi.org/10.1007/s40593-022-00295-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40593-022-00295-1