Abstract
Low-stakes assessment has gained attention in recent years due to its link to enhancing learning effects and its essential role in learning evaluation. Unlike high-stakes assessments, low-stakes assessments have little or no consequences for learners’ academic performance, and are designed to support the feedback-oriented learning process. Providing multiple low-stakes assessments to students yields significantly greater long-term retention of knowledge and skills. However, learners may not give their best efforts when taking low-stakes assessments, which could lead to poorer learning outcomes. Using emerging technologies such as social robots in the learning environment could foster interactive learning, engagement, and motivation for learning assessments. Therefore, integrating low-stakes assessments and robots might encourage students to exert greater effort while performing learning tasks. This study aimed to discover the impacts of robot-based multiple low-stakes assessments on students’ oral presentation performance, collective efficacy, and learning attitude. A quasi-experiment was conducted in two sixth-grade classes of elementary students. The Robot-based Multiple Low-Stakes Assessment (Robot-MLSA) was randomly assigned to one class, while the Computer-based Multiple Low-Stakes Assessment (C-MLSA) was assigned to another class. The findings showed that the Robot-MLSA could enhance students’ oral presentation performance, support their collective efficacy, and improve their learning attitude toward robots. Furthermore, an in-depth discussion of students’ learning perceptions and experience is provided to explore the effectiveness of the Robot-MLSA.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Al Hakim, V. G., Yang, S. H., Liyanawatta, M., Wang, J. H., & Chen, G. D. (2022). Robots in situated learning classrooms with immediate feedback mechanisms to improve students’ learning performance. Computers & Education, 182, 104483. https://doi.org/10.1016/j.compedu.2022.104483.
Alemi, M., Meghdari, A., & Ghazisaedy, M. (2015). The impact of social robotics on L2 learners’ anxiety and attitude in english vocabulary acquisition. International Journal of Social Robotics, 7(4), 523–535. https://doi.org/10.1007/s12369-015-0286-y
Balkibekov, K., Meiirbekov, S., Tazhigaliyeva, N., & Sandygulova, A. (2016). Should robots win or lose? Robot’s losing playing strategy positively affects child learning. In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 706–711). IEEE. https://doi.org/10.1109/ROMAN.2016.7745196.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
Bloom, B. S. (1971). Handbook on formative and summative evaluation of student learning. McGraw-Hill.
Bui, H. P., & Nguyen, T. T. T. (2022). Classroom assessment and learning motivation: insights from secondary school EFL classrooms. International Review of Applied Linguistics in Language Teaching. https://doi.org/10.1515/iral-2022-0020
Burke, L. A., & Williams, J. M. (2012). Two thinking skills assessment approaches: Assessment of pupils’ thinking skills and individual thinking skills assessments. Thinking Skills and Creativity, 7(1), 62–68. https://doi.org/10.1016/j.tsc.2011.11.002
Butler, A. C., Godbole, N., & Marsh, E. J. (2013). Explanation feedback is better than correct answer feedback for promoting transfer of learning. Journal of Educational Psychology, 105(2), 290. https://doi.org/10.1037/a0031026.
Carduner, J. (2002). Using classroom assessment techniques to improve foreign language composition courses. Foreign Language Annals, 35(5), 543–553. https://doi.org/10.1111/j.1944-9720.2002.tb02722.x.
Chang, C. C., Hwang, G. J., & Chen, K. F. (2023). Fostering professional trainers with robot-based digital storytelling: A brainstorming, selection, forming and evaluation model for training guidance. Computers & Education. https://doi.org/10.1016/j.compedu.2023.104834
Chen Hsieh, J., & Lee, J. S. (2021). Digital storytelling outcomes, emotions, grit, and perceptions among EFL middle school learners: Robot-assisted versus powerpoint-assisted presentations. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2021.1969410
Creswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among five approaches. Sage.
Cross, K. P., & Angelo, T. A. (1988). Classroom Assessment Techniques: A Handbook for Faculty. The National Center for Research to Improve Postsecondary Teaching and Learning, The University of Michigan.
Engwall, O., & Lopes, J. (2020). Interaction and collaboration in robot-assisted language learning for adults. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2020.1799821
Engwall, O., Lopes, J., Cumbal, R., Berndtson, G., Lindström, R., Ekman, P., & Mekonnen, M. (2022). Learner and teacher perspectives on robot-led L2 conversation practice. ReCALL. https://doi.org/10.1017/S0958344022000027
Han, J. H., Jo, M. H., Jones, V., & Jo, J. H. (2008). Comparative study on the educational use of home robots for children. Journal of Information Processing Systems, 4(4), 159–168. https://doi.org/10.3745/JIPS.2008.4.4.159.
Hong, Z. W., Huang, Y. M., Hsu, M., & Shen, W. W. (2016). Authoring robot-assisted instructional materials for improving learning performance and motivation in EFL classrooms. Journal of Educational Technology & Society, 19(1), 337–349. https://doi.org/10.2307/jeductechsoci.19.1.337. https://www.jstor.org/stable/.
Hsia, L. H., & Sung, H. Y. (2020). Effects of a mobile technology-supported peer assessment approach on students’ learning motivation and perceptions in a college flipped dance class. International Journal of Mobile Learning and Organisation, 14(1), 99–113.
Hsiao, H. S., Lin, Y. W., Lin, K. Y., Lin, C. Y., Chen, J. H., & Chen, J. C. (2022). Using robot-based practices to develop an activity that incorporated the 6E model to improve elementary school students’ learning performances. Interactive Learning Environments, 30(1), 85–99. https://doi.org/10.1080/10494820.2019.1636090.
Hsu, T. C., & Liang, Y. S. (2021). Simultaneously improving computational thinking and foreign language learning: Interdisciplinary media with plugged and unplugged approaches. Journal of Educational Computing Research, 59(6), 1184–1207. https://doi.org/10.1177/0735633121992480.
Hwang, G. J., Yang, T. C., Tsai, C. C., & Yang, S. J. (2009). A context-aware ubiquitous learning environment for conducting complex science experiments. Computers & Education, 53(2), 402–413. https://doi.org/10.1016/j.compedu.2009.02.016.
Iio, T., Maeda, R., Ogawa, K., Yoshikawa, Y., Ishiguro, H., Suzuki, K., & Hama, M. (2019). Improvement of Japanese adults’ English speaking skills via experiences speaking to a robot. Journal of Computer Assisted Learning, 35(2), 228–245. https://doi.org/10.1111/jcal.12325.
Jaipal-Jamani, K., & Angeli, C. (2017). Effect of robotics on elementary preservice teachers’ self-efficacy, science learning, and computational thinking. Journal of Science Education and Technology, 26, 175–192. https://doi.org/10.1007/s10956-016-9663-z.
Jawad, L. F., Majeed, B. H., & ALRikabi, H. T. S. (2021). The impact of CATs on mathematical thinking and logical thinking among fourth-class scientific students. International Journal of Emerging Technologies in Learning (Online), 16(10), 194. https://doi.org/10.3991/ijet.v16i10.22515.
Konstantakis, M., Lykiardopoulou, A., Lykiardopoulou, E., Tasiouli, G., & Heliades, G. (2022). An exploratory study of Mobile-based scenarios for Foreign Language Teaching in Early Childhood. Education Sciences, 12(5), 306. https://doi.org/10.3390/educsci12050306.
Kulik, J. A., & Kulik, C. L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58(1), 79–97. https://doi.org/10.3102/00346543058001079.
Lee, S., Noh, H., Lee, J., Lee, K., Lee, G. G., Sagong, S., & Kim, M. (2011). On the effectiveness of robot-assisted language learning. ReCALL, 23(1), 25–58. https://doi.org/10.1017/S0958344010000273.
Liang, J. C., & Hwang, G. J. (2023). A robot-based digital storytelling approach to enhancing EFL learners’ multimodal storytelling ability and narrative engagement. Computers & Education, 201, 104827. https://doi.org/10.1016/j.compedu.2023.104827.
Liu, E. Z. F. (2010). Early adolescents’ perceptions of educational robots and learning of robotics. British Journal of Educational Technology, 41(3), E44–E47. https://doi.org/10.1111/j.1467-8535.2009.00944.x.
Liu, O. L., Rios, J. A., & Borden, V. (2015). The effects of motivational instruction on college students’ performance on low-stakes assessment. Educational Assessment, 20(2), 79–94. https://doi.org/10.1080/10627197.2015.1028618.
McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381–392. https://doi.org/10.3758/BRM.42.2.381.
Mercer, N., Warwick, P., & Ahmed, A. (2017). An oracy assessment toolkit: Linking research and development in the assessment of students’ spoken language skills at age 11–12. Learning and Instruction, 48, 51–60. https://doi.org/10.1016/j.learninstruc.2016.10.005.
Mertler, C. (2016). Classroom assessment: A practical guide for educators. Routledge.
Min, Q., Wang, Z., & Liu, N. (2019). Integrating a cloud learning environment into English-medium instruction to enhance non-native English-speaking students’ learning. Innovations in Education and Teaching International, 56(4), 493–504. https://doi.org/10.1080/14703297.2018.1483838.
Nomura, T., Kanda, T., & Suzuki, T. (2006). Experimental investigation into influence of negative attitudes toward robots on human–robot interaction. Ai & Society, 20(2), 138–150. https://doi.org/10.1007/s00146-005-0012-7.
Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005). When does feedback facilitate learning of words? Journal of Experimental Psychology: Learning Memory and Cognition, 31(1), 3. https://doi.org/10.1037/0278-7393.31.1.3.
Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813.
Randall, N. (2019). A survey of robot-assisted language learning (RALL). ACM Transactions on Human-Robot Interaction (THRI), 9(1), 1–36.
Roediger, I. I. I., H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. https://doi.org/10.1016/j.tics.2010.09.003.
Rosenberg-Kima, R. B., Koren, Y., & Gordon, G. (2020). Robot-supported collaborative learning (RSCL): Social robots as teaching assistants for higher education small group facilitation. Frontiers in Robotics and AI, 6, 148. https://doi.org/10.3389/frobt.2019.00148.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. https://doi.org/10.1007/BF00117714.
Schut, S., van Tartwijk, J., Driessen, E., van der Vleuten, C., & Heeneman, S. (2020). Understanding the influence of teacher–learner relationships on learners’ assessment perception. Advances in Health Sciences Education, 25(2), 441–456. https://doi.org/10.1007/s10459-019-09935-z.
Schüttpelz-Brauns, K., Hecht, M., Hardt, K., Karay, Y., Zupanic, M., & Kämmer, J. E. (2020). Institutional strategies related to test-taking behavior in low stakes assessment. Advances in Health Sciences Education, 25(2), 321–335. https://doi.org/10.1007/s10459-019-09928-y.
Serholt, S., Barendregt, W., Leite, I., Hastie, H., Jones, A., Paiva, A., & Castellano, G. (2014, August). Teachers’ views on the use of empathic robotic tutors in the classroom. In The 23rd IEEE International Symposium on Robot and Human Interactive Communication (pp. 955–960). IEEE.
Sisman, B., Gunay, D., & Kucuk, S. (2019). Development and validation of an educational robot attitude scale (ERAS) for secondary school students. Interactive Learning Environments, 27(3), 377–388. https://doi.org/10.1080/10494820.2018.1474234.
Socratous, C., & Ioannou, A. (2022). Evaluating the impact of the curriculum structure on group metacognition during collaborative problem-solving using educational robotics. TechTrends, 66(5), 771–783. https://doi.org/10.1007/s11528-022-00738-5.
Srisawasdi, N., Pondee, P., & Bunterm, T. (2018). Preparing pre-service teachers to integrate mobile technology into science laboratory learning: An evaluation of technology-integrated pedagogy module. International Journal of Mobile Learning and Organisation, 12(1), 1–17.
Stordeur, M. F., Nils, F., & Colognesi, S. (2022). No, an oral presentation is not just something you prepare at home! elementary teachers’ practices supporting preparation of oral presentations. L1-Educational Studies in Language and Literature. https://doi.org/10.21248/l1esll.2022.22.1.417
Sweller, J. (2010). Cognitive load theory: Recent theoretical advances. In J. L. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive load theory (pp. 29–47). Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.004
Tanaka, F., & Matsuzoe, S. (2012). Children teach a care-receiving robot to promote their learning: Field experiments in a classroom for vocabulary learning. Journal of Human-Robot Interaction, 1(1), 78–95. https://doi.org/10.5898/JHRI.1.1.Tanaka.
Timpe-Laughlin, V., Sydorenko, T., & Dombi, J. (2022). Human versus machine: Investigating L2 learner output in face-to-face versus fully automated role-plays. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2022.2032184
Tung, F. W. (2016). Child perception of humanoid robot appearance and behavior. International Journal of Human-Computer Interaction, 32(6), 493–502. https://doi.org/10.1080/10447318.2016.1172808.
van den Berghe, R., Verhagen, J., Oudgenoeg-Paz, O., Van der Ven, S., & Leseman, P. (2019). Social robots for language learning: A review. Review of Educational Research, 89(2), 259–295.
van der Lans, R. M., & Maulana, R. (2018). The use of secondary school student ratings of their teacher’s skillfulness for low-stake assessment and high-stake evaluation. Studies in Educational Evaluation, 58, 112–121. https://doi.org/10.3102/0034654318821286.
van Doremalen, J., Boves, L., Colpaert, J., Cucchiarini, C., & Strik, H. (2016). Evaluating automatic speech recognition-based language learning systems: A case study. Computer Assisted Language Learning, 29(4), 833–851. https://doi.org/10.1080/09588221.2016.1167090.
Wang, S. L., & Lin, S. S. (2007). The effects of group composition of self-efficacy and collective efficacy on computer-supported collaborative learning. Computers in Human Behavior, 23(5), 2256–2268. https://doi.org/10.1016/j.chb.2006.03.005.
Wedenborn, A., Wik, P., Engwall, O., & Beskow, J. (2019). The effect of a physical robot on vocabulary learning. arXiv preprint arXiv:1901.10461.
Wellsby, M., & Pexman, P. M. (2014). Developing embodied cognition: Insights from children’s concepts and language processing. Frontiers in Psychology, 5, 506. https://doi.org/10.3389/fpsyg.2014.00506.
Wiklund-Hörnqvist, C., Jonsson, B., & Nyberg, L. (2014). Strengthening concept learning by repeated testing. Scandinavian Journal of Psychology, 55(1), 10–16. https://doi.org/10.1111/sjop.12093.
Wise, S. L., & DeMars, C. E. (2003). Examinee motivation in low-stakes assessment: problems and potential solutions: Paper Presented At the Annual Meeting of the American Association of Higher Education Assessment Conference. Seattle, Canada.
Wu, W. C. V., Wang, R. J., & Chen, N. S. (2015). Instructional design using an in-house built teaching assistant robot to enhance elementary school English-as-a-foreign-language learning. Interactive Learning Environments, 23(6), 696–714. https://doi.org/10.1080/10494820.2013.792844.
Yang, F. C. O., Lai, H. M., & Wang, Y. W. (2023). Effect of augmented reality-based virtual educational robotics on programming students’ enjoyment of learning, computational thinking skills, and academic achievement. Computers & Education, 195, 104721.
Yang, Y. T. C., Chen, Y. C., & Hung, H. T. (2022). Digital storytelling as an interdisciplinary project to improve students’ English speaking and creative thinking. Computer Assisted Language Learning, 35(4), 840–862. https://doi.org/10.1080/09588221.2020.1750431.
Acknowledgements
This study is supported in part by the National Science and Technology Council of Taiwan under contract numbers NSTC 112-2410-H-011-012-MY3 and MOST 111-2410-H-011 -007 -MY3. The study is also supported by the “Empower Vocational Education Research Center” of National Taiwan University of Science and Technology (NTUST) from the Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan. The authors would also like to thank Ms. Yu Chun Lin for her technical support during the learning treatment of this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors would like to declare that there is no conflict of interest in this study.
Consent to participate
The participants were protected by hiding their personal information during the research process. They knew that the participation was voluntary and they could withdraw from the study at any time.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Darmawansah, D., Hwang, GJ. Effects of robot-based multiple low-stakes assessments on students’ oral presentation performance, collective efficacy, and learning attitude. Education Tech Research Dev (2024). https://doi.org/10.1007/s11423-024-10360-2
Accepted:
Published:
DOI: https://doi.org/10.1007/s11423-024-10360-2