Abstract
Mobile-assisted Language Learning (MALL) has been widely adopted in teaching and learning, yet there has been scant research concerning speaking test adaptation. An emerging type of mobile application is designed to facilitate test takers’ performance in a high-stakes speaking test (e.g., the International English Language Testing System (IELTS)). Such an exam-oriented mobile application provides customized learning opportunity with automatic feedbacks through artificial intelligence (AI) technology for users to enhance their speaking skills. This study aims to explore the attitudes of test takers on using exam-oriented mobile application to adapt in testing environment as influenced by their perceptions through the theory of Technology Acceptance Model. 235 Chinese IELTS test takers with experience of using such applications were invited to fill out an online questionnaire. Collected data were analysed through statistical method, textual analysis, word cloud approach, and sentiment analysis. Results revealed that test takers’ perceived usefulness and perceived ease of use towards the exam-oriented mobile application explained their attitudes to use such an application. They also expressed the concern of personalized AI function to support speaking test adaptation. Implications for educators, test taker, and application developers are provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets used in the current study are available from the corresponding author on reasonable request.
References
Ahmad, M., Qasim, H. M., & Khushi, Q. (2019). Specifying criteria for the assessment of speaking skill: A library based review. International Journal of Educational Sciences, 24(1–3), 23–33. https://doi.org/10.31901/24566322.2019/24.1-3.1075
Ahmed, A. A. A., Hassan, I., Pallathadka, H., Keezhatta, M. S., Haryadi, R. N., Mashhadani, Z. I. A., Attwan, L. Y., & Rohi, A. (2022). MALL and EFL learners’ speaking: Impacts of Duolingo and WhatsApp applications on speaking accuracy and fluency. Education Research International, 1–10. https://doi.org/10.1155/2022/6716474
Ahn, T. Y., & Lee, S. M. (2016). User experience of a mobile speaking application with automatic speech recognition for EFL learning. British Journal of Educational Technology, 47(4), 778–786. https://doi.org/10.1111/bjet.12354
Alsatuey, M. C. (2011). Perceived benefits and drawbacks of synchronous voice-based computer-mediated communication in the foreign language classroom. Computer-Assisted Language Learning, 24(5), 419–432. https://doi.org/10.1080/09588221.2011.574639
Alzahrani, M. (2019). The reasons behind the poor performance of Saudi students in IELTS. I-manager’s Journal on English Language Teaching, 9(1), 38–49. https://doi.org/10.26634/jelt.9.1.15375
Andujar, A., & Cruz-Martínez, M. S. (2020). Cognitive test anxiety in high-stakes oral examinations: Face-to-face or computer-based? Language Learning in Higher Education, 10(2), 445–467. https://doi.org/10.1515/cercles-2020-2029
Azli, W. U. A. W., Shah, P. M., & Mohamad, M. (2018). Perception on the usage of mobile assisted language learning (MALL) in English as a second language (ESL) learning among vocational college students. Creative Education, 9(1), 84. https://doi.org/10.4236/ce.2018.91008
Baleghizadeh, S., & Oladrostam, E. (2010). The effect of mobile assisted language learning (MALL) on grammatical accuracy of EFL students. Mextesol Journal, 34(2), 77–86.
Burston, J. (2015). Twenty years of MALL project implementation: A meta-analysis of learning outcomes. ReCALL, 27(1), 4–20. https://doi.org/10.1017/S0958344014000159
Chen, J. M. (2017). Three levels of push-pull dynamics among Chinese international students’ decision to study abroad in the Canadian context. Journal of International Students, 7(1), 113–135. https://doi.org/10.32674/jis.v7i1.248
Chen, C. M., & Hsu, S. H. (2008). Personalized intelligent mobile learning system for supporting effective English learning. Educational Technology and Society, 11(3), 153–180.
Chen, M. Y., & Lin, Y. M. (2023). Mobile-Assisted Language Assessment for Adult EFL Learners: Recommendations from a Systematic Review. In S. W. Chong and H. Reinders (eds.), Innovation in Learning-oriented Language Assessment (pp. 237–256). Palgrave Macmillan. https://doi.org/10.1007/978-3-031-18950-0_14
Chung, H. H., Chen, S. C., & Kuo, M. H. (2015). A study of EFL college students’ acceptance of mobile learning. Procedia-Social and Behavioral Sciences, 176, 333–339. https://doi.org/10.1016/j.sbspro.2015.01.479
British Council. (2019). IELTS grows to 3.5 million a year. https://www.ielts.org/news/2019/ielts-grows-to-three-and-a-half-million-a-year
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Mis Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
Depaolo, C., & Wilkinson, K. (2014). Get your head into the clouds: Using word clouds for analyzing qualitative assessment data. TechTrends, 58(3), 38–44. https://doi.org/10.1007/s11528-014-0750-9
Dewitt, B., Fischhoff, B., Davis, A. L., Broomell, S. B., Roberts, M. S., & Hanmer, J. (2019). Exclusion criteria as measurements I: Identifying invalid responses. Medical Decision Making, 1–11. https://doi.org/10.1177/0272989X19856617
Ducate, L., & Lomicka, L. (2009). Podcasting: An effective tool for honing language students’ pronunciation? Language Learning and Technology, 13(3), 66–86.
Evans, S., & Green, C. (2007). Why EAP is necessary: A survey of Hong Kong tertiary students. Journal of English for Academic Purposes, 6, 3–17. https://doi.org/10.1016/j.jeap.2006.11.005
Gangaiamaran, R., & Pasupathi, M. (2017). Review on use of mobile apps for language learning. International Journal of Applied Engineering Research, 12(21), 11242–11251.
Ghani, M. T., Hamzah, M., Ramli, S., Daud, W. A. A. W., Romli, T. R. M., & Mokhtar, N. N. M. (2019). A questionnaire-based approach on technology acceptance model for mobile digital game-based learning. Journal of Global Business and Social Entrepreneurship, 5(14), 11–21.
Golonka, E. M., Bowles, A. R., Frank, V. M., Richardson, D. L., & Freynik, S. (2014). Technologies for foreign language learning: A review of technology types and their effectiveness. Computer Assisted Language Learning, 27(1), 70–105. https://doi.org/10.1080/09588221.2012.700315
Gromik, N. A. (2012). Cell phone video recording feature as a language learning tool: A case study. Computers and Education, 58(1), 223–230. https://doi.org/10.1016/j.compedu.2011.06.013
Hsieh, J. S., Huang, Y., & Wu, W. V. (2017). Technological acceptance of LINE in flipped EFL oral training. Computers in Human Behavior, 70, 178–190. https://doi.org/10.1016/j.chb.2016.12.066
Huang, M., & Li, Y. (2020). Research progress and prospect of artificial intelligence education in China: Statistical analysis based on CNKI Journal literature. Cross-Cultural Communication, 16(4), 44–53.
Hwang, W., Guo, B., Hoang, A., Chang, C., & Wu, N. (2022). Facilitating authentic contextual EFL speaking and conversation with smart mechanisms and investigating its influence on learning achievements. Computer Assisted Language Learning, 1–27. https://doi.org/10.1080/09588221.2022.2095406
Jung, H. K. (2011). The correction of learner’s English pronunciation errors through speech recognition reading program. Multimedia-Assisted Language Learning, 14(3), 291–314. https://doi.org/10.15702/mall.2011.14.3.291
Karakaya, K., & Bozkurt, A. (2022). Mobile-assisted language learning (MALL) research trends and patterns through bibliometric analysis: Empowering language learners through ubiquitous educational technologies. System, 110, 102925. https://doi.org/10.1016/j.system.2022.102925
Kholis, A. (2021). Elsa speak app: Automatic speech recognition (ASR) for supplementing English pronunciation skills. Pedagogy: Journal of English Language Teaching, 9(1), 1–14. https://doi.org/10.32332/joelt.v9i1.2723
Kim, H. S. (2013). Emerging mobile apps to improve English listening skills. Multimedia-Assisted Language Learning, 16(2), 11–30. https://doi.org/10.15702/mall.2013.16.2.11
Knoch, U., Huisman, A., Elder, C., Kong, X., & McKenna, A. (2020). Drawing on repeat test takers to study test preparation practices and their links to score gains. Language Testing, 37(4), 550–572. https://doi.org/10.1177/0265532220927407
Kukulska-Hulme, A. (2009). Will mobile learning change language learning? ReCALL, 21(2), 157–165. https://doi.org/10.1017/S0958344009000202
Lei, T. (2021). The influence of the IELTS speaking test preparation on second language socialization of post-secondary international students in Canada. [Masters thesis]. University of Calgary. http://hdl.handle.net/1880/113336
Leinonen, T., Keune, A., Veermans, M., & Toikkanen, T. (2016). Mobile apps for reflection in learning: A design research in k-12 education. British Journal of Educational Technology, 47(1), 184–202. https://doi.org/10.1111/bjet.12224
Lestary, S. (2020). Perceptions and experience of mobile-assisted language learning for IELTS preparation: A case study of Indonesian Learners. International Journal of Information and Education Technology, 10(1), 67–73. https://doi.org/10.18178/ijiet.2020.10.1.1341
Lewis, J. R. (2019). Comparison of four TAM item formats: Effect of response option labels and order. Journal of Usability Studies, 14(4), 224–236.
Li, M., & Zou, B. (2022). Using artificial intelligence technology as a tutor for EFL speaking practice. In J. Colpaert and G. Stockwell (eds.), Smart CALL: Personalization, Contextualization, Socialization (pp. 82–100). Castledown. https://doi.org/10.29140/9781914291012-5
Li, Z., & Hegelheimer, V. (2013). Mobile-assisted grammar exercises: Effects on self-editing in L2 writing. Language Learning and Technology, 17(3), 135–156.
Li, R. (2020). Using artificial intelligence in learning English as a foreign language: An examination of IELTS LIULISHUO. In V. Pixel (Ed.), The Proceedings of the 13th International Conference Innovation in Language Learning Virtual Edition (pp. 186–190). https://doi.org/10.32629/jher.v1i2.178
Loewen, S., Crowther, D., Isbell, D. R., Kim, K. M., Maloney, J., Miller, Z. F., & Rawal, H. (2019). Mobile-assisted language learning: A Duolingo case study. ReCALL, 31(3), 293–311. https://doi.org/10.1017/S0958344019000065
Luo, M., & Cheng, L. (2020, July). Exploration of interactive foreign language teaching mode based on artificial intelligence. In 2020 International Conference on Computer Vision, Image and Deep Learning (pp. 285–290). Chong-qing, China: IEEE. https://doi.org/10.1109/CVIDL51233.2020.00-84
Ma, Q., & Liu, L. (2004). The technology acceptance model: A meta-analysis of empirical findings. Journal of Organizational and End User Computing, 16(1), 59–72. https://doi.org/10.4018/joeuc.2004010104
Mazer, J. P., Murphy, R. E., & Simonds, C. J. (2007). I’ll see you on “Facebook”: The effects of computer-mediated teacher self-disclosure on student motivation, affective learning, and classroom climate. Communication Education, 56(1), 1–17. https://doi.org/10.1080/03634520601009710
McKee, A. (2014). The Media and Communications in Australia (4th Edition). Routledge
Mei, B., Qi, W., Huang, X., & Huang, S. (2022). Speeko: An artificial intelligence-assisted personal public speaking coach. Technology Review, 1–5. https://doi.org/10.1177/00336882221107955
Metwally, A. A., Hakami, S. A., & Alabdaly, A. H. (2022). Factors contributing to the gap between listening and speaking skills performance among university students post covid-19. Journal of Language Teaching and Research, 13(5), 925–935. https://doi.org/10.17507/jltr.1305.04
Mihaylova, M., Gorin, S., Reber, T. P., & Rothen, N. (2022). A Meta-Analysis on Mobile Assisted Language Learning Applications: Benefits and Risks. Psychologica Belgica, 62(1), 252–271. https://doi.org/10.5334/pb.1146
Nakatsuhara, F., Inoue, C., Berry, V., & Galaczi, E. (2016). Exploring performance across two delivery modes for the same L2 speaking test: Face-to-face and video-conferencing delivery. A preliminary comparison of test-taker and examiner behavior. IELTS Partnership Research Papers, 1, 1–67.
Nakatsuhara, F., Inoue, C., Berry, V., & Galaczi, E. (2021). Video-conferencing speaking tests: Do they measure the same construct as face-to-face tests? Assessment in Education: Principles, Policy and Practice, 28(4), 369–388.
Official IELTS Website (2022). Test takers performance 2022. https://www.ielts.org/for-researchers/test-statistics/test-taker-performance
Ounis, A. (2017). The assessment of speaking skills at the tertiary level. International Journal of English Linguist, 7(4), 95–112. https://doi.org/10.5539/ijel.v7n4p95
Rajendran, T., & Yunus, M. M. (2021). A systematic literature review on the use of mobile-assisted language learning (MALL) for enhancing speaking skills among ESL and EFL learners. International Journal of Academic Research in Progressive Education and Development, 10(1), 586–609. https://doi.org/10.6007/IJARPED/v10-i1/8939
Rao, P. S. (2019). The importance of speaking skills in English classrooms. Alford Council of International English and Literature Journal, 2(2), 6–18.
Saritepeci, M., Duran, A., & Ermis, U. F. (2019). A new trend in preparing for foreign language exam (YDS) in Turkey: Case of WhatsApp in mobile learning. Education and Information Technologies, 24, 2677–2699. https://doi.org/10.1007/s10639-019-09893-4
Sharadgah, T. A., & Sa’di, R. A. (2022). A systematic review of research on the use of artificial intelligence in English language teaching and learning (2015–2021): What are the current effects? Journal of Information Technology Education: Research, 21, 337–377. https://doi.org/10.28945/4999
Singh, C. (2012). Word cloud analysis of the BJGP. British Journal of General Practice, 148. https://doi.org/10.3399/bjgp12X630142
Taber, K. S. (2018). The use of Cronbach’s Alpha when developing and reporting research instruments in science education. Research in Science Education, 48, 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
Tarighat, S., & Khodabakhsh, S. (2016). Mobile-assisted language assessment: Assessing speaking. Computers in Human Behavior, 64, 409–413. https://doi.org/10.1016/j.chb.2016.07.014
Tonekaboni, A. M. (2019). Effects of mobile assisted language learning (MALL) on speaking proficiency (a case of learn English daily mobile app). 2nd International Conference on Advanced Research in Humanities and Art, (pp. 1–13). Venice, Italy.
Traxler, J., & Kukulska-Hulme, A. (2015). Conclusion: Contextual challenges for the next generation. In Mobile Learning (pp. 208–226). Routledge. https://doi.org/10.4324/9780203076095-13
Wagner, E. (2020). Duolingo English Test (Revised Version July 2019). Language Assessment Quarterly, 17(3), 300–315. https://doi.org/10.1080/15434303.2020.1771343
Zhang, R., & Zou, D. (2020). Types, purposes, and effectiveness of state-of-the-art technologies for second and foreign language learning. Computer Assisted Language Learning, 35(4), 696–742. https://doi.org/10.1080/09588221.2020.1744666
Zhou, Z. (2021). A systematic literature review on the use of mobile-assisted language learning (MALL) for enhancing speaking skills in Chinese EFL context. InternationalJournal of Frontiers in Sociology, 3(15), 12–24. https://doi.org/10.25236/IJFS.2021.031502
Zou, B., & Li, J. (2015). Exploring mobile apps for English language teaching and learning. Critical CALL –Proceedings of the 2015 EUROCALL Conference, Padova, Italy, 564–568. https://doi.org/10.14705/rpnet.2015.000394
Zou, B., Liviero, S., Hao, M., & Wei, C. (2020). Artificial intelligence technology for EAP speaking skills: Student perceptions of opportunities and challenges. In M. Freiermuth and N. Zarrinabadi (eds.), Technology and the Psychology of Second Language Learners and Users (pp. 433–463). Palgrave Macmillan. https://doi.org/10.1007/978-3-030-34212-8_17
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
None.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, Q., Chan, K.K. Test takers’ attitudes of using exam-oriented mobile application as a tool to adapt in a high-stakes speaking test. Educ Inf Technol 29, 219–237 (2024). https://doi.org/10.1007/s10639-023-12297-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-023-12297-0