Abstract
Technology-enhanced language learning has exerted positive effects on the performance and engagement of L2 learners. Since the advent of tools based on recent advancement in artificial intelligence (AI), educators have made major strides in applying state-of-the-art technologies to writing classrooms. In November 2022, an AI-powered chatbot named ChatGPT capable of automatic text generation was introduced to the public. The study tried to apply ChatGPT’s text generation feature in a one-week L2 writing practicum. The study adopted a qualitative approach to investigate students’ behaviors and reflections in their exposure to ChatGPT in writing classrooms. The developmental features in learning activities and reflective perceptions were triangulated for the piloting evaluation of the impact of ChatGPT on L2 writing learners. The findings revealed the affordance and potential applicability of the tool in L2 writing pedagogy. Additionally, the tool also showcased an automatic workflow that could maximize the efficiency in composing writing. However, participants generally expressed their concern with its threats to academic honesty and educational equity. The study impelled the reconceptualization of plagiarism in the new era, development of regulatory policies and pedagogical guidance to regulate proper utilization of the tool. Being a pioneering effort, the study accentuated future research directions for more insights into the application of ChatGPT in L2 learning, and the establishment of corresponding pedagogical adjustments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
All data generated or analysed during this study are included in this published article.
References
Adams, D., & Chuah, K.-M. (2022). Artificial Intelligence-Based Tools in Research Writing. In P. P. Churi, S. Joshi, M. Elhoseny, & A. Omrane (Eds.), Artificial Intelligence in Higher Education: A Practical Approach (pp. 169–184). CRC Press.
Alshater, M. (2022). Exploring the Role of Artificial Intelligence in Enhancing Academic Performance: A Case Study of ChatGPT. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4312358
Aydın, Ö., & Karaarslan, E. (2022). OpenAI ChatGPT Generated Literature Review: Digital Twin in Healthcare. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4308687
Benzon, W. L. (2023). Discursive Competence in ChatGPT, Part 1: Talking with Dragons Version 2. SSRN Scholarly Paper. https://ssrn.com/abstract=4318832
Bikowski, D., & Vithanage, R. (2016). Effects of web-based collaborative writing on individual L2 writing development. Language Learning & Technology, 20(1), 79–99.
Bommarito II, M., & Katz, D. M. (2022).GPT Takes the Bar Exam (arXiv:2212.14402). arXiv. https://doi.org/10.48550/arXiv.2212.14402
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40. https://doi.org/10.3316/QRJ0902027
Bowen, N. E. J. A., & Nanni, A. (2021). Piracy, playing the system, or poor policies? Perspectives on plagiarism in Thailand. Journal of English for Academic Purposes, 51, 100992. https://doi.org/10.1016/j.jeap.2021.100992
Braun, V., & Clarke, V. (2012). Thematic analysis. In APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004
Cancino, M., & Panes, J. (2021). The impact of Google Translate on L2 writing quality measures: Evidence from Chilean EFL high school learners. System, 98, 102464. https://doi.org/10.1016/j.system.2021.102464
Chien, S.-C. (2017). Taiwanese College Students’ Perceptions of Plagiarism: Cultural and Educational Considerations. Ethics & Behavior, 27(2), 118–139. https://doi.org/10.1080/10508422.2015.1136219
Crosthwaite, P., Storch, N., & Schweinberger, M. (2020). Less is more? The impact of written corrective feedback on corpus-assisted L2 error resolution.Journal of Second Language Writing, 49, 100729. https://doi.org/10.1016/j.jslw.2020.100729
Dalglish, S. L., Khalid, H., & McMahon, S. A. (2020). Document analysis in health policy research: The READ approach. Health Policy and Planning, 35(10), 1424–1431. https://doi.org/10.1093/heapol/czaa064
Davies, L. J. P., & Howard, R. M. (2016). Plagiarism and the Internet: Fears, Facts, and Pedagogies. In T. Bretag (Ed.), Handbook of Academic Integrity (pp. 591–606). Springer. https://doi.org/10.1007/978-981-287-098-8_16
Davis, D. F., Golicic, S. L., & Boerstler, C. N. (2011). Benefits and challenges of conducting multiple methods research in marketing. Journal of the Academy of Marketing Science, 39(3), 467–479. https://doi.org/10.1007/s11747-010-0204-7
Dizon, G., & Gayed, J. M. (2021). Examining the Impact of Grammarly on the Quality of Mobile L2 Writing. JALT CALL Journal, 17(2), 74–92. https://doi.org/10.29140/jaltcall.v17n2.336
Doyle, S. (2007). Member checking with older women: A framework for negotiating meaning. Health Care for Women International, 28(10), 888–908. https://doi.org/10.1080/07399330701615325
Ebadi, S., & Amini, A. (2022). Examining the roles of social presence and human-likeness on Iranian EFL learners’ motivation using artificial intelligence technology: A case of CSIEC chatbot.Interactive Learning Environments, 0(0), 1–19 https://doi.org/10.1080/10494820.2022.2096638
Fernández-Gómez, E., Martín-Salvador, A., Luque-Vara, T., Sánchez-Ojeda, M. A., Navarro-Prado, S., & Enrique-Mirón, C. (2020). Content Validation through Expert Judgement of an Instrument on the Nutritional Knowledge, Beliefs, and Habits of Pregnant Women. Nutrients, 12(4), 1136. https://doi.org/10.3390/nu12041136
Fitria, T. N. (2022). Avoiding Plagiarism of Students’ Scientific Writing by Using the QuillBot Paraphraser.Elsya : Journal of English Language Studies, 4(3), Article 3. https://doi.org/10.31849/elsya.v4i3.9917
Flowerdew, J., & Li, Y. (2007). Plagiarism and second language writing in an electronic age. Annual Review of Applied Linguistics, 27, 161–183. https://doi.org/10.1017/S0267190508070086
Frye, B. L. (2022). Should Using an AI Text Generator to Produce Academic Writing Be Plagiarism? SSRN Scholarly Paper. https://ssrn.com/abstract=4292283
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers (p. 2022.12.23.521610). bioRxiv. https://doi.org/10.1101/2022.12.23.521610
Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How Does ChatGPT Perform on the Medical Licensing Exams? The Implications of Large Language Models for Medical Education and Knowledge Assessment (p. 2022.12.23.22283901). medRxiv. https://doi.org/10.1101/2022.12.23.22283901
Goh, E. (2013). Plagiarism Behavior Among Undergraduate Students in Hospitality and Tourism Education. Journal of Teaching in Travel & Tourism, 13(4), 307–322. https://doi.org/10.1080/15313220.2013.839295
Guo, K., Wang, J., & Chu, S. K. W. (2022). Using chatbots to scaffold EFL students’ argumentative writing. Assessing Writing, 54, 100666. https://doi.org/10.1016/j.asw.2022.100666
Hagendorff, T., Fabi, S., & Kosinski, M. (2022). Machine intuition: Uncovering human-like intuitive decision-making in GPT-3.5 (arXiv:2212.05206). arXiv. https://doi.org/10.48550/arXiv.2212.05206
Haque, M. U., Dharmadasa, I., Sworna, Z. T., Rajapakse, R. N., & Ahmad, H. (2022).‘I think this is the most disruptive technology’: Exploring Sentiments of ChatGPT Early Adopters using Twitter Data (arXiv:2212.05856). arXiv. https://doi.org/10.48550/arXiv.2212.05856
Hayes, N., & Introna, L. (2005). Systems for the Production of Plagiarists? The Implications Arising from the Use of Plagiarism Detection Systems in UK Universities for Asian Learners. Journal of Academic Ethics, 3(1), 55–73. https://doi.org/10.1007/s10805-006-9006-4
Hsu, H.-C. (2019). Wiki-mediated collaboration and its association with L2 writing development: An exploratory study. Computer Assisted Language Learning, 32(8), 945–967. https://doi.org/10.1080/09588221.2018.1542407
Hu, G., & Lei, J. (2012). Investigating Chinese University Students’ Knowledge of and Attitudes Toward Plagiarism From an Integrated Perspective. Language Learning, 62(3), 813–850. https://doi.org/10.1111/j.1467-9922.2011.00650.x
Jabotinsky, H. Y., & Sarel, R. (2022). Co-authoring with an AI? Ethical Dilemmas and Artificial Intelligence. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4303959
Jamieson, S., & Howard, R. (2019). Rethinking the relationship between plagiarism and academic integrity. Revue Internationale Des Technologies En Pédagogie Universitaire / International Journal of Technologies in Higher Education, 16(2), 69–85. https://doi.org/10.18162/ritpu-2019-v16n2-07
Jamshed, S. (2014). Qualitative research method-interviewing and observation. Journal of Basic and Clinical Pharmacy, 5(4), 87–88. https://doi.org/10.4103/0976-0105.141942
Jeon, J. (2021). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 0(0), 1–27. https://doi.org/10.1080/09588221.2021.1987272
Jinrong, L., & Mimi, L. (2018). Turnitin and peer review in ESL academic writing classrooms.Language Learning & Technology, 22(1) 27–41 10125/44576
Keck, C. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2 writers. Journal of Second Language Writing, 15(4), 261–278. https://doi.org/10.1016/j.jslw.2006.09.006
Keck, C. (2010). How Do University Students Attempt to Avoid Plagiarism? A Grammatical Analysis of Undergraduate Paraphrasing Strategies. Writing and Pedagogy, 2(2), 193–222. https://doi.org/10.1558/wap.v2i2.193
Kılıçkaya, F. (2020). Using a Chatbot, Replika, to Practice Writing Through Conversations in L2 English: A Case Study. In M. Kruk & M. Peterson (Eds.), Advances in Educational Technologies and Instructional Design (pp. 221–238). IGI Global. https://doi.org/10.4018/978-1-7998-2591-3.ch011
Kohnke, L. (2022). A Pedagogical Chatbot: A Supplemental Language Learning Tool.RELC Journal, 00336882211067054. https://doi.org/10.1177/00336882211067054
Kolb, A., & Kolb, D. (2017). Experiential Learning Theory as a Guide for Experiential Educators in Higher Education. Experiential Learning & Teaching in Higher Education, 1(1), 7–44.
Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44, 100450. https://doi.org/10.1016/j.asw.2020.100450
Kurniati, E. Y., & Fithriani, R. (2022). Post-Graduate Students’ Perceptions of Quillbot Utilization in English Academic Writing Class. Journal of English Language Teaching and Linguistics, 7(3), 437–451. https://doi.org/10.21462/jeltl.v7i3.852
Lee, L. (2020). An Exploratory Study of Using Personal Blogs for L2 Writing in Fully Online Language Courses. In B. Zou & M. Thomas (Eds.), Recent Developments in Technology-Enhanced and Computer-Assisted Language Learning (pp. 145–163). Information Science Reference. https://doi.org/10.4018/978-1-7998-1282-1.ch007
Li, Y., & Casanave, C. P. (2012). Two first-year students’ strategies for writing from sources: Patchwriting or plagiarism? Journal of Second Language Writing, 21(2), 165–180. https://doi.org/10.1016/j.jslw.2012.03.002
Liu, G.-Z., Lu, H.-C., Lin, V., & Hsu, W.-C. (2018). Cultivating undergraduates’ plagiarism avoidance knowledge and skills with an online tutorial system. Journal of Computer Assisted Learning, 34(2), 150–161. https://doi.org/10.1111/jcal.12223
Mannion, P., Siegel, M., Li, Z., Pham, Q. N., & Alshaikhi, A. (2019). Technology-Enhanced L2 Writing: A Systematic Literature Review Analysis and Synthesis. Journal of Foreign Language Education and Technology, 4(1), 127–150.
Merkel, W. (2020). A case study of undergraduate L2 writers’ concerns with source-based writing and plagiarism. TESOL Journal, 11(3), e00503. https://doi.org/10.1002/tesj.503
Miller, K. S., Lindgren, E., & Sullivan, K. P. H. (2008). The Psycholinguistic Dimension in Second Language Writing: Opportunities for Research and Pedagogy Using Computer Keystroke Logging. TESOL Quarterly, 42(3), 433–454. https://doi.org/10.1002/j.1545-7249.2008.tb00140.x
Morris, T. H. (2020). Experiential Learning – a Systematic Review and Revision of Kolb’s Model. Interactive Learning Environments, 28(8), 1064–1077. https://doi.org/10.1080/10494820.2019.1570279
Munoz-Luna, R. (2015). Main Ingredients for Success in L2 Academic Writing: Outlining. Drafting and Proofreading. PLOS ONE, 10(6), e0128309. https://doi.org/10.1371/journal.pone.0128309
Nagata, R., Hashiguchi, T., & Sadoun, D. (2020). Is the Simplest Chatbot Effective in English Writing Learning Assistance? In L.-M. Nguyen, X.-H. Phan, K. Hasida, & S. Tojo (Eds.), Computational Linguistics (245–256). Springer. https://doi.org/10.1007/978-981-15-6168-9_21
Nazari, N., Shabbir, M. S., & Setiawan, R. (2021). Application of Artificial Intelligence powered digital writing assistant in higher education: Randomized controlled trial. Heliyon, 7(5), e07014. https://doi.org/10.1016/j.heliyon.2021.e07014
Neumann, H., Leu, S., & McDonough, K. (2019). L2 writers’ use of outside sources and the related challenges. Journal of English for Academic Purposes, 38, 106–120. https://doi.org/10.1016/j.jeap.2019.02.002
Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033
OpenAI. (2022, November 30). ChatGPT: Optimizing Language Models for Dialogue. OpenAI. https://openai.com/blog/chatgpt/
Pecorari, D. (2001). Plagiarism and International Students: How the English-Speaking University Responds. In D. D. Belcher & A. R. Hirvela (Eds.), Linking Literacies: Perspectives on L2 Reading-Writing Connections (pp. 229–245). University of Michigan Press.
Pecorari, D. (2003). Good and original: Plagiarism and patchwriting in academic second-language writing. Journal of Second Language Writing, 12(4), 317–345. https://doi.org/10.1016/j.jslw.2003.08.004
Pecorari, D., & Petrić, B. (2014). Plagiarism in second-language writing. Language Teaching, 47(3), 269–302. https://doi.org/10.1017/S0261444814000056
Pecorari, D. (2022). Plagiarism and English for academic purposes: A research agenda.Language Teaching, 1–15. https://doi.org/10.1017/S0261444821000495
Qadir, J. (2022). Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education. TechRxiv https://doi.org/10.36227/techrxiv.21789434.v1
Roe, J., & Perkins, M. (2022). What are Automated Paraphrasing Tools and how do we address them? A review of a growing threat to academic integrity. International Journal for Educational Integrity, 18(1), 15. https://doi.org/10.1007/s40979-022-00109-w
Saeed, M. A., & Al Qunayeer, H. S. (2022). Exploring teacher interactive e-feedback on students’ writing through Google Docs: Factors promoting interactivity and potential for learning. The Language Learning Journal, 50(3), 360–377. https://doi.org/10.1080/09571736.2020.1786711
Sánchez-Guardiola Paredes, C., Aguaded Ramírez, E. M., & Rodríguez-Sabiote, C. (2021). Content Validation of a Semi-Structured Interview to Analyze the Management of Suffering. International Journal of Environmental Research and Public Health, 18(21), 11393. https://doi.org/10.3390/ijerph182111393
Şener, B., & Mede, E. (2022). Promoting learner autonomy and improving reflective thinking skills through reflective practice and collaborative learning.Innovation in Language Learning and Teaching, 1–16 https://doi.org/10.1080/17501229.2022.2047694
Seyyedrezaei, M. S., Amiryousefi, M., Gimeno-Sanz, A., & Tavakoli, M. (2022). A meta-analysis of the relative effectiveness of technology-enhanced language learning on ESL/EFL writing performance: Retrospect and prospect.Computer Assisted Language Learning, 1–34 https://doi.org/10.1080/09588221.2022.2118782
Stapleton, P. (2012). Gauging the effectiveness of anti-plagiarism software: An empirical study of second language graduate writers. Journal of English for Academic Purposes, 11(2), 125–133. https://doi.org/10.1016/j.jeap.2011.10.003
Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays—Should professors worry? Nature. https://doi.org/10.1038/d41586-022-04397-7
Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity? (arXiv:2212.09292). arXiv. https://doi.org/10.48550/arXiv.2212.09292
Taheri, P., & Nazmi, R. (2021). Improving EFL Learners’ Argumentative Writing Ability: Teacher vs. Peer Scaffolding. Teaching English Language, 15(2), 299–333. https://doi.org/10.22132/tel.2021.143348
Traniello, J. F. A., & Bakker, T. C. M. (2016). Intellectual theft: Pitfalls and consequences of plagiarism. Behavioral Ecology and Sociobiology, 70(11), 1789–1791. https://doi.org/10.1007/s00265-016-2207-y
Tsai, S.-C. (2019). Using google translate in EFL drafts: A preliminary investigation. Computer Assisted Language Learning, 32(5–6), 510–526. https://doi.org/10.1080/09588221.2018.1527361
Tuzi, F. (2004). The impact of e-feedback on the revisions of L2 writers in an academic writing course. Computers and Composition, 21(2), 217–235. https://doi.org/10.1016/j.compcom.2004.02.003
Wahle, J. P., Ruas, T., Foltýnek, T., Meuschke, N., & Gipp, B. (2022). Identifying Machine-Paraphrased Plagiarism. In M. Smits (Ed.), Information for a Better World: Shaping the Global Future (pp. 393–413). Springer International Publishing. https://doi.org/10.1007/978-3-030-96957-8_34
Wenzlaff, K., & Spaeth, S. (2022). Smarter than Humans? Validating how OpenAI’s ChatGPT Model Explains Crowdfunding, Alternative Finance and Community Finance. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4302443
Wu, L., Wu, Y., & Zhang, X. (2021). L2 Learner Cognitive Psychological Factors About Artificial Intelligence Writing Corrective Feedback. English Language Teaching, 14(10), Article 10. https://doi.org/10.5539/elt.v14n10p70
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. (2022).The Death of the Short-Form Physics Essay in the Coming AI Revolution (arXiv:2212.11661). arXiv. https://doi.org/10.48550/arXiv.2212.11661
Yeh, E. (2021). Intentional Plagiarism? Strategies for Teaching Language Learners Academic Integrity. Kappa Delta Pi Record, 57(3), 132–137. https://doi.org/10.1080/00228958.2021.1935506
Yoon, H., & Hirvela, A. (2004). ESL student attitudes toward corpus use in L2 writing. Journal of Second Language Writing, 13(4), 257–283. https://doi.org/10.1016/j.jslw.2004.06.002
Zhai, X. (2022). ChatGPT User Experience: Implications for Education. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4312418
Funding
Young Researcher Program of Xinyang Agriculture and Forestry University (QN2022049). The relevant fund was granted to Da Yan, the sole author of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
Learning Log for the ChatGPT & Writing Practicum
Appendix 2
Interview Protocol
-
1.
What are the effects of ChatGPT on your completion of writing tasks in the practicum?
-
2.
What are the advantages of ChatGPT over its precedents and alternatives?
-
3.
How do you evaluate the quality of textual output from ChatGPT?
-
4.
What is the most striking feature of ChatGPT in your experience?
-
5.
Does ChatGPT pose any threat to the learning and teaching of English writing?
-
6.
If everyone is equipped with the knowledge of ChatGPT in the near future, how do you think about it?
-
7.
How do you think about the rapid spread of ChatGPT in online communities?
-
8.
Do you support or oppose the usage of ChatGPT in writing classrooms?
-
9.
Do you think the official introduction of ChatGPT in language education will have a bright future?
-
10.
If we are to use ChatGPT regularly in classrooms, what do you think we need to adjust most urgently?
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yan, D. Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Educ Inf Technol 28, 13943–13967 (2023). https://doi.org/10.1007/s10639-023-11742-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-023-11742-4