Abstract
The integration of ChatGPT into educational systems underscores the critical importance of reevaluating the relevance of digital literacy skills. This study empirically tested, for the first time, the theoretical premise that the use of ChatGPT could render certain digital skills obsolete by assuming competencies that students were previously required to master. The study aimed to propose an instructional approach that combines a problem-based learning with ChatGPT activity (AIPBL). Subsequently, it sought to quantitatively measure the impact of AIPBL on students’ digital literacy competencies, comparing it to a similar PBL assignment devoid of AI technology (PBL) and a control group engaged in lecture-based learning (LBL). In addition, students’ experiences with ChatGPT were qualitatively evaluated. The study enrolled 223 healthcare management students, collecting 441 observations through pre-post-tests. Analysis of covariance and thematic analysis of qualitative data were employed. The results revealed that AIPBL led students to utilize fewer digital literacy skills compared to the PBL group, which displayed improvement across all three digital literacy components. Specifically, AIPBL did not significantly enhance participants’ perceptions of access and analysis skills, but it did bolster their evaluation skills between pre- and post-tests. Conversely, the LBL group showed no significant differences between their pre- and post-test results. In conjunction with qualitative findings, this study contributes to existing research by affirming the necessity for updated AI literacy skills in education and highlighting the evolving role of literacy skills in the context of AI integration in higher education.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Data are available upon request.
References
Abdulai, A. F., Tiffere, A. H., Adam, F., & Kabanunye, M. M. (2021). COVID-19 information-related digital literacy among online health consumers in a low-income country. International Journal of Medical Informatics, 145, 104322. https://doi.org/10.1016/j.ijmedinf.2020.104322.
Ahmed, S. T., & Roche, T. (2021). Making the connection: Examining the relationship between undergraduate students’ digital literacy and academic success in an English medium instruction (EMI) university. Education and Information Technologies, 26(4), 4601–4620. https://doi.org/10.1007/s10639-021-10443-0.
Ali, O., Abdelbaki, W., Shrestha, A., Elbasi, E., Alryalat, M. A. A., & Dwivedi, Y. K. (2023). A systematic literature review of artificial intelligence in the healthcare sector: Benefits, challenges, methodologies, and functionalities. Journal of Innovation & Knowledge, 8(1), 100333. https://doi.org/10.1016/j.jik.2023.100333.
Alt, D., & Raichel, N. (2020a). Enhancing perceived digital literacy skills and creative self-concept through gamified learning environments: Insights from a longitudinal study. International Journal of Educational Research, 101. https://doi.org/10.1016/j.ijer.2020.101561.
Alt, D., & Raichel, N. (2020b). Reflective journaling and metacognitive awareness: Insights from a longitudinal study in higher education. Reflective Practice, 21, 145–158.https://doi.org/10.1080/14623943.2020.1716708.
Arias López, M. D. P., Ong, B. A., Frigola, B., Fernández, X., Hicklent, A. L., Obeles, R. S., & Celi, A. J., L. A (2023). Digital literacy as a new determinant of health: A scoping review. PLOS Digital Health, 2(10), e0000279. https://doi.org/10.1371/journal.pdig.0000279.
Aronson, J. K. (2022). Artificial intelligence in pharmacovigilance: An introduction to terms, concepts, applications, and limitations. Drug Safety, 45(5), 407–418. https://doi.org/10.1007/s40264-022-01156-5.
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023). Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability, 15(17), 12983. https://doi.org/10.3390/su151712983.
Baro, E. E., Obaro, O. G., & Aduba, E. D. (2019). An assessment of digital literacy skills and knowledge-based competencies among librarians working in university libraries in Africa. Digital Library Perspectives, 35(3/4), 172–192. https://doi.org/10.1108/DLP-04-2019-0013.
Bentler, P. M. (2006). Equation 6 structural equations program manual. Multivariate Software, Inc. http://www.econ.upf.edu/~satorra/CourseSEMVienna2010/EQSManual.pdf.
Brørs, G., Larsen, M. H., Hølvold, L. B., & Wahl, A. K. (2023). eHealth literacy among hospital health care providers: A systematic review. BMC Health Services Research, 23(1), 1144. https://doi.org/10.1186/s12913-023-10103-8.
Buchholz, B. A., DeHart, J., & Moorman, G. (2020). Digital citizenship during a global pandemic: Moving beyond digital literacy. Journal of Adolescent & Adult Literacy, 64(1), 11–17. https://doi.org/10.1002/jaal.1076.
Carmi, E., Yates, S. J., Lockley, E., & Pawluczuk, A. (2020). Data citizenship: Rethinking data literacy in the age of disinformation, misinformation, and malinformation. Internet Policy Review, 9(2), 1–22. https://doi.org/10.14763/2020.2.1481.
Cattaneo, A. A., Antonietti, C., & Rauseo, M. (2022). How digitalised are vocational teachers? Assessing digital competence in vocational education and looking at its underlying factors. Computers & Education, 176. https://doi.org/10.1016/j.compedu.2021.104358.
Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20(1), 1–25. https://doi.org/10.1186/s41239-023-00408-3.
Cohen, J. (1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213. https://doi.org/10.1037/h0026256.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Merrill.
Das, D., Kumar, N., Longjam, L. A., Sinha, R., Roy, A. D., Mondal, H., & Gupta, P. (2023). Assessing the capability of ChatGPT in Answering First-and second-order knowledge questions on Microbiology as per competency-based Medical Education Curriculum. Cureus, 15(3). https://doi.org/10.7759/cureus.36034.
Deshpande, N., Arora, V. M., Vollbrecht, H., Meltzer, D. O., & Press, V. (2023). eHealth literacy and patient Portal Use and Attitudes: Cross-sectional observational study. JMIR Human Factors, 10, e40105. https://doi.org/10.2196/40105.
Eager, B., & Brunton, R. (2023). Prompting higher education towards AI-augmented teaching and learning practice. Journal of University Teaching & Learning Practice, 20(5). https://doi.org/10.53761/1.20.5.02.
EU (2023). Shaping Europe’s digital future. Digital skills and jobs. https://digital-strategy.ec.europa.eu/.
Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education, 9, e46885. https://doi.org/10.2196/46885.
Falloon, G. (2020). From digital literacy to digital competence: The teacher digital competency (TDC) framework. Educational Technology Research and Development, 68, 2449–2472. https://doi.org/10.1007/s11423-020-09767-4.
Fuchs, K. (2023). Exploring the opportunities and challenges of NLP models in higher education: Is Chat GPT a blessing or a curse? Frontiers in Education. https://doi.org/10.3389/feduc.2023.1166682. 8.
Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks Standards and Evaluations, 2(4), 100089. https://doi.org/10.1016/j.tbench.2023.100089.
He, N., Yan, Y., Wu, Z., Cheng, Y., Liu, F., Li, X., & Zhai, S. (2023). Chat GPT-4 significantly surpasses GPT-3.5 in drug information queries. Journal of Telemedicine and Telecare, 1–3. https://doi.org/10.1177/1357633X231181922.
Hemmo-Lotem, M., & Shani, M. (2018). Changing the paradigm in medicine in the wake of the Fourth Industrial Revolution. Hrefuaa (the Medicine), 157, 797–801. (Hebrew).
Hemmo-Lotem, M., Tzezana, R., & Levtzion-Korach, O. (2021). The information revolution in medicine. Hrefuaa (the Medicine), 160, 24–29. (Hebrew).
Heston, T. F., & Khun, C. (2023). Prompt engineering in medical education. International Medical Education, 2(3), 198–205. https://doi.org/10.3390/ime2030019.
Hsu, M. H., Chan, T. M., & Yu, C. S. (2023). Termbot: A chatbot-based crossword game for gamified medical terminology learning. International Journal of Environmental Research and Public Health, 20(5), 4185. https://doi.org/10.3390/ijerph20054185.
Javaid, M., Haleem, A., & Singh, R. P. (2023). ChatGPT for healthcare services: An emerging stage for an innovative perspective. BenchCouncil Transactions on Benchmarks Standards and Evaluations, 3(1), 100105. https://doi.org/10.1016/j.tbench.2023.100105.
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274.
Kohnen, A. M., Mertens, G. E., & Boehm, S. M. (2020). Can middle schoolers learn to read the web like experts? Possibilities and limits of a strategy-based intervention. Journal of Media Literacy Education, 12(2), 64–79. https://doi.org/10.23860/JMLE-2020-12-2-6.
Korkmaz Aslan, G., Kartal, A., Turan, T., Taşdemir Yiğitoğlu, G., & Kocakabak, C. (2021). Association of electronic health literacy with health-promoting behaviours in adolescents. International Journal of Nursing Practice, 27(2), e12921. https://doi.org/10.1111/ijn.12921.
Kreinsen, M., & Schulz, S. (2023). Towards the triad of digital literacy, data literacy, and AI literacy in teacher education: A discussion in light of the accessibility of novel generative AI. A position paper. https://doi.org/10.1177/0163443710393382.
Kuatbekov, A., Vershitskaya, E., Kosareva, I., & Ananishnev, V. (2023). E-Learning as a basis for the development of media competences in students. Journal of Information Science, 49(4), 1111–1125. https://doi.org/10.1177/01655515211040656.
Kuek, A., & Hakkennes, S. (2020). Healthcare staff digital literacy levels and their attitudes towards information systems. Health Informatics Journal, 26(1), 592–612. https://doi.org/10.1177/1460458219839613.
Kwiatkowska, W., & Wiśniewska-Nogaj, L. (2022). Digital skills and online collaborative learning: The study report. Electronic Journal of e-Learning, 20(5), 510–522. https://doi.org/10.34190/ejel.20.5.2412.
Manne, R., & Kantheti, S. C. (2021). Application of artificial intelligence in healthcare: Chances and challenges. Current Journal of Applied Science and Technology, 40(6), 78–89. https://doi.org/10.9734/cjast/2021/v40i631320.
Mikalef, P., Lemmer, K., Schaefer, C., Ylinen, M., Fjørtoft, S. O., Torvatn, H. Y., & Niehaves, B. (2022). Enabling AI capabilities in government agencies: A study of determinants for European municipalities. Government Information Quarterly, 39(4), 101596. https://doi.org/10.1016/j.giq.2021.101596.
Mohamad-Hani, T., Aljamaan, F., Malki, K. H., Alhasan, K., Altamimi, I., Aljarbou, R., & Al-Eyadhy, A. (2023). ChatGPT and the future of digital health: A study on healthcare workers’ perceptions and expectations. Healthcare, 11(13), 1812. https://doi.org/10.3390/healthcare11131812.
Morgan, A., Sibson, R., & Jackson, D. (2022). Digital demand and digital deficit: Conceptualising digital literacy and gauging proficiency among higher education students. Journal of Higher Education Policy and Management, 44(3), 258–275. https://doi.org/10.1080/1360080X.2022.2030275.
Musi, E., Aloumpi, M., Carmi, E., Yates, S., & O’Halloran, K. (2022). Developing fake news immunity: Fallacies as misinformation triggers during the pandemic. Online Journal of Communication and Media Technologies, 12(3). https://doi.org/10.30935/ojcmt%2F12083.
Naamati-Schneider, L. (2023). The effect of digitalization on service orientation and service perception among Israeli healthcare professionals: A qualitative study. Digital Health, 9. https://doi.org/10.1177/20552076231191892.
Oh, S. S., Kim, K. A., Kim, M., Oh, J., Chu, S. H., & Choi, J. (2021). Measurement of digital literacy among older adults: Systematic review. Journal of Medical Internet Research, 23(2), e26145. https://doi.org/10.2196/26145.
Palumbo, R., Nicola, C., & Adinolfi, P. (2022). Addressing health literacy in the digital domain: Insights from a literature review. Kybernetes, 51(13), 82–97. https://www.emerald.com/insight/0368-492X.htm.
Pangrazio, L., & Sefton-Green, J. (2020). The social utility of ‘data literacy’. Learning Media and Technology, 45(2), 208–220. https://doi.org/10.1080/17439884.2020.1707223.
Papp-Zipernovszky, O., Horváth, M. D., Schulz, P. J., & Csabai, M. (2021). Generation gaps in digital health literacy and their impact on health information seeking behavior and health empowerment in Hungary. Frontiers in Public Health, 9, 635943. https://doi.org/10.3389/fpubh.2021.635943.
Polizzi, G. (2020). Digital literacy and the national curriculum for England: Learning from how the experts engage with and evaluate online content. Computers & Education, 152, 103859. https://doi.org/10.1016/j.compedu.2020.103859.
Ren, W., Zhu, X., & Yang, J. (2022). The SES-based difference of adolescents’ digital skills and usages: An explanation from family cultural capital. Computers & Education, 177. https://doi.org/10.1016/j.compedu.2021.104382.
Saravi, B., Hassel, F., Ülkümen, S., Zink, A., Shavlokhova, V., Couillard-Despres, S., & Lang, G. M. (2022). Artificial intelligence-driven prediction modeling and decision making in spine surgery using hybrid machine learning models. Journal of Personalized Medicine, 12(4), 509. https://doi.org/10.3390/jpm12040509.
Sharma, S., Oli, N., & Thapa, B. (2019). Electronic health-literacy skills among nursing students. Advances in Medical Education and Practice, 10, 527–532.
Tekkeşin, A. İ. (2019). Artificial intelligence in healthcare. Anatolian Journal of Cardiology, 22(1), 8–9. https://doi.org/10.14744/AnatolJCardiol.2019.28.
TerKonda, S. P., & Fish, E. M. (2023). Artificial intelligence viewed through the lens of state regulation. Intelligence-Based Medicine, 100088. https://doi.org/10.1016/j.ibmed.2023.100088.
Tinmaz, H., Fanea-Ivanovici, M., & Baber, H. (2023). A snapshot of digital literacy. Library Hi Tech News, 40(1), 20–23. https://doi.org/10.1108/LHTN-12-2021-0095.
Tripto, J., Ben-Zvi Assaraf, O., Snapir, Z., & Amit, M. (2016). The ‘What is a system’ reflection interview as a knowledge integration activity for high school students’ understanding of complex systems in human biology. International Journal of Science Education, 38(4), 564–595. https://doi.org/10.1080/09500693.2016.1150620.
Tynes, B. M., Stewart, A., Hamilton, M., & Willis, H. A. (2021). From Google Searches to Russian Disinformation: Adolescent critical race digital literacy needs and skills. International Journal of Multicultural Education, 23(1), 110–130. https://doi.org/10.18251/ijme.v23i1.2463.
van Laar, E., van Deursen, A. J., & van Dijk, J. A. (2022). Developing policy aimed at 21st-century digital skills for the creative industries: An interview study with founders and managing directors. Journal of Education and Work, 35(2), 195–209. https://doi.org/10.1080/13639080.2022.2036710.
Vu, D. N., Phan, D. T., Nguyen, H. C., Le, L. T., Nguyen, H. C., Ha, T. H., & Van Duong, T. (2021). Impacts of digital healthy diet literacy and healthy eating behavior on fear of COVID-19, changes in mental health, and health-related quality of life among front-line health care workers. Nutrients, 13(8), 2656. https://doi.org/10.3390/nu13082656.
Wang, N., Wang, L., Ma, Z., & Wang, S. (2022). From knowledge seeking to knowledge contribution: A social capital perspective on knowledge sharing behaviors in online Q&A communities. Technological Forecasting and Social Change, 182. https://doi.org/10.1016/j.techfore.2022.121864.
York, T. J., Raj, S., Ashdown, T., & Jones, G. (2023). Clinician and computer: A study on doctors’ perceptions of artificial intelligence in skeletal radiography. BMC Medical Education, 23(1), 1–10. https://doi.org/10.1186/s12909-022-03976-6.
Zhao, Y., Llorente, A. M. P., & Gómez, M. C. S. (2021). Digital competence in higher education research: A systematic literature review. Computers & Education, 168. https://doi.org/10.1016/j.compedu.2021.104212.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
LNS: Conceptualization, data curation, methodology, writing- original draft preparation writing- reviewing and editing.
DA: Conceptualization, data curation, methodology, writing- original draft preparation writing- reviewing and editing.
The authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
This study was approved by the Ethics Committee of the first author’s College (approval no.396) as meeting the requirements of ethical research. All participants signed a written informed consent regarding participation in the study.
Competing interests
The authors declare that they have no competing interests.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Naamati-Schneider, L., Alt, D. Beyond digital literacy: The era of AI-powered assistants and evolving user skills. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12694-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10639-024-12694-z