Abstract
This article investigates an artificial intelligence language model, ChatGPT, and its ability to complete graduate-level instructional design assignments. The approach subjected ChatGPT to a needs, task, and learner analysis for a 12th-grade media literacy module and benchmarked its performance by expert evaluation and measurements via grading rubrics. Initial results show ChatGPT’s ability to write graduate-level instructional design assignments but with some challenges and frustration in the bot's ability to customize for location and school-specifics. Key benefits of the findings include new ways to challenge instructional design students, faculty customization of future assignments, and the potential of AI as a collaboration tool in the instructional design and educational process.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albirini, A. (2007). The crisis of educational technology, and the prospect of reinventing education. Educational Technology & Society, 10(1), 227–236.
Banathy, B. H. (1992). A systems view of education: Concepts and principles for effective practice. Educational Technology.
Bienkowski, M., Feng, M., & Means, B. (2014). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief (pp. 1–60).
Ceres, P. (n.d.). ChatGPT Is Coming for Classrooms. Don’t Panic. Wired. Retrieved October 2, 2023, from https://www.wired.com/story/chatgpt-is-coming-for-classrooms-dont-panic/
ChatGPT Passed an MBA Exam. What’s Next? (n.d.). Knowledge at Wharton. Retrieved February 23, 2023, from https://knowledge.wharton.upenn.edu/podcast/wharton-business-daily-podcast/chatgpt-passed-an-mba-exam-whats-next/. Accessed 10/2/23
Chiu, T. K. F. (2021). A holistic approach to the design of artificial intelligence (AI) Education for K-12 Schools. TechTrends, 65(5), 796–807. https://doi.org/10.1007/s11528-021-00637-1
Chocarro, R., Cortinas, Monica, & Marcos-Matas, G. (2021). Teachers’ attitudes towards chatbots in education: A technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educational Studies, 1–19. https://doi.org/10.1080/03055698.2020.1850426
Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of denerative artificial intelligence. Journal of Science Education and Technology, 32(3), 444–452.
Design Thinking. (n.d.). Retrieved October 2, 2023, from https://hbr.org/2008/06/design-thinking
Dimitriadis, G. (2020). Evolution in education: Chatbots. Homo Virtualis, 3(1), 47. https://doi.org/10.12681/homvir.23456
Eisenstein, E. L. (1980). The printing press as an agent of change. Cambridge University Press.
Elias, T. (2011). Learning analytics: Definitions, processes and potential.
Faye, C. J. F., Cathy. (2018). A history of panic over entertainment technology. Behavioral Scientist. https://behavioralscientist.org/history-panic-entertainment-technology/. Accessed 10/2/23
Grafton, A. (1999). The footnote: A curious history. Harvard University Press.
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S., Santos, O., Rodrigo, M., Cukurova, M., Bittencourt, I., & Koedinger, K. (2021). Ethics of AI in education: Towards a community-wide framework. International Journal of Artificial Intelligence in Education, 32. https://doi.org/10.1007/s40593-021-00239-1
Huang, W. H.-Y., & Soman, D. (2013). A practitioner’s guide to gamification of education. https://inside.rotman.utoronto.ca/behaviouraleconomicsinaction/files/2013/09/GuideGamificationEducationDec2013.pdf
Huang, K. (2023). Alarmed by A.I. chatbots, universities start revamping how they teach. The New York Times. https://www.nytimes.com/2023/01/16/technology/chatgpt-artificial-intelligence-universities.html. Accessed 10/2/23
Kerly, A., Hall, P., & Bull, S. (2007). Bringing Chatbots into education: Towards natural language negotiation of open learner models. Knowledge-Based Systems, 20, 177–185. https://doi.org/10.1016/j.knosys.2006.11.014
Luckin, R. (2017). Towards artificial intelligence-based assessment systems. Nature Human Behaviour, 1(3), 0028.
Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Buckingham Shum, S., Gašević, D., & Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education: Artificial Intelligence, 3, 100056. https://doi.org/10.1016/j.caeai.2022.100056
Marsicano, C. [@ChrisMarsicano]. (2023, January 16). I expect I’m going to get blasted for this by some, but here is my new ChatGPT policy for my [Tweet]. Twitter. https://twitter.com/ChrisMarsicano/status/1615015435069448192. Accessed 10/2/23
Mckie, I. A. S., & Narayan, B. (2019). Enhancing the academic library experience with chatbots: An exploration of research and implications for practice. Journal of the Australian Library and Information Association, 68(3), 268–277. https://doi.org/10.1080/24750158.2019.1611694
Meadows, D. H. (2008). Thinking in systems: A primer. Chelsea Green Publishing.
Moraes Neto, A. J., & Fernandes, M. A. (2019). Chatbot and conversational analysis to promote collaborative learning in distance education. 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), 2161–377X, 324–326. https://doi.org/10.1109/ICALT.2019.00102
Norman, D. A., & Stappers, P. J. (2016). DesignX: Complex sociotechnical systems. She Ji: The Journal of Design, Economics, and Innovation, 1(2), 83–106.
Norris, P. (2001). Digital divide: Civic engagement, information poverty and the internet World-Wide. In Info-The journal of policy, regulation and strategy for telecommunications, 5(1), 77–78. https://doi.org/10.1108/146366903322008287
Reigeluth, C. M., & Carr-Chellman, A. A. (2009). Instructional-Design theories and models, Volume III: Building a common knowledge base. Routledge. https://www.routledge.com/Instructional-Design-Theories-and-Models-Volume-III-Building-a-Common/Reigeluth-Carr-Chellman/p/book/9780805864564. Accessed 10/2/23
Reiser, R. A. (2001). A history of instructional design and technology: Part II: A history of instructional design. Educational Technology Research and Development, 49(2), 57–6. https://doi.org/10.1007/BF02504928
Roose, K. (2023). Don’t Ban ChatGPT in Schools. Teach With It. The New York Times. https://www.nytimes.com/2023/01/12/technology/chatgpt-schools-teachers.html. Accessed 10/2/23
Russell, S., & Norvig, P. (2003). Artificial intelligence: A modern approach. In Artificial Intelligence—AI (2nd ed., Vol. 82), Prentice Hall.
Saettler, L. P. (2004). The evolution of American educational technology. Information Age Publishing.
Selwyn, N. (2019). Should robots replace teachers? (1st ed.). Polity Press. Retrieved from https://www.perlego.com/book/1536481/should-robots-replace-teachers-ai-and-the-future-of-education-pdf. Accessed 10/2/23
Senge, P. M. (1990). The fifth discipline: The art and practice of the learning organization (1st ed). Doubleday/Currency. http://catdir.loc.gov/catdir/enhancements/fy0703/90002991-s.html. Accessed 10/2/23
Shum, S. B., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-Centred learning analytics. Journal of Learning Analytics, 6(2), 2. https://doi.org/10.18608/jla.2019.62.1
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
Srinivasan, V. (2022). AI & learning: A preferred future | Elsevier Enhanced Reader. Computers and Education: Artificial Intelligence, 3. https://doi.org/10.1016/j.caeai.2022.100062
Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning and Teaching, 6(1), 1. https://doi.org/10.37074/jalt.2023.6.1.17
Tomlinson, C. A. (2001). How to Differentiate Instruction in Mixed-Ability Classrooms. Upper Saddle River, NJ: Pearson Education.
Turkle, S. (1995). Life on the screen: Identity in the age of the internet. Simon & Schuster.
Whitford, E. (2023). ChatGPT And AI Will Fuel New EdTech Boom. Forbes. Retrieved February 23, 2023, from https://www.forbes.com/sites/emmawhitford/2023/01/18/chatgpt-and-ai-will-fuel-new-edtech-boom/. Accessed 10/2/23
Winkler, R., & Soellner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis. Academy of Management Proceedings, 2018(1), 15903. https://doi.org/10.5465/ambpp.2018.15903abstract
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet? - A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4. https://doi.org/10.3389/frai.2021.654924
Yang, S., & Evans, C. (2019, November 5). Opportunities and challenges in using AI chatbots in higher education. Proceedings of the 2019 3rd International Conference on Education and E-Learning. https://doi.org/10.1145/3371647.3371659
Zahour, O. (2020). Towards a chatbot for educational and vocational guidance in Morocco: Chatbot E-Orientation. International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 2479–2487. https://doi.org/10.30534/ijatcse/2020/237922020
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Parsons, B., Curry, J.H. Can ChatGPT Pass Graduate-Level Instructional Design assignments? Potential Implications of Artificial Intelligence in Education and a Call to Action. TechTrends 68, 67–78 (2024). https://doi.org/10.1007/s11528-023-00912-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11528-023-00912-3