Abstract
This paper shares results from a pedagogical experiment that assigns undergraduates to “cheat” on a final class essay by requiring their use of text-generating AI software. For this assignment, students harvested content from an installation of GPT-2, then wove that content into their final essay. At the end, students offered a “revealed” version of the essay as well as their own reflections on the experiment. In this assignment, students were specifically asked to confront the oncoming availability of AI as a writing tool. What are the ethics of using AI this way? What counts as plagiarism? What are the conditions, if any, we should place on AI assistance for student writing? And how might working with AI change the way we think about writing, authenticity, and creativity? While students (and sometimes GPT-2) offered thoughtful reflections on these initial questions, actually composing with GPT-2 opened their perspectives more broadly on the ethics and practice of writing with AI. In this paper, I share how students experienced those issues, connect their insights to broader conversations in the humanities about writing and communication, and explain their relevance for the ethical use and evaluation of language models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
The course syllabus and module assignments are available open access in the MLA CORE repository: https://doi.org/10.17613/0h18-5p41.
Code availability
Not applicable.
Notes
Plagiarism appears as its own section of “Academic Misconduct” (8.4) in NC State’s student conduct policy (“POL 11.35.01 – Code of Student Conduct” 2020).
As defined by Long and Magerko, AI literacy includes “a set of competencies that enables individuals to critically evaluate AI technologies; communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the workplace” (2020, 2). See also Ng et al. for a review of recent approaches to defining AI literacy (2021).
Long and Magerko recommend that “Researchers seeking to foster AI literacy may want to avoid misleading tactics like Turing deceptions and black-box algorithms” (2020, 8)—and I could not disagree more. Inviting students into this experience instead resembles how Meredith Broussard works with AI “to commit acts of investigative journalism” into its consequences (2018, 6).
Such cross-disciplinary approaches might especially be warranted given the landscape of ethics training in CS programs (Raji et al. 2021).
The course syllabus and module assignments are available open access in the MLA CORE repository: http://dx.doi.org/10.17613/0h18-5p41.
For an extensive introduction to this class of machine learning software as well as its emerging uses and risks, see Bommasani et al. (2021).
Scott has since moved to a position at the software company Sourcegraph.
A quick demo of GPT-2 can be done online with the site “Write With Transformer” created by the company Hugging Face: https://transformer.huggingface.co/doc/distil-gpt2 In a subsequent version of this assignment, students used the open-source language model GPT-J which also makes a web-based demo available in a browser: https://6b.eleuther.ai/.
We considered training the model on student’s own writing for better results, but realized that would require a prohibitive and potentially invasive amount of training data from students: 50 + MB their writing in plain text. We agreed a simpler approach would still achieve the assignment’s goals.
This may especially apply in context of ESL and language diversity, as students and/or instructors disqualify their expression against standard written English.
See also Reid on the “possibility space” of nonhuman rhetoric (2020).
Personal conversation with the author. Rieder’s own experiments in physical computing and new media composition are similarly interested in hybridity. He describes this as “everting” new media, or “infusing the real with aspects of the virtual.”
For an extensive introduction to assemblage in writing studies, as well as current examples in pedagogical practice, see Yancey and McElroy (2017).
As Lauren Goodlad complains, “while there is increasing talk of making AI ‘ethical,’ ‘democratic,’ and ‘human-centered,’ scholars in the humanities seldom shape these discussions” (Goodlad and Dimock 2021, 317).
References
Anson CM (2006) Can’t touch this: reflections on the servitude of computers as readers. In: Patricia FE, Richard H (eds) Machine scoring of student essays: truth and consequences. Utah State University Press, Logan, pp 38–56
Anson CM, Perelman L (2017) Machines can evaluate writing well. In: Ball CE, Loewe DM (eds) Bad ideas about writing. Digital Publishing Institute/West Virginia University Libraries, Morgantown, pp 278–286
Bailey J (2020) How teachers can prepare for AI-based writing. Turnitin. https://www.turnitin.com/blog/how-teachers-can-prepare-for-ai-based-writing. May 21, 2020.
Battista A, Katherine B, Marybeth M (2020) Data literacy in media studies: strategies for collaborative teaching of critical data analysis and visualization. J Interact Technol Pedag. https://jitp.commons.gc.cuny.edu/data-literacy-in-media-studies-strategies-for-collaborative-teaching-of-critical-data-analysis-and-visualization/.
Bender EM, Timnit G, Amy M-M, Schmargaret S (2021) On the dangers of stochastic parrots: can language models be too big?” In: FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 1–14. Virtual Event, Canada: ACM. https://doi.org/10.1145/3442188.3445922.
Bommasani R, Drew AH, Ehsan A, Russ A, Simran A, Sydney VA, Michael SB, et al (2021) On the opportunities and risks of foundation models. http://arxiv.org/abs/2108.07258.
Boyd D, Crawford K (2012) Critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon. Inform Commun Soc 15(5):662–679. https://doi.org/10.1080/1369118X.2012.678878
Boyle C (2016) Writing and rhetoric and/as posthuman practice. Coll Engl 78(6):532–554
Branwen G (2020) GPT-3 creative fiction. Gwern.Net. 2020. https://www.gwern.net/GPT-3.
Broussard M (2018) Artificial unintelligence: how computers misunderstand the world. MIT Press, Cambridge
Crawford K (2021) Atlas of AI: power, politics, and the planetary costs of artificial intelligence. Yale University Press, New Haven
D’Ignazio C, Klein L (2020) Data feminism. MIT Press, Cambridge
Dragga S, Gong G (1989) Editing: the design of rhetoric. Routledge, London
Elkins K, Jon C (2020) Can GPT-3 pass a Writer’s turing test? J Cult Anal. https://doi.org/10.22148/001c.17212
Flannery KT (1991) Composing and the question of agency. Coll Engl 53(6):701–713. https://doi.org/10.2307/377895
Goodlad LME, Dimock WC (2021) AI and the human. PMLA 136(2):317–319. https://doi.org/10.1632/S0030812921000079
Haraway DJ (2016) A cyborg manifesto: science, technology, and socialist-feminism in the late twentieth century. In: Manifestly Haraway. University of Minnesota Presshttps://doi.org/10.5749/minnesota/9780816650477.003.0001
Hayles NK (2012) How we think: digital media and contemporary technogenesis. The University of Chicago Press, Chicago London
Henrickson L (2019) Towards a new sociology of the text: the hermeneutics of algorithmic authorship. PhD Dissertation, Loughborough University.
Henrickson L (2021) Reading computer-generated texts. Camb Univ Press. https://doi.org/10.1017/9781108906463
Hern A (2019) New AI fake text generator may be too dangerous to release, say creators. The Guardian. https://www.theguardian.com/technology/2019/feb/14/elon-musk-backed-ai-writes-convincing-news-fiction. February 14, 2019
Howard RM (1992) A plagiarism pentimento. J Teach Writ 11(2):233–245
Howard RM (1995) Plagiarisms, authorships, and the academic death penalty. Coll Engl 57(7):788–806. https://doi.org/10.2307/378403
Hua M, Rita R (2020) Playing with unicorns: AI dungeon and citizen NLP. Digit Hum Quart 14(4). http://www.digitalhumanities.org/dhq/vol/14/4/000533/000533.html.
Johnson-Eilola J, Selber SA (2007) Plagiarism, originality, assemblage. Comput Compos 24(4):375–403. https://doi.org/10.1016/j.compcom.2007.08.003
Kreps S, Miles M (2020) Not your father’s bots. https://www.foreignaffairs.com/articles/2019-08-02/not-your-fathers-bots. April 16, 2020.
Lang A, Quinn D, Annie KL (2021) The ghost in Anouk’s Laptop. The Data-Sitters Club (blog). https://datasittersclub.github.io/site/dsc9. February 17, 2021
Laquintano T, Annette V (2017) How automated writing systems affect the circulation of political information online. Lit Compos Stud 5(2):43–62. https://doi.org/10.21623/1.5.2.4
Lea R (2020) If a novel was good, would you care if it was created by artificial intelligence?. The Guardian. http://www.theguardian.com/commentisfree/2020/jan/27/artificial-intelligence-computer-novels-fiction-write-books. January 27, 2020
Long D, Brian M (2020) What is AI literacy? Competencies and design considerations. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. CHI ’20. Association for Computing Machinery, New York https://doi.org/10.1145/3313831.3376727.
Manjavacas E, Folgert K, Ben B, Mike K (2017) Synthetic literature: writing science fiction in a co-creative process. In: Proceedings of the Workshop on Computational Creativity in natural Language Generation (CC-NLG 2017), 29–37. Association for Computational Linguistics, Santiago de Compostela. https://doi.org/10.18653/v1/W17-3904.
Marcus G, Ernest D (2020) GPT-3, bloviator: OpenAI’s language generator has no idea what it’s talking about—MIT technology review. In: MIT Technology Review (blog). https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/. August 22, 2020
Metz C, Scott B (2019) How AI could be weaponized to spread disinformation. The New York Times, June 7, 2019.
Miller S (2004) Rescuing the subject : a critical introduction to rhetoric and the writer. In: Paperback (ed). Southern Illinois University Press, Carbondale
Miller CR (2007) What can automation tell us about agency? Rhetor Soc Q 37(2):137–157. https://doi.org/10.1080/02773940601021197
Ng DT, Kit JK, Leung L, Chu KWS, Qiao MS (2021) AI literacy: definition, teaching, evaluation and ethical issues. Proc Assoc Inform Sci Technol 58(1):504–509. https://doi.org/10.1002/pra2.487
Otsuki GJ (2020) OK computer: to prevent students cheating with AI text-generators, we should bring them into the classroom. In: The Conversation (blog). https://theconversation.com/ok-computer-to-prevent-students-cheating-with-ai-text-generators-we-should-bring-them-into-the-classroom-129905. January 23, 2020.
POL 11.35.01—code of student conduct (2020) NC State University. 2020. https://policies.ncsu.edu/policy/pol-11-35-01/.
Price M (2002) Beyond ‘gotcha!’: situating plagiarism in policy and pedagogy. Coll Compos Commun 54(1):88–115. https://doi.org/10.2307/1512103
Radford A, Jeffrey W, Dario A, Daniela A, Jack C, Miles B, Ilya S (2019) Better language models and their implications. OpenAI. https://openai.com/blog/better-language-models/. February 14, 2019.
Raji ID, Morgan KS, Razvan A (2021) You can’t sit with us: exclusionary pedagogy in AI ethics education. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 515–25. FAccT ’21. Association for Computing Machinery, New York. https://doi.org/10.1145/3442188.3445914.
Reid A (2020) Synthetic speech and negotiation: AI’s nonhuman rhetoric. Enculturation. http://enculturation.net/synthetic_speech_and_negotiation.
Seabrook J (2019) Can a machine learn to write for the new yorker? https://www.newyorker.com/magazine/2019/10/14/can-a-machine-learn-to-write-for-the-new-yorker. October 7, 2019.
Sloan R (2016) Writing with the Machine. Robin Sloan (blog). 2016. https://www.robinsloan.com/notes/writing-with-the-machine/.
Standage T, Seth S (2018) The box that AI lives. In: The Secret History of the Future. http://www.slate.com/articles/podcasts/secret_history_of_the_future/2018/09/a_200_year_old_chess_playing_robot_explains_the_internet.html. Accessed September 28, 2018.
Underwood T (2021) Science fiction hasn’t prepared us to imagine machine learning. In: The Stone and the Shell (blog). https://tedunderwood.com/2021/02/02/why-sf-hasnt-prepared-us-to-imagine-machine-learning/. February 2, 2021
Yancey KB, Stephen M (eds) (2017) Assembling composition. In: CCCC studies in writing and rhetoric. In: Conference on College Composition and Communication of the National Council of Teachers of English, Urbana
Acknowledgements
My thanks to the students in the HON 202 seminar “Data and the Human” for their enthusiastic participation in this experiment. Many thanks also to NC State University colleagues including Scott Bailey, Zachary Beare, Chris Anson, Helen Burgess, and David Rieder for sharing their perspectives and recommendations. Thanks also to peer reviewers for constructive suggestions that improved this article.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
Not applicable.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Additional declarations for articles in life science journals that report the results of studies involving humans and/or animals
Not applicable.
Ethics approval
Federal regulations allow specific categories of human subjects research to be exempt from continuing IRB review [45 CFR 46.104(d)]. Exemption Category 1 applies to research conducted in established or commonly accepted educational settings involving normal educational practices, including research on classroom instructional strategies.
Consent to participate
All enrolled students completed the major assignments as course requirements.
Consent for publication
All students gave their emailed consent to have their essays included and quoted in my report. Quotes from student papers have been anonymized in this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fyfe, P. How to cheat on your final paper: Assigning AI for student writing. AI & Soc 38, 1395–1405 (2023). https://doi.org/10.1007/s00146-022-01397-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-022-01397-z