Playing Story Creation Games with Large Language Models: Experiments with GPT-3.5

Abstract

We created a web application where human users can play a story creation game with OpenAI’s GPT-3.5, based on the Tell Tale card game. Tell Tale requires players to generate a brand new and coherent story based on a set of initial story elements, making the game a useful structure for exploring how well GPT-3.5 performs in generating coherent and engaging narratives. We show that GPT-3.5 performs remarkably well in generating such a narrative based on a random set of initial story elements, and that GPT-3.5 is even able to incorporate other literary elements such as suspense and flashbacks into its stories to enhance them and make them more engaging. By having human testers play Tell Tale with GPT-3.5 through our web application, we also demonstrate GPT-3.5’s strong potential to be used as an interactive storytelling system, one that can both write and evaluate different narratives. We evaluate this potential using both quantitative and qualitative data from the human testers. Results indicate that, while GPT-3.5’s narrative abilities are far from perfect, large language models have great potential in many different automated narrative situations.

The project or effort depicted was or is sponsored by the U.S. Army Research Laboratory (ARL) under contract number W911NF-14-D-0005, and that the content of the information does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.