Skip to main content
SpringerLink
Log in

Prompt Engineering for Narrative Choice Generation

  • Conference paper
  • First Online:
Interactive Storytelling (ICIDS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14383))

Included in the following conference series:

Abstract

Large language models (LLMs) have recently revolutionized performance on a variety of natural language generation tasks, but have yet to be studied in terms of their potential for generating reasonable character choices as well as subsequent decisions and consequences given a narrative context. We use recent (not yet available for LLM training) film plot excerpts as an example initial narrative context and explore how different prompt formats might affect narrative choice generation by open-source LLMs. The results provide a first step toward understanding effective prompt engineering for future human-AI collaborative development of interactive narratives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Akoury, N., Wang, S., Whiting, J., Hood, S., Peng, N., Iyyer, M.: Storium: a dataset and evaluation platform for machine-in-the-loop story generation. arXiv preprint arXiv:2010.01717 (2020)

  2. Ammanabrolu, P., et al.: Story realization: expanding plot events into sentences. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 7375–7382 (2020)

    Google Scholar 

  3. Anand, Y., Nussbaum, Z., Duderstadt, B., Schmidt, B., Mulyar, A.: Gpt4all: training an assistant-style chatbot with large scale data distillation from GPT-3.5-turbo. GitHub (2023)

    Google Scholar 

  4. Asimov, I.: I, Robot, vol. 1. Spectra (2004)

    Google Scholar 

  5. Barber, H., Kudenko, D.: Generation of adaptive dilemma-based interactive narratives. IEEE Trans. Comput. Intell. AI Games 1(4), 309–326 (2009)

    Article  Google Scholar 

  6. Braff, Z.D.: A Good Person (2023)

    Google Scholar 

  7. Brants, T., Popat, A.C., Xu, P., Och, F.J., Dean, J.: Large language models in machine translation (2007)

    Google Scholar 

  8. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)

    Google Scholar 

  9. Calderwood, A., Wardrip-Fruin, N., Mateas, M.: Spinning coherent interactive fiction through foundation model prompts. In: ICCC (2022)

    Google Scholar 

  10. Elnahla, N.: Black mirror: Bandersnatch and how Netflix manipulates us, the new gods. Consumption Markets Cult. 23(5), 506–511 (2020)

    Article  Google Scholar 

  11. Fan, A., Lewis, M., Dauphin, Y.: Hierarchical neural story generation. arXiv preprint arXiv:1805.04833 (2018)

  12. Freiknecht, J., Effelsberg, W.: Procedural generation of interactive stories using language models. In: Proceedings of the 15th International Conference on the Foundations of Digital Games, pp. 1–8 (2020)

    Google Scholar 

  13. Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M.M., Dalsgaard, P.: Mapping the landscape of creativity support tools in HCI. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–18 (2019)

    Google Scholar 

  14. Garcia, L., Martens, C.: Carambola: enforcing relationships between values in value-sensitive agent design. In: Vosmeer, M., Holloway-Attaway, L. (eds.) Interactive Storytelling. ICIDS 2022. LNCS, vol. 13762, pp. 83–90. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-22298-6_5

  15. Harmon, S.: An expressive dilemma generation model for players and artificial agents. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 12, pp. 176–182 (2016)

    Google Scholar 

  16. Holl, E., Melzer, A.: Moral minds in gaming: a quantitative case study of moral decisions in detroit: become human. J. Media Psychol. Theor. Methods Appl. 34(5), 287–298 (2021)

    Article  Google Scholar 

  17. Keskar, N.S., McCann, B., Varshney, L.R., Xiong, C., Socher, R.: Ctrl: a conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 (2019)

  18. Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. Adv. Neural. Inf. Process. Syst. 35, 22199–22213 (2022)

    Google Scholar 

  19. Kolhoff, L., Nack, F.: How relevant is your choice? In: Cardona-Rivera, R.E., Sullivan, A., Young, R.M. (eds.) ICIDS 2019. LNCS, vol. 11869, pp. 73–85. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33894-7_9

    Chapter  Google Scholar 

  20. Kreminski, M., Mateas, M.: A coauthorship-centric history of interactive emergent narrative. In: Mitchell, A., Vosmeer, M. (eds.) ICIDS 2021. LNCS, vol. 13138, pp. 222–235. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92300-6_21

    Chapter  Google Scholar 

  21. Lanzi, P.L., Loiacono, D.: ChatGPT and other large language models as evolutionary engines for online interactive collaborative game design. arXiv preprint arXiv:2303.02155 (2023)

  22. Mateas, M., Mawhorter, P.A., Wardrip-Fruin, N.: Intentionally generating choices in interactive narratives. In: ICCC, pp. 292–299 (2015)

    Google Scholar 

  23. Nichols, E., Gao, L., Gomez, R.: Collaborative storytelling with large-scale neural language models. In: Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, pp. 1–10 (2020)

    Google Scholar 

  24. Reynolds, L., McDonell, K.: Prompt programming for large language models: beyond the few-shot paradigm. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–7 (2021)

    Google Scholar 

  25. Roemmele, M., Gordon, A.S.: Creative help: a story writing assistant. In: Schoenau-Fog, H., Bruni, L.E., Louchart, S., Baceviciute, S. (eds.) ICIDS 2015. LNCS, vol. 9445, pp. 81–92. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27036-4_8

    Chapter  Google Scholar 

  26. Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E., Singh, S.: AutoPrompt: eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 (2020)

  27. Swanson, R., Gordon, A.S.: Say anything: a massively collaborative open domain story writing companion. In: Spierling, U., Szilas, N. (eds.) ICIDS 2008. LNCS, vol. 5334, pp. 32–40. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89454-4_5

    Chapter  Google Scholar 

  28. Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

  29. Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. Adv. Neural. Inf. Process. Syst. 35, 24824–24837 (2022)

    Google Scholar 

  30. White, J., et al.: A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv preprint arXiv:2302.11382 (2023)

  31. Wu, T., Terry, M., Cai, C.J.: AI chains: Transparent and controllable human-AI interaction by chaining large language model prompts. In: Proceedings of the 2022 CHI conference on Human Factors in Computing Systems, pp. 1–22 (2022)

    Google Scholar 

  32. Xu, R., Zhu, C., Zeng, M.: Narrate dialogues for better summarization. In: Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 3565–3575 (2022)

    Google Scholar 

  33. Ye, J., et al.: A comprehensive capability analysis of GPT-3 and GPT-3.5 series models. arXiv preprint arXiv:2303.10420 (2023)

  34. Yuan, A., Coenen, A., Reif, E., Ippolito, D.: WordCraft: story writing with large language models. In: 27th International Conference on Intelligent User Interfaces, pp. 841–852 (2022)

    Google Scholar 

  35. Zamfirescu-Pereira, J., Wong, R.Y., Hartmann, B., Yang, Q.: Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–21 (2023)

    Google Scholar 

  36. Zhang, N., et al.: Differentiable prompt makes pre-trained language models better few-shot learners. arXiv preprint arXiv:2108.13161 (2021)

  37. Zhou, Y., et al.: Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910 (2022)

  38. Zhou, Y., Zhao, Y., Shumailov, I., Mullins, R., Gal, Y.: Revisiting automated prompting: are we actually doing better? arXiv preprint arXiv:2304.03609 (2023)

Download references

Author information

Authors and Affiliations

  1. Bowdoin College, Brunswick, ME, 04011, USA

    Sarah Harmon & Sophia Rutman

Authors
  1. Sarah Harmon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophia Rutman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarah Harmon .

Editor information

Editors and Affiliations

  1. University of Skövde, Skövde, Sweden

    Lissa Holloway-Attaway

  2. University of Central Florida, Orlando, FL, USA

    John T. Murray

6 Appendix

6 Appendix

This appendix includes descriptions and examples of each type of failure category observed in the choice/decision/consequence generation task. Figure 3 provides a sample prompt with an invented plot excerpt that serves as a running example as each failure type is considered in Tables 5, 6 and 7. An example of a successful response is provided in Fig. 4.

Table 5. Common catastrophic failure (does not answer the prompt) categories and corresponding example responses in response to Fig. 3’s example prompt.
Full size table
Table 6. Common severe failure (partially understands the task) categories and corresponding example responses in response to Fig. 3’s example prompt.
Full size table
Table 7. Common mild failure types (understands the task, but response quality is poor) and corresponding example responses in response to Fig. 3’s example prompt.
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Harmon, S., Rutman, S. (2023). Prompt Engineering for Narrative Choice Generation. In: Holloway-Attaway, L., Murray, J.T. (eds) Interactive Storytelling. ICIDS 2023. Lecture Notes in Computer Science, vol 14383. Springer, Cham. https://doi.org/10.1007/978-3-031-47655-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47655-6_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47654-9

  • Online ISBN: 978-3-031-47655-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation