Skip to main content

The Chronicles of ChatGPT: Generating and Evaluating Visual Novel Narratives on Climate Change Through ChatGPT

  • Conference paper
  • First Online:
Interactive Storytelling (ICIDS 2023)

Abstract

This paper explores the potential of utilizing ChatGPT, a large language model (LLM), for generating and evaluating visual novel (VN) game stories in the context of global warming awareness through a VN game. The study involves generating two stories using ChatGPT, one with given global warming related keywords as an inspiration for ChatGPT along with a specified ending and another without, and evaluating them based on several linguistic criteria: coherence, inspiration, readability, word complexity, and narrative fluency. Results reveal that keywords-inspired story exhibit higher coherence, while the basic one demonstrate greater inspiration. The findings highlight the advantages of each story and emphasize the value of AI-driven narrative generation in creating engaging and informative experiences. Furthermore, the study introduces an innovative approach by employing ChatGPT as an evaluator for the story quality, by combining various prompt engineering techniques showcasing the diverse applications of LLMs in interactive storytelling. This work contributes to the growing field of LLM-based story generation and underscores the potential of AI-driven narratives in fostering awareness and engagement on critical issues like climate change.

M. C. Gursesli and P. Taveekitworachai—These authors contributed equally.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://chat.openai.com.

  2. 2.

    The full conversation is available at https://bit.ly/chars-places.

References

  1. Alabdulkarim, A., Li, S., Peng, X.: Automatic story generation: challenges and attempts. arXiv preprint arXiv:2102.12634 (2021)

  2. Antle, A.N., Tanenbaum, T.J., Macaranas, A., et al.: Games for change: looking at models of persuasion through the lens of design. In: Playful User Interfaces: Interfaces that Invite Social and Physical Interaction, pp. 163–184 (2014)

    Google Scholar 

  3. Antle, A.N., Warren, J.L., May, A., et al.: Emergent dialogue: eliciting values during children’s collaboration with a tabletop game for change. In: Proceedings of the 2014 Conference on Interaction Design and Children, pp. 37–46 (2014)

    Google Scholar 

  4. Böhm, O., Jacobeit, J., Glaser, R., et al.: Flood sensitivity of the Bavarian alpine foreland since the late middle ages in the context of internal and external climate forcing factors. Hydrol. Earth Syst. Sci. 19(12), 4721–4734 (2015)

    Article  Google Scholar 

  5. Bontchev, B., Antonova, A., Terzieva, V., et al.: “let us save venice’’–an educational online maze game for climate resilience. Sustainability 14(1), 7 (2021)

    Article  Google Scholar 

  6. Bostan, B., Marsh, T.: The ‘Interactive’ of interactive storytelling: customizing the gaming experience. In: Yang, H.S., Malaka, R., Hoshino, J., Han, J.H. (eds.) ICEC 2010. LNCS, vol. 6243, pp. 472–475. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15399-0_63

    Chapter  Google Scholar 

  7. Brown, T.B., et al.: Language models are few-shot learners (2020)

    Google Scholar 

  8. Camingue, J., Carstensdottir, E., Melcer, E.F.: What is a visual novel? In: Proceedings of the ACM on Human-Computer Interaction, CHI PLAY, vol. 5, pp. 1–18 (2021)

    Google Scholar 

  9. Camingue, J., Melcer, E.F., Carstensdottir, E.: A (visual) novel route to learning: a taxonomy of teaching strategies in visual novels. In: Proceedings of the 15th International Conference on the Foundations of Digital Games, pp. 1–13 (2020)

    Google Scholar 

  10. De Araújo Luz Junior, J., Rodrigues, M.A.F., Hammer, J.: A storytelling game to foster empathy and connect emotionally with breast cancer journeys. In: 2021 IEEE 9th International Conference on Serious Games and Applications for Health(SeGAH), pp. 1–8 (2021). https://doi.org/10.1109/SEGAH52098.2021.9551860

  11. Engelbrecht, F., Monteiro, P.: Climate change: the ipcc’s latest assessment report. Quest 17(3), 34–35 (2021)

    Google Scholar 

  12. Fernández Galeote, D., Hamari, J.: Game-based climate change engagement: analyzing the potential of entertainment and serious games. In: Proceedings of the ACM on Human-Computer Interaction, CHI PLAY, vol. 5, pp. 1–21 (2021)

    Google Scholar 

  13. Ferreira, M., Nunes, N., Nisi, V.: Interacting with climate change: a survey of HCI and design projects and their use of transmedia storytelling. In: Mitchell, A., Vosmeer, M. (eds.) ICIDS 2021. LNCS, vol. 13138, pp. 338–348. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92300-6_33

    Chapter  Google Scholar 

  14. Fiorenza, M., Duradoni, M., Barbagallo, G., et al.: Implicit association test (iat) toward climate change: a prisma systematic review. Curr. Res. Ecol. Social Psychol. 4, 100103 (2023). https://doi.org/10.1016/j.cresp.2023.100103. https://www.sciencedirect.com/science/article/pii/S2666622723000163

  15. Flood, S., Cradock-Henry, N.A., Blackett, P., et al.: Adaptive and interactive climate futures: systematic review of ‘serious games’ for engagement and decision-making. Environ. Res. Lett. 13(6), 063005 (2018)

    Article  Google Scholar 

  16. Guan, J., Huang, F., Zhao, Z., Zhu, X., Huang, M.: A knowledge-enhanced pretraining model for commonsense story generation. Trans. Assoc. Comput. Linguist. 8, 93–108 (2020)

    Article  Google Scholar 

  17. Janakiraman, S., Watson, S.L., Watson, W.R.: Using game-based learning to facilitate attitude change for environmental sustainability. J. Educ. Sustain. Dev. 12(2), 176–185 (2018). https://doi.org/10.1177/0973408218783286

    Article  Google Scholar 

  18. Johnsson, F., Kjärstad, J., Rootzén, J.: The threat to climate change mitigation posed by the abundance of fossil fuels. Clim. Policy 19(2), 258–274 (2019)

    Article  Google Scholar 

  19. Kampa, A., Haake, S., Burelli, P.: Storytelling in serious games. In: Dörner, R., Göbel, S., Kickmeier-Rust, M., Masuch, M., Zweig, K. (eds.) Entertainment Computing and Serious Games. LNCS, vol. 9970, pp. 521–539. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46152-6_19

    Chapter  Google Scholar 

  20. Kar, S.: The impact of visuals on storytelling in visual novels (2023)

    Google Scholar 

  21. Laamarti, F., Eid, M., Saddik, A.E.: An overview of serious games. Int. J. Comput. Games Technol. 2014, 11 (2014)

    Article  Google Scholar 

  22. Liestoel, G.: Augmented reality storytelling - narrative design and reconstruction of a historical event in situ. Int. J. Interact. Mobile Technol. (iJIM) 13(12), 196–209 (2019). https://doi.org/10.3991/ijim.v13i12.11560. https://online-journals.org/index.php/i-jim/article/view/11560

  23. Liu, J., Liu, A., Lu, X., et al.: Generated knowledge prompting for commonsense reasoning (2022)

    Google Scholar 

  24. Martucci, A., Gursesli, M.C., Duradoni, M., Guazzini, A., et al.: Overviewing gaming motivation and its associated psychological and sociodemographic variables: a prisma systematic review. Human Behav. Emerg. Technol. 2023 (2023)

    Google Scholar 

  25. Min, S., et al.: Rethinking the role of demonstrations: what makes in-context learning work? (2022)

    Google Scholar 

  26. Naul, E., Liu, M.: Why story matters: a review of narrative in serious games. J. Educ. Comput. Res. 58(3), 687–707 (2020)

    Article  Google Scholar 

  27. Ndulue, C., Orji, R.: Games for change-a comparative systematic review of persuasive strategies in games for behavior change. IEEE Trans. Games 15(2), 121–133 (2023). https://doi.org/10.1109/TG.2022.3159090

    Article  Google Scholar 

  28. Orji, R., Mandryk, R.L., Vassileva, J.: Improving the efficacy of games for change using personalization models. ACM Trans. Comput.-Hum. Interact. (TOCHI) 24(5), 1–22 (2017)

    Article  Google Scholar 

  29. Øygardslia, K., Weitze, C.L., Shin, J.: The educational potential of visual novel games: principles for design (2020)

    Google Scholar 

  30. Schneider, S.H.: What is ‘dangerous’ climate change? Nature 411(6833), 17–19 (2001)

    Article  Google Scholar 

  31. Shaker, N., Togelius, J., Nelson, M.J.: Procedural Content Generation in Games. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-319-42716-4

    Book  Google Scholar 

  32. Sheepy, E.: Get water!: exploring the adult player’s experience of a mobile game for change. Master’s thesis, Concordia University (2015). https://spectrum.library.concordia.ca/id/eprint/980066/

  33. Simon, N., Muise, C.: TattleTale: storytelling with planning and large language models. In: ICAPS Workshop on Scheduling and Planning Applications (2022)

    Google Scholar 

  34. Skaraas, S.B., Gomez, J., Jaccheri, L.: Playing with empathy through a collaborative storytelling game. In: Clua, E., Roque, L., Lugmayr, A., Tuomi, P. (eds.) ICEC 2018. LNCS, vol. 11112, pp. 254–259. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99426-0_26

    Chapter  Google Scholar 

  35. Smiley, J.: Thirteen Ways of Looking at the Novel. Faber & Faber (2014)

    Google Scholar 

  36. Song, Z., Sun, Y., Ruijters, V., Lc, R.: Climate influence: implicit game-based interactive storytelling for climate action purpose. In: Mitchell, A., Vosmeer, M. (eds.) ICIDS 2021. LNCS, vol. 13138, pp. 425–429. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92300-6_42

    Chapter  Google Scholar 

  37. Steinemann, S.T., Mekler, E.D., Opwis, K.: Increasing donating behavior through a game for change: the role of interactivity and appreciation. In: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, CHI PLAY 2015, pp. 319–329. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2793107.2793125

  38. Sudhakaran, S., González-Duque, M., Glanois, C., et al.: Mariogpt: open-ended text2level generation through large language models (2023)

    Google Scholar 

  39. Swanson, B., Mathewson, K., Pietrzak, B., Chen, S., Dinalescu, M.: Story centaur: large language model few shot learning as a creative writing tool. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pp. 244–256 (2021)

    Google Scholar 

  40. Taveekitworachai, P., Abdullah, F., Dewantoro, M.F., et al.: Chatgpt4pcg competition: character-like level generation for science birds (2023)

    Google Scholar 

  41. Thue, D., Bulitko, V., Spetch, M., Wasylishen, E.: Interactive storytelling: a player modelling approach. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 3, pp. 43–48 (2007)

    Google Scholar 

  42. Todd, G., Earle, S., Nasir, M.U., et al.: Level generation through large language models. In: Proceedings of the 18th International Conference on the Foundations of Digital Games, FDG 2023. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3582437.3587211

  43. Wang, J., Liang, Y., Meng, F., et al.: Is chatgpt a good nlg evaluator? a preliminary study (2023)

    Google Scholar 

  44. Wang, K., Tekler, Z.D., Cheah, L., et al.: Evaluating the effectiveness of an augmented reality game promoting environmental action. Sustainability 13(24), 13912 (2021)

    Article  Google Scholar 

  45. Wei, J., Wang, X., Schuurmans, D., et al.: Chain-of-thought prompting elicits reasoning in large language models (2023)

    Google Scholar 

  46. White, J., Fu, Q., Hays, S., et al.: A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023)

  47. Yin, L., Ring, L., Bickmore, T.: Using an interactive visual novel to promote patient empowerment through engagement. In: Proceedings of the International Conference on the foundations of digital Games, pp. 41–48 (2012)

    Google Scholar 

  48. Yuan, A., Coenen, A., Reif, E., et al.: Wordcraft: story writing with large language models. In: 27th International Conference on Intelligent User Interfaces, pp. 841–852 (2022)

    Google Scholar 

  49. Zhao, Z., Han, F., Ma, X.: A live storytelling virtual reality system with programmable cartoon-style emotion embodiment. In: 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 102–1027. IEEE (2019)

    Google Scholar 

  50. Zhao, Z., Song, S., Duah, B., et al.: More human than human: llm-generated narratives outperform human-llm interleaved narratives. In: Proceedings of the 15th Conference on Creativity and Cognition, pp. 368–370 (2023)

    Google Scholar 

  51. Zwiers, F., Hegerl, G.: Attributing cause and effect. Nature 453(7193), 296–297 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Pittawat Taveekitworachai .

Editor information

Editors and Affiliations

Appendix

Appendix

1.1 A. Generated Knowledge of Criteria

1.2 B. VN Game Story Evaluation

We make our prompts, source code, and raw data available at https://github.com/Pittawat2542/chatgpt-visual-novel-evaluation.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gursesli, M.C. et al. (2023). The Chronicles of ChatGPT: Generating and Evaluating Visual Novel Narratives on Climate Change Through ChatGPT. In: Holloway-Attaway, L., Murray, J.T. (eds) Interactive Storytelling. ICIDS 2023. Lecture Notes in Computer Science, vol 14384. Springer, Cham. https://doi.org/10.1007/978-3-031-47658-7_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47658-7_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47657-0

  • Online ISBN: 978-3-031-47658-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics