Recognizing Coherent Narrative Blog Content

  • James Ryan
  • Reid Swanson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10045)


Interactive storytelling applications have at their disposal massive numbers of human-authored stories, in the form of narrative weblog posts, from which story content could be harvested and repurposed. Such repurposing is currently inhibited, however, in that many blog narratives are not sufficiently coherent for use in these applications. In a narrative that is not coherent, the order of the events in the narrative is not clear given the text of the story. We present the results of a study exploring automatic methods for estimating the coherence of narrative blog posts. In the end, our simplest model—one that only considers the degree to which story text is capitalized and punctuated—vastly outperformed a baseline model and, curiously, a series of more sophisticated models. Future work may use this simple model as a baseline, or may use it along with the classifier that it extends to automatically extract large numbers of narrative blog posts from the web for purposes such as interactive storytelling.


Coherence Blogs Content harvesting Machine learning 



This work would not have been possible without Marilyn Walker, who provided mentorship and funded the annotation procedure presented in this paper.


  1. 1.
    Adam, C., Cavedon, L.: A companion robot that can tell stories. In: Proceedings of Intelligent Virtual Agents (2013)Google Scholar
  2. 2.
    Antoun, C., Antoun, M., Ryan, J.O., Samuel, B., Swanson, R., Walker, M.A.: Generating natural language retellings from Prom Week play traces. In: Proceedings of the Procedural Content Generation (2015)Google Scholar
  3. 3.
    Barzilay, R., Lapata, M.: Modeling local coherence: an entity-based approach. In: Proceedings of the ACL (2005)Google Scholar
  4. 4.
    Barzilay, R., Lee, L.: Catching the drift: probabilistic content models, with applications to generation and summarization. In: Proceedings of the NAACL HLT (2004)Google Scholar
  5. 5.
    Bickmore, T., Schulman, D., Yin, L.: Engagement vs. deceit: virtual humans with human autobiographies. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS (LNAI), vol. 5773, pp. 6–19. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-04380-2_4 CrossRefGoogle Scholar
  6. 6.
    Burton, K., Java, A., Soboroff, I.: The ICWSM 2009 Spinn3r dataset. In: Proceedings of the Weblogs and Social Media (2009)Google Scholar
  7. 7.
    Eisenberg, J.D., Yarlott, W.V.H., Finlayson, M.A.: Comparing extant story classifiers: results and new directions. In: Proceedings of the CMN (2016)Google Scholar
  8. 8.
    Elsner, M., Charniak, E.: A unified local and global model for discourse coherence. In: Proceedings of the NAACL (2007)Google Scholar
  9. 9.
    Elsner, M., Charniak, E.: Coreference-inspired coherence modeling. In: Proceedings of the ACL (2008)Google Scholar
  10. 10.
    Frisina, P.G., Borod, J.C., Lepore, S.J.: A meta-analysis of the effects of written emotional disclosure on the health outcomes of clinical populations. Nerv. Ment. Dis. 192(9), 629–634 (2004)CrossRefGoogle Scholar
  11. 11.
    Gerber, M., Gordon, A.S., Sagae, K.: Open-domain commonsense reasoning using discourse relations from a corpus of weblog stories. In: Proceedings of the Formalisms and Methodology for Learning by Reading (2010)Google Scholar
  12. 12.
    Gordon, A., Bejan, C., Sagae, K.: Commonsense causal reasoning using millions of personal stories. In: Proceedings of the AAAI (2011)Google Scholar
  13. 13.
    Gordon, A., Swanson, R.: Identifying personal stories in millions of weblog entries. In: Proceedings of the Weblogs and Social Media (2009)Google Scholar
  14. 14.
    Gordon, A.S., Cao, Q., Swanson, R.: Automated story capture from internet weblogs. In: Proceedings of the Knowledge Capture (2007)Google Scholar
  15. 15.
    Gordon, A.S., Wienberg, C., Sood, S.O.: Different strokes of different folks: searching for health narratives in weblogs. In: Proceedings of the Social Computing (2012)Google Scholar
  16. 16.
    Guinaudeau, C., Strube, M.: Graph-based local coherence modeling. In: Proceedings of the ACL (2013)Google Scholar
  17. 17.
    Guzdial, M., Harrison, B., Li, B., Riedl, M.O.: Crowdsourcing open interactive narrative. In: Proceedings of the Foundations of Digital Games (2015)Google Scholar
  18. 18.
    Hand, D.J., et al.: Classifier technology and the illusion of progress. Stat. Sci. 21, 1–15 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Harrison, B., Riedl, M.O.: Towards learning from stories: an approach to interactive machine learning. In: Proceedings of the AAAI (2015)Google Scholar
  20. 20.
    Hu, C., Walker, M.A., Neff, M., Fox Tree, J.E.: Storytelling agents with personality and adaptivity. In: Brinkman, W.-P., Broekens, J., Heylen, D. (eds.) IVA 2015. LNCS (LNAI), vol. 9238, pp. 181–193. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-21996-7_19 CrossRefGoogle Scholar
  21. 21.
    Kincaid, J.P., et al.: Derivation of new readability formulas for Navy enlisted personnel. Technical report, DTIC Document (1975)Google Scholar
  22. 22.
    Labov, W.: Uncovering the event structure of narrative. In: Round Table on Language and Linguistics (2003)Google Scholar
  23. 23.
    Lapata, M., Barzilay, R.: Automatic evaluation of text coherence: models and representations. In: Proceedings of the IJCAI (2005)Google Scholar
  24. 24.
    Li, B., Lee-Urban, S., Appling, D.S., Riedl, M.O.: Crowdsourcing narrative intelligence. Adv. Cogn. Syst. 2(1), 25–42 (2012)CrossRefGoogle Scholar
  25. 25.
    Li, B., Lee-Urban, S., Johnston, G., Riedl, M.: Story generation with crowdsourced plot graphs. In: Proceedings of the AAAI (2013)Google Scholar
  26. 26.
    Li, B., Thakkar, M., Wang, Y., Riedl, M.O.: Data-driven alibi story telling for social believability. In: Proceedings of the Social Believability in Games (2014)Google Scholar
  27. 27.
    Lin, Z., Ng, H.T., Kan, M.Y.: Automatically evaluating text coherence using discourse relations. In: Proceedings of the ACL: HLT (2011)Google Scholar
  28. 28.
    Louis, A., Nenkova, A.: A coherence model based on syntactic patterns. In: Proceedings of the EMNLP-CoNLL (2012)Google Scholar
  29. 29.
    Lukin, S.M., Bowden, K., Barackman, C., Walker, M.A.: Personabank: a corpus of personal narratives and their story intention graphs. In: Proceedings of the LREC (2016)Google Scholar
  30. 30.
    Lukin, S.M., Reed, L.I., Walker, M.A.: Generating sentence planning variations for story telling. In: Proceedings of the SIGDIAL (2015)Google Scholar
  31. 31.
    Lukin, S.M., Ryan, J.O., Walker, M.A.: Automating direct speech variations in stories and games. In: Proceedings of the GAMNLP (2014)Google Scholar
  32. 32.
    Lukin, S.M., Walker, M.A.: Narrative variations in a virtual storyteller. In: Brinkman, W.-P., Broekens, J., Heylen, D. (eds.) IVA 2015. LNCS (LNAI), vol. 9238, pp. 320–331. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-21996-7_34 CrossRefGoogle Scholar
  33. 33.
    Manshadi, M., Swanson, R., Gordon, A.S.: Learning a probabilistic model of event sequences from internet weblog stories. In: Proceedings of the FLAIRS (2008)Google Scholar
  34. 34.
    Michaels, S.: “Sharing time”: children’s narrative styles and differential access to literacy. Lang. Soc. 10(03), 423–442 (1981)CrossRefGoogle Scholar
  35. 35.
    Owsley, S.H., Hammond, K.J., Shamma, D.A., Sood, S.: Buzz: telling compelling stories. In: Proceedings of the Multimedia (2006)Google Scholar
  36. 36.
    Paolacci, G., Chandler, J., Ipeirotis, P.G.: Running experiments on amazon mechanical turk. Judgment Decis. Making 5(5), 411–419 (2010)Google Scholar
  37. 37.
    Pennebaker, J.W., Francis, L.E., Booth, R.J.: LIWC: Linguistic Inquiry and Word Count., Austin (2001)Google Scholar
  38. 38.
    Pennebaker, J.W., Seagal, J.D.: Forming a story: the health benefits of narrative. Clin. Psychol. 55(10), 1243–1254 (1999)CrossRefGoogle Scholar
  39. 39.
    Pitler, E., Nenkova, A.: Revisiting readability: a unified framework for predicting text quality. In: Proceedings of the EMNLP (2008)Google Scholar
  40. 40.
    Prasad, R., et al.: The Penn Discourse Treebank 2.0 annotation manual (2007)Google Scholar
  41. 41.
    Rishes, E., Lukin, S.M., Elson, D.K., Walker, M.A.: Generating different story tellings from semantic representations of narrative. In: Koenitz, H., Sezen, T.I., Ferri, G., Haahr, M., Sezen, D., C̨atak, G. (eds.) ICIDS 2013. LNCS, vol. 8230, pp. 192–204. Springer, Heidelberg (2013). doi: 10.1007/978-3-319-02756-2_24 CrossRefGoogle Scholar
  42. 42.
    Roemmele, M.: Writing stories with help from recurrent neural networks. In: Proceedings of the AAAI (2015)Google Scholar
  43. 43.
    Roemmele, M., Gordon, A.S.: Creative help: a story writing assistant. In: Schoenau-Fog, H., Bruni, L.E., Louchart, S., Baceviciute, S. (eds.) ICIDS 2015. LNCS, vol. 9445, pp. 81–92. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-27036-4_8 CrossRefGoogle Scholar
  44. 44.
    Santorini, B.: Part-of-speech tagging guidelines for the Penn Treebank Project (3rd revision) (1990)Google Scholar
  45. 45.
    Soricut, R., Marcu, D.: Discourse generation using utility-trained coherence models. In: Proceedings of the COLING/ACL (2006)Google Scholar
  46. 46.
    Swanson, R., Gordon, A.S.: Say anything: a massively collaborative open domain story writing companion. In: Spierling, U., Szilas, N. (eds.) ICIDS 2008. LNCS, vol. 5334, pp. 32–40. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-89454-4_5 CrossRefGoogle Scholar
  47. 47.
    Traum, D., et al.: Hassan: a virtual human for tactical questioning. In: Proceedings of the SIGDIAL (2007)Google Scholar
  48. 48.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Expressive Intelligence StudioUniversity of California, Santa CruzSanta CruzUSA
  2. 2.Institute for Creative TechnologiesUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations