Skip to main content

SceneMaker: Creative Technology for Digital StoryTelling

Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST,volume 196)

Abstract

The School of Creative Arts & Technologies at Ulster University (Magee) has brought together the subject of computing with creative technologies, cinematic arts (film), drama, dance, music and design in terms of research and education. We propose here the development of a flagship computer software platform, SceneMaker, acting as a digital laboratory workbench for integrating and experimenting with the computer processing of new theories and methods in these multidisciplinary fields. We discuss the architecture of SceneMaker and relevant technologies for processing within its component modules. SceneMaker will enable the automated production of multimodal animated scenes from film and drama scripts or screenplays. SceneMaker will highlight affective or emotional content in digital storytelling with particular focus on character body posture, facial expressions, speech, non-speech audio, scene composition, timing, lighting, music and cinematography. Applications of SceneMaker include automated simulation of productions and education and training of actors, screenwriters and directors.

Keywords

  • SceneMaker
  • Natural language processing
  • Speech processing
  • Artificial intelligence (AI)
  • Affective computing
  • Computer graphics
  • Cinematography
  • 3D visualisation
  • Digital storytelling
  • Storyboards
  • Film
  • Drama
  • Dance
  • Music technology
  • Design

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-55834-9_4
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-55834-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   72.00
Price excludes VAT (USA)
Fig. 1.

References

  1. Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J.: SceneMaker: intelligent multimodal visualisation of natural language scripts. In: Coyle, L., Freyne, J. (eds.) AICS 2009. LNCS (LNAI), vol. 6206, pp. 144–153. Springer, Heidelberg (2010). doi:10.1007/978-3-642-17080-5_17

    CrossRef  Google Scholar 

  2. Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J.: SceneMaker: automatic visualisation of screenplays. In: Mertsching, B., Hund, M., Aziz, Z. (eds.) KI 2009. LNCS (LNAI), vol. 5803, pp. 265–272. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04617-9_34

    CrossRef  Google Scholar 

  3. Ma, M., McKevitt, P.: Virtual human animation in natural language visualisation. Artif. Intell. Rev. 25(1–2), 37–53 (2006). Special Issue on the 16th Artificial Intelligence and Cognitive Science Conference (AICS 2005)

    CrossRef  Google Scholar 

  4. Muñoz, K., Mc Kevitt, P., Lunney, T., Noguez, J., Neri, L.: Designing and evaluating emotional student models for game-based learning. In: Ma, M., Oikonomou, A., Jain, L.C. (eds.) Serious Games and Edutainment Applications, London, England, pp. 245–272. Springer, Heidelberg (2011). Also presented at the First International Workshop on Serious Games Development and Applications (SGDA 2010), School of Computing, University of Derby, England, 8 July 2011

    Google Scholar 

  5. Mulholland, E., Mc Kevitt, P., Lunney, T., Farren, J., Wilson, J.: 360-MAM-affect: sentiment analysis with the google prediction API and EmoSenticNet. In: Proceedings of the 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN-2015), Politecnico di Torino, Turin (Torino), Italy, 10–12, 1–5 June 2015. EUDL EAI Endorsed Transactions on Scalable Information Systems vol. 2, no. 6, p. e5, November 2015

    Google Scholar 

  6. Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. Oxford University Press, England (1997)

    Google Scholar 

  7. Mehrabian, A.: Framework for a comprehensive description and measurement of emotional states. Gen. Soc. Gen. Psychol. Monogr. 121(3), 339–361 (1995). Heldref Publishing

    Google Scholar 

  8. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)

    CrossRef  Google Scholar 

  9. Francisco V., Hervás R., Gervás P.: Two different approaches to automated mark up of emotions in text. In: Bramer M., Coenen F., Tuson A. (eds.) Research and Development in Intelligent Systems XXIII. Springer, London (2007)

    Google Scholar 

  10. Strapparava, C., Mihalcea, R.: Learning to identify emotions in text. In: Proceedings of the 2008 ACM Symposium on Applied Computing, SAC 2008, pp. 1556–1560. ACM, New York (2008)

    Google Scholar 

  11. Su, W.-P., Pham, B., Wardhani, A.: Personality and emotion-based high-level control of affective story characters. IEEE Trans. Vis. Comput. Graph. 13(2), 281–293 (2007)

    CrossRef  Google Scholar 

  12. Liu, H., Lieberman, H., Selker, T.: A model of textual affect sensing using real-world knowledge. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, IUI 2003, pp. 125–132. ACM, New York (2003)

    Google Scholar 

  13. Liu, H., Singh, P.: ConceptNet: a practical commonsense reasoning toolkit. BT Technol. J. 22(4), 211–226 (2004). Springer, The Netherlands

    Google Scholar 

  14. Shaikh, M.A.M., Prendinger, H., Ishizuka, M.: A linguistic interpretation of the OCC emotion model for affect sensing from text. In: Tao, J., Tan, T. (eds.) Affective Information Processing, pp. 45–73. Springer, London (2009)

    Google Scholar 

  15. Prendinger, H., Ishizuka, M.: SCREAM: scripting emotion-based agent minds. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 1, AAMAS 2002, pp. 350–351. ACM, New York (2002)

    Google Scholar 

  16. Gebhard, P.: ALMA - layered model of affect. In: Proceedings of the 4th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005), Utrecht University, The Netherlands, pp. 29–36. ACM, New York (2005)

    Google Scholar 

  17. De Raad, B.: The big five personality factors. In: The Psycholexical Approach to Personality. Hogrefe & Huber, Cambridge, MA, USA (2000)

    Google Scholar 

  18. Mancini, M., Pelachaud, C.: Implementing distinctive behavior for conversational agents. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS, vol. 5085, pp. 163–174. Springer, Heidelberg (2009). doi:10.1007/978-3-540-92865-2_17

    CrossRef  Google Scholar 

  19. Coyne, B., Sproat, R.: WordsEye: an automatic text-to-scene conversion system. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 487–496. ACM Press, Los Angeles (2001)

    Google Scholar 

  20. De Melo, C., Paiva, A.: Multimodal expression in virtual humans. Comput. Animation Virtual Worlds 17(3–4), 239–348 (2006)

    CrossRef  Google Scholar 

  21. Liu, Z., Leung, K.: Script visualization (ScriptViz): a smart system that makes writing fun. Soft Comput. 10(1), 34–40 (2006)

    CrossRef  Google Scholar 

  22. Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J.: NewsViz: emotional visualization of news stories. In: Inkpen, D., Strapparava, C. (eds.) Proceedings of the NAACL-HLT Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, Millennium Biltmore Hotel, Los Angeles, CA, USA, pp. 125–130, 5 June 2010

    Google Scholar 

  23. Rasheed, Z., Sheikh, Y., Shah, M.: On the use of computable features for film classification. IEEE Trans. Circ. Syst. Video Technol. 15(1), 52–64 (2005). IEEE Circuits and Systems Society

    CrossRef  Google Scholar 

  24. Shim, H., Kang, B.G.: CAMEO - camera, audio and motion with emotion orchestration for immersive cinematography. In: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, ACE 2008, vol. 352, pp. 115–118. ACM, New York (2008)

    Google Scholar 

  25. Kuo, F., Chiang, M., Shan, M., Lee, S.: Emotion-based music recommendation by association discovery from film music. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, MULTIMEDIA 2005, pp. 507–510. ACM, New York (2005)

    Google Scholar 

  26. Wahlster, W.: Smartkom: Foundations of Multimodal Dialogue Systems. Springer, Berlin/Heidelberg (2006)

    CrossRef  Google Scholar 

  27. Final Draft (2016). http://www.finaldraft.com. Accessed 11 Apr 2016

  28. Maguire, T.: Performing Story on the Contemporary Stage. Palgrave MacMillan, Basingstoke (2015)

    CrossRef  Google Scholar 

  29. Fitzpatrick, L.: Gender and affect in testimonial performance. Irish Univ. Rev. 45(1), 126–140 (2015)

    CrossRef  Google Scholar 

  30. Solon, A.J., Mc Kevitt, P., Curran, K.: TeleMorph: a fuzzy logic approach to network-aware transmoding in mobile intelligent multimedia presentation systems. In: Dumitras, A., Radha, H., Apostolopoulos, J., Altunbasak, Y. (eds.) Special issue on Network-Aware Multimedia Processing and Communications (2007). IEEE Journal of Selected Topics in Signal Processing vol. 1, no. 2, pp. 254–263, August 2007

    Google Scholar 

  31. Bridges, B., Graham, R.: Electroacoustic music as embodied cognitive praxis: Denis Smalley’s theory of spectromorphology as an implicit theory of embodied cognition. In: Proceedings of the Electroacoustic Music Studies Network Conference, The Art of Electroaoustic Music (EMS 2015), University of Sheffield, England

    Google Scholar 

  32. Moore, P., Lyons, F., O’Hanlon, G.: The river still sings. In: Digital Arts and Humanities Conference, Ulster University, Magee, Derry/Londonderry, Northern Ireland, 10–13 September. Digital Humanities Ireland, 20 pp. (2013)

    Google Scholar 

  33. Melvin, A.: Sonic motifs, structure and identity in Steve McQueen’s hunger. Soundtrack 4(1), 23–32 (2011)

    MathSciNet  CrossRef  Google Scholar 

  34. Cunningham, H.: General Architecture for Text Engineering (GATE) (2016). http://www.gate.ac.uk. Accessed 11 Apr 2016

  35. Connexor (2016). http://www.connexor.com/nlplib. Accessed 11 Apr 2016

  36. Tesniere, L.: Elements de syntaxe structurale. Klincksieck, Paris (1959)

    Google Scholar 

  37. Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  38. Strapparava, C., Valitutti, A.: WordNet-affect: an affective extension of WordNet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal, pp. 1083–1086, 26–28 May 2004

    Google Scholar 

  39. Lexical Conceptual Structure (LCS) Database (2016). http://www.umiacs.umd.edu/~bonnie/LCS_Database_Documentation.html. Accessed 11 Apr 2016

  40. Gelbukh, A.: EmoSenticNet (2014). http://www.gelbukh.com/emosenticnet/. Accessed 20 Feb 2014

  41. RapidMiner: RapidMiner (2015). https://rapidminer.com/. Accessed 4 May 2015

  42. Humanoid Animation Working Group (2016). http://www.h-anim.org. Accessed 11 Apr 2016

  43. FreeTTS 1.2 - A speech synthesizer written entirely in the JavaTM programming language (2016). http://freetts.sourceforge.net/docs/index.php. Accessed 11 Apr 2016

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Paul Mc Kevitt .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper

Akser, M. et al. (2017). SceneMaker: Creative Technology for Digital StoryTelling. In: Brooks, A., Brooks, E. (eds) Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT DLI 2016 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-55834-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55834-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55833-2

  • Online ISBN: 978-3-319-55834-9

  • eBook Packages: Computer ScienceComputer Science (R0)