Draw Your Own Story: Paper and Pencil Interactive Storytelling

  • Edirlei Soares de Lima
  • Bruno Feijó
  • Simone Barbosa
  • Antonio L. Furtado
  • Angelo Ciarlini
  • Cesar Pozzer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6972)


In this paper, we present a storytelling system able to dramatize interactive narratives in augmented reality over a conventional sheet of paper. The system allows users to freely interact with virtual characters by sketching objects on the paper. Users interacting with the system can indirectly affect the characters’ decisions, even to the point of radically subverting the storyline.


Interactive Storytelling Augmented Reality Sketch-Based Interface 


  1. 1.
    Hagbi, N., Grasset, R., Bergig, O., Billinghurst, M., El-Sana, J.: In-Place Sketching for Content Authoring in Augmented Reality Games. In: Proceedings of IEEE Virtual Reality Conference 2010, Waltham, pp. 91–94 (2010)Google Scholar
  2. 2.
    Vogelsang, A., Signer, B.: The Lost Cosmonaut: An Interactive Narrative Environment on the Basis of Digitally Enhanced Paper. In: Proceedings of 3rd International Conference on Virtual Storytelling, Strasbourg, France, pp. 270–279 (2005)Google Scholar
  3. 3.
    Kuka, D., Elias, O., Martins, R., Lindinger, C., Pramböck, A., Jalsovec, A., Maresch, P., Hörtner, H., Brandl, P.: DEEP SPACE: High Resolution VR Platform for Multi-user Interactive Narratives. In: Proceedings of the 2nd Joint International Conference on Interactive Digital Storytelling, Guimarães, Portugal, pp. 185–196 (2009)Google Scholar
  4. 4.
    Bayon, V., Wilson, J.R., Stanton, D., Boltman, A.: Mixed reality storytelling environments. Virtual Reality 7(1), 54–63 (2003)CrossRefGoogle Scholar
  5. 5.
    Dow, S., Mehta, M., Lausier, A., MacIntyre, B., Mateas, M.: Initial Lessons from AR-Façade, An Interactive Augmented Reality Drama. In: ACM SIGCHI Conference on Advances in Computer Entertainment, Los Angeles (2006)Google Scholar
  6. 6.
    Mateas, M.: Interactive Drama, Art, and Artificial Intelligence. Ph.D. Thesis - School of Computer Science, Carnegie Mellon University, Pittsburgh (2002)Google Scholar
  7. 7.
    Cavazza, M., Charles, F., Mead, S.J., Martin, O., Marichal, X., Nandi, A.: Multimodal acting in mixed reality interactive storytelling. IEEE Multimedia 11(3), 30-39 (2004)Google Scholar
  8. 8.
    Huynh, D.T., Raveendran, K., Xu, Y., Spreen, K., MacIntyre, B.: Art of Defense: A Collaborative Handheld Augmented Reality Board Game. In: Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games, pp. 135–142 (2009)Google Scholar
  9. 9.
    Leitner, J., Köffel, C., Haller, M.: Bridging the gap between real and virtual objects for tabletop game. International Journal of Virtual Reality 7(3), 1–5 (2009)Google Scholar
  10. 10.
    Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17, 124–129 (1971)CrossRefGoogle Scholar
  11. 11.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  12. 12.
    El-Naqa, I., Yang, Y., Wernick, M.N., Galatsanos, N.P., Nishikawa, R.M.: A support vector machine approach for detection of microcalcifications. IEEE Transactions on Medical Imaging 21(12), 1552–1563 (2002)CrossRefGoogle Scholar
  13. 13.
    Cai, T., Hofmann, T.: Hierarchical document categorization with support vector machines. In: Proceedings of the 13th Conference on Information and Knowledge Management (2004)Google Scholar
  14. 14.
    Lima, E.S., Pozzer, C., Ornellas, M., Ciarlini, A., Feijó, B., Furtado, A.: Virtual Cinematography Director for Interactive Storytelling. In: Proceedings of the International Conference on Advances in Computer Entertainment Technology, Athens, Greece, pp. 263–270 (2009)Google Scholar
  15. 15.
    Hu, M.K.: Visual problem recognition by moment invariants. IRE Transactions on Information Theory 8, 179–187 (1962)zbMATHGoogle Scholar
  16. 16.
    Zhou, Z., Cheok, A.D., Tedjokusumo, J., Omer, G.S.: wIzQubesTM - A Novel Tangible Interface for Interactive Storytelling in Mixed Reality. International Journal of Virtual Reality 7(4), 9–15 (2008)Google Scholar
  17. 17.
    Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications 21(6), 34–47 (2001)CrossRefGoogle Scholar
  18. 18.
    Krevelen, D.W.F., van Poelman, R.: A survey of augmented reality technologies, applications and limitations. Int. J. of Virtual Reality 9(2), 1–20 (2010)Google Scholar
  19. 19.
    Jaimes, A., Sebe, N.: Multimodal Human-computer Interaction: A Survey. Computer Vision and Image Understanding 108(1-2), 116–134 (2007)CrossRefGoogle Scholar
  20. 20.
    Malkawi, A., Srinivasan, R.: Multimodal human–computer interaction for immersive visualization: integrating speech–gesture recognitions and augmented reality for indoor environments. In: Proceedings of the Seventh IASTED Conference on Computer Graphics and Imaging, pp. 171–175. ACTA Press, Kauai (2004)Google Scholar
  21. 21.
    Dow, S., Mehta, M., Harmon, E., MacIntyre, B., Mateas, M.: Presence and Engagement in an Interactive Drama. In: Proc. of SIGCHI Conf. on Human Factors in Computing Systems (CHI 2007), San Jose, pp. 1475–1484 (2007)Google Scholar
  22. 22.
    Cavazza, M., Lugrin, J.-L., Pizzi, D., Charles, F.: Madame bovary on the holodeck: immersive interactive storytelling. In: Proc. of the 15th International Conf. on Multimedia (MULTIMEDIA 2007), pp. 651–660 (2007)Google Scholar
  23. 23.
    Igarashi, T., Zeleznik, B.: Sketch-based interaction – Guest Editors’ Introduction. IEEE Computer Graphics and Applications 47(1), 26–27 (2007)CrossRefGoogle Scholar
  24. 24.
    Sutherland, I.E.: Sketchpad: a man-machine graphical communication system. In: Proc. of AFIPS 1963, pp. 329–346. ACM, New York (1963)Google Scholar
  25. 25.
    Bergig, O., Hagbi, N., El-Sana, J., Billinghurst, M.: In-place 3D sketching for authoring and augmenting mechanical systems. In: Proc. of the 2009 8th IEEE Int. Symposium on Mixed and Augmented Reality (ISMAR 2009), pp. 87–94 (2009)Google Scholar
  26. 26.
    Cai, Y., Miao, C., Tan, A.-H., Shen, Z.: A hybrid of plot-based and character-based interactive storytelling. In: Hui, K.-c., Pan, Z., Chung, R.C.-k., Wang, C.C.L., Jin, X., Göbel, S., Li, E.C.-L. (eds.) EDUTAINMENT 2007. LNCS, vol. 4469, pp. 260–273. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  27. 27.
    Ciarlini, A.E.M., Pozzer, C.T., Furtado, A.L., Feijo, B.: A logic-based tool for interactive generation and dramatization of stories. In: Proc. of the ACM SIGCHI Int. Conf. on Advances in Computer Entertainment Technology, Valencia, pp. 133–140 (2005)Google Scholar
  28. 28.
    Ciarlini, A.E.M., Camanho, M.M., Doria, T.R., Furtado, A.L., Pozzer, C.T., Feijo, B.: Planning and interaction levels for TV Storytelling. In: 1st Joint Int. Conf. on Interactive Digital Storytelling, Erfurt, Germany, pp. 198–209 (2008)Google Scholar
  29. 29.
    Silva, F.G.A., Ciarlini, A.E.M., Siqueira, S.W.M.: Nondeterministic Planning for Generating Interactive Plots. In: Kuri-Morales, A., Simari, G.R. (eds.) IBERAMIA 2010. LNCS(LNAI), vol. 6433, pp. 133–143. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  30. 30.
    Cavazza, M., Charles, F., Mead, S.: Character-based interactive storytelling. IEEE Intelligent Systems, Special Issue on AI in Interactive Entertainment 17(4), 17–24 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Edirlei Soares de Lima
    • 1
  • Bruno Feijó
    • 1
  • Simone Barbosa
    • 1
  • Antonio L. Furtado
    • 1
  • Angelo Ciarlini
    • 2
  • Cesar Pozzer
    • 3
  1. 1.Departamento de InformáticaPUC-RioBrazil
  2. 2.Departamento de Informática AplicadaUNIRIOBrazil
  3. 3.Departamento de Eletrônica e ComputaçãoUFSMSanta MariaBrazil

Personalised recommendations