Abstract
In this chapter, we present Interact—a project which builds question-answering virtual humans based on pre-recorded video testimonies for Holocaust education. It was created to preserve the powerful and engaging experience of listening to, and interacting with, Holocaust survivors, allowing future generations of audience access to their unique stories. Interact demonstrates how advanced filming techniques, 3D graphics and natural language processing can be integrated and applied to specially recorded testimonies to enable users to ask questions and receive answers from virtualised individuals. This provides a new and rich interactive narrative of remembrance to engage with primary testimony. We briefly reviewed the literature of conversational natural language interfaces; discussed the design and development of Interact, including how we mapped the current proceedings of testimony and question-answering session to human-computer interaction and how we generated/predicted questions for each survivor using a lifeline chart; the 3D data capture process, generating 3D human; and natural language processing; and argued that this new form of mixed reality is a promising media to overcome the uncanny valley. Subjective and objective evaluation is also reported. The chapter is a longer version of a short paper presented at the ACM OzCHI conference (Ma et al., Interact: a mixed reality virtual survivor for holocaust testimonies. In: The proceedings of 27th annual meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI ‘15). ACM, New York, pp 250–254, 2015).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Artstein, R., Traum, D., Alexander, O., Leuski, A., Jones, A., Georgila, K., Debevec, P., Swartout, W., Maio, H. and Smith, S. Time-offset interaction with a holocaust survivor. In: Proceedings of the 19th international conference on intelligent user interfaces, pp. 163–168, ACM, New York (2014)
Brenton, H., Cillies, M., Ballin, D., Chatting, D.: The Uncanny Valley: does it exist? In the 11th international conference on human-computer interaction. Lawrence Erlbaum Associates, Las Vegas (2005)
Byrne, W., Doermann, D., Franz, M., Gustman, S., Hajic, J., Oard, D., Picheny, M., Psutka, J., Ramabhadran, B., Soergel, D., et al.: Automatic recognition of spontaneous speech for access to multilingual oral history archives. IEEE Speech Audio Process. 12(4), 420–435 (2004)
Delecroix, F., Morge, M., Routier, J.: A virtual selling agent which is persuasive and adaptive. In: Ossowski, S. (ed.) Agreement Technologies, pp. 625–645. Springer, Dordrecht (2012)
Fellbaum, C.: WordNet. Blackwell Publishing Ltd. (1998).
Hoque, M., Courgeon, M., Martin, J. Mutlu, B., Picard, R.: MACH: my automated conversation coach. In The proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computing, pp. 697–706. ACM, New York (2013)
Leuski, A., Traum, D.: NPCEditor: creating virtual human dialogue using information retrieval techniques. AI Mag. 32(2), 42–56 (2011)
Ma, M., Coward, S., Walker, C.: Interact: a mixed reality virtual survivor for holocaust testimonies. In: the Proceedings of 27th annual meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI ‘15), 250–254, ACM, New York (2015)
Milgram, P., Kishino, A.F.: Taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst., 1321–1329 (1994)
Pandorabots.com: Chatbot Captain Kirk. [Online] Available at: http://sheepridge.pandorabots.com/pandora/talk?botid=fef38cb4de345ab1&skin=iframe-voice (n.d.). Accessed 1 Apr 2016.
Swartout, W., Traum, D., Artstein, R., Noren, D., Debevec, P., Bronnenkant, K., Williams, J., Leuski, A., Narayanan, S., Piepol, D. Ada and Grace: toward realistic and engaging virtual museum guides. 10th International Conference on Intelligent Virtual Agents, pp. 286–300 (2010)
Talbot, T., Sagae, K., John, B., Rizzo, A.: Sorting out the virtual patient: how to exploit artificial intelligence, game technology and sound educational practices to create engaging role-playing simulations. Int. J. Gaming Comput.-Mediat. Simul. 4(3), 1–19 (2012)
Wilks, Y., Catizone, R., Worgan, S., Dingli, A., Moore, R., Field, D., Cheng, W.: A prototype for a conversational companion for reminiscing about images. Comput. Speech Lang. 25, 140–157., Elsevier (2011)
Acknowledgements
The Interact project research and technical development were supported by the Digital R&D Fund for the Arts, which is jointly funded by NESTA, Arts and Humanities Research Council and public funding by the National Lottery through Arts Council England. The project as a whole was also supported by a range of funders including the Pears Foundation and the Association of Jewish Refugees.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Ma, M., Coward, S., Walker, C. (2017). Question-Answering Virtual Humans Based on Pre-recorded Testimonies for Holocaust Education. In: Ma, M., Oikonomou, A. (eds) Serious Games and Edutainment Applications . Springer, Cham. https://doi.org/10.1007/978-3-319-51645-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-51645-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51643-1
Online ISBN: 978-3-319-51645-5
eBook Packages: Computer ScienceComputer Science (R0)