In an attempt to understand what is and is not currently possible in terms of creating a chatbot which could form the basis of an avatar-level digital immortalisation, Daden, KSharp, the University of Worcester and the University of WarwickFootnote 1 have been working to create a virtual persona, which is a digital representation of some of the memories, knowledge, experiences, personality and opinions of a specific living physical human (Savin-Baden and Burden 2018). This work should be contrasted with, for example, the digital copies of Holocaust survivors Pinchas Gutter and Eva Schloss created by the USC Shoah Foundation (McMullan 2016) which work off a fixed store of around 1000 recordings of sentence level responses and have no flexibility in what they do. The intention, in the current study, is that a user would be able to interact with the Virtual Persona in the same way as they would with its physical subject, and that the Virtual Persona would present the same highly subjective and possibly flawed and biased information, views and opinions as its physical subject.
The scenario that is driving the development of the Virtual Persona is not the creation of a digital immortal per se, but rather the creation of a persona which could be ‘left behind’ when an employee moves from one job to another. It could be accessed by their successor in order to get advice on how to do the job, and opinions and information on projects, clients, customers, suppliers, technologies, procedures and staff. The project and persona are very much seen in the context of knowledge management and intended to explore how a virtual Persona could aid in knowledge capture, retention, distribution, access and use.
In the context of digital immortality though, the ability to access the acquired knowledge, experiences and insights of someone who is no longer available could offer guidance and support for those still living. Indeed a digital immortal would be created almost by accident if a persona were created for the purpose of retaining corporate knowledge if the subject suddenly died prematurely in an accident or from a terminal disease. It is the likelihood of such unfortunate incidents occurring that require an early consideration of the ethical, moral and legal issues around digital immortality.
For the purpose of the experiment and in order to keep within ethical guidelines, the subject was allowed to both filter and edit material provided where they felt that the real information could be compromising, since in reality, they were still continuing in their job. It should also be noted that creating a chatbot of a specific person being ‘interviewed’ in a one-on-one situation where the user/interviewer knows that they are talking to a chatbot places a very high bar on any attempt to pass the Turing Test—the benchmark for evaluating chatbots (Turing 1950). In contrast, the authors have been involved in building chatbots for two ‘covert’ Turing Tests, where the participants did not know they were talking to a chatbot and where the chatbot represented a generic personality and, in these cases, has achieved deception rates (i.e. percentage of users thinking they were talking to a human) of 80% (Gilbert and Forney 2015) and 100% (Burden et al. 2016). Thus, whilst being able to completely fool a user was beyond the scope of this project, there may well be lower levels of performance which can be achieved which still yield a useful tool, and the knowledge and expertise gained along the way may also have application in other areas of knowledge management. Since the virtual persona was based on a subject called Barry, it was only natural that it became referred to as Virtual Barry.
Virtual Barry Development
Virtual Barry was developed in an iterative manner over an approximate a 2-year period. Several reviews were held to check progress and identify areas for further work, which included:
-
face-to-face interviews followed by audio transcription
-
Skype voice interviews
-
Skype text-chat interviews
-
answering questions loaded onto an interview application so that the Subject could answer questions asynchronously
-
completion of spreadsheet grids to ensure consistent data collection on topics such as customers, projects and employees
-
import of curated data from the Subjects’ mobile phone, web browser, address book and calendar
-
entity extraction on social media posts and documents produced by the Subject
A disadvantage of several of these were that (a) the Subject was aware of the human involved in collecting the information and (b) the data had to be coded manually. In the later stages of the project, the Subject was given the ability to chat with the Virtual Persona. Throughout 2018, qualitative and quantitative user evaluations have been undertaken by people who either knew the Subject or knew the Subjects type of job and areas of expertise, or who knew both. This data is currently being analysed and will be the subject of further papers. This paper will present a broader reflection of some of the lessons learnt in virtual persona development and then examine the implications for digital immortalisation.
An image of the current (April 18) Virtual Barry interface is at Fig. 1.
Virtual Barry Reflections
During the Virtual Barry project, it became evident that the virtual persona was manifest through a combination of different elements within the system, which included the user interface, content, word choice and conversational style. The project is also looking at a more direct mapping of psychological profile information onto Virtual Barry, but that will not be considered here. However, the idea of there being different ‘flavours’ of the persona have emerged. Each of these elements will be considered in turn.
User Interface
If the virtual persona looks like Barry and sounds like Barry, then users are probably less inclined to focus on the content of what Virtual Barry says. Considering these two elements:
Image
Whilst the Sitepal system has enabled the team to animate an image of Barry, it loses some believability as it speaks, with the only just acceptable lipsync, ‘white space’ for teeth and facial distortions during some expressions. Having a better animated avatar is likely to improve the believability of Virtual Barry. As noted previously, the user interface is not a high priority for the project and the Sitepal system is still relatively unique in terms of ease of use and integration. Better systems are available as bespoke applications or within 3D tools such as Unity3D, but would take more time and effort to integrate. The project team continues to seek a better solution.
Voice
Until recently, only a relatively limited number of voice fonts were available from the main text-to-speech companies which has severely limited the ability to match Barry’s voice (for example, although Sitepal provides access to 4 leading text-to-speech engines covering almost 100 voices they offer, only 3 British English male voices and 5 British English female voices). However, in 2018, there has been a growth in companies offering personalised voice fonts, such as LyreBird (https://lyrebird.ai/), VocalID (https://www.vocalid.co/) and Modeltalker (https://www.modeltalker.org/). The team is evaluating these to see if they offer a credible alternative within the constraints of the project. In addition, Google has also released its own text-to-speech system, which whilst not offering a better match, does seem to provide a more modulated and expressive generic voice.
Content
As well as factual and interest information, Virtual Barry also needs to reflect the practical expertise of Barry. At the moment, Virtual Barry has very little of this expertise knowledge (in Human Factors in the Barry case), and this needs to be added to the system. However, much is procedural and related to decision choices and processes and, so, forms more of an aspect of procedural memory, which may be less amenable to the knowledge-graph approach and reflect instead aspects of more traditional expert systems.
Word and Phrase Choice
Verbatim responses from Barry which are encoded into Virtual Barry naturally have a high degree of ‘Barryness’. However, the longer such responses are, the fewer situations in which they will work. There is also a discontinuity when Virtual Barry moves from a constructed response to a verbatim response and back again. A key goal has to be to try and minimise that discontinuity and enable ‘Barryness’ to show through in shorter and more machine-generated responses. One way of minimising this discontinuity is to ensure that constructed responses are using the phrases and words that Barry would use. This two-stage process requires ensuring the responses sound human rather than machine like, and then tuning them to Barry’s vocabulary and style. It is also notable that there is a difference in even Barry’s ‘voice’ between text conversations and spoken conversations, and so the team has switched to using text conversations via Skype instead of audio interviews in order to collect data that are a better match.