Keywords

1 Introduction

Most behaviour change applications in today’s market are one-size-fits-all, and evidence suggests that personalisation is needed in order to bring about effective, engaging and long-lasting change [8]. Traditionally obtained using text questionnaires, personality traits have been used for personalising behaviour change applications [6]. However, completing questionnaires requires focused attention, as numerous cognitive processes are involved when answering questions about one’s behaviours [3]. Past research has indicated that focused attention is an important component of user engagement [2], and when users have fun, there is increase in cognitive absorption of technology use [1]. Measuring and improving user engagement is important to the Human Computer Interaction (HCI) community, as is evidenced by past literature [4, 9] that focused on this.

We present a study with a modified psychometric questionnaire using images and gifs (including pop culture references) to obtain the Big 5 personality traits. Inspired by the growth of communication through emojis, gifs, images and videos, over text, we test if the addition of a visual layer on top of psychometric questionnaires is as accurate in capturing personality as a gold-standard questionnaire; and makes the questionnaire more fun - potentially improving the engagement of commercial application that use these. Work has been done in the past to obtain personality scores by creating image based quizzesFootnote 1, and a patent [10] has been filed on this system. Our work does not create a new personality quiz for detecting personality traits, but rather augments visual imagery to a gold-standard personality questionnaire to improve its perception with users.

2 Study - Methodology

Our study required participants to complete two questionnaires: 1. the baseline questionnaire - the gold-standard that provides ‘ground truth’ data and 2. the visual questionnaire - our modified questionnaire. We do this to compare the scores and perception of our modified questionnaire with the gold-standard. The questionnaires were spaced one week apart so that participants don’t rely on memory to provide answers. Half of the participants were asked to complete the baseline questionnaire first, while the other half were asked to complete the visual questionnaire first.

Before the first questionnaire, we provided an onboarding survey to collect demographics, including gender and age range. At the end of both questionnaires, we provided an exit survey with two questions: 1. ‘Ease of Completion’ to understand how easy the questionnaire was to complete and 2. ‘Pleasure’ to understand how fun the questionnaire was. We use a 5 point Likert scale for these. The questionnaire was anonymous and no personal data including names and email IDs were collected from participants. Users were given unique codes to answer both questionnaires, enabling us to match responses from the same user.

There are multiple questionnaires to obtain the Big 5 personality traits, including the mini-IPIP developed by Donnellan et al. [5]. In this work, the authors developed a 20 item questionnaire with a likert scale to obtain the Big 5 personality traits, namely Extroversion, Agreeableness, Conscientiousness, Neuroticism and Openness. The questionnaire can be broken down into 4 brackets, with each bracket containing 1 question related to each of the 5 personalities. The questionnaire is well regarded as the gold-standard in the Psychological community, for its length and accuracy. We use this questionnaire as the baseline, over which we build the visual questionnaire.

In the visual questionnaire, we used questions from the baseline in the same order with the same connotation, but modified it to suit images and gifs that we selected. The selection of images or gifs for answer options was based on mutual agreement between the researchers conducting the study, with subsequent validation from a subject expert. We provided three options for each question in a Likert scale, ranging from positive/agree/strong to negative/disagree/week. This was done to reduce the cognitive effort it takes for the user to select between strongly agree and agree (and the opposite), and to overcome the difficulty of obtaining images that clearly differentiate two consecutive options. While it would have been ideal to use a 5 point Likert scale, and most questionnaires and surveys use this, evidence has indicated that 3 point Likert scales are just as accurate to obtain user opinions and can be used for surveys that don’t need a breadth of statistical tests [7].

Fig. 1.
figure 1

Question to obtain a component of (a) Extroversion, (b) Conscientiousness

Full size image

Two questions from the visual questionnaire are presented in Fig. 1. These are similar to the questions presented in the baseline questionnaire, albeit in a different form. To maintain consistency in the scores obtained from the baseline and visual questionnaires, the options for the baseline questionnaire were also presented in a 3 point Likert scale. The scores for each trait are calculated in the manner defined in [5] - positive questions were scored in the same order, and negative questions were reverse scored. Each trait has four questions that contribute to its score.

The study was performed by 26 participants, employed at a research institute in Barcelona, Spain. The call for participation was sent out to over 60 individuals using an internal mailing list, and participants registered to volunteer. The study was performed in late 2018. The number of females to males was 48% to 52%, and participants belonged to the age ranges of 18–24 (9%), 25–34 (56%), 35–44 (26%) and 45–54 (9%). Participants were citizens of countries including Spain, United Kingdom, United States, Germany and others, and included a mix of native and non-native English speakers.

3 Results

First we obtained the correlations (\(r-values\)) by regressing the trait scores obtained from both questionnaires. This is presented in Table 1. High correlation scores with significant \(p-values\) indicate that the visual questionnaire is able to obtain the personality scores with a high similarity as the text based questionnaire. We also observed that no difference was observed between the two groups of participants, i.e., correlations obtained from participants who took the baseline questionnaire first were similar to those who took the visual questionnaire first.

Table 1. Correlations (\(r-values\)) obtained between the visual questionnaire and the baseline questionnaire
Full size table

Next we obtained the correlation between the answer from each question in a bracket and the personality score obtained from the baseline, and the highest correlations are shown in Table 1. We note that the correlations are not comparable to those obtained from using all questions. Subsequently, we selected the 2 questions from the visual questionnaire with the highest correlations to obtain a score with 2 questions and correlated this with the baseline scores. It is observed that the correlations obtained are similar to those obtained using all 4 questions, indicating that using 2 questions for each trait, or 10 questions overall can provide similar scores versus using 20.

Fig. 2.
figure 2

Bar charts representing the rating distribution of (a) Ease of Completion, (b) Pleasure

Full size image

The results obtained from the exit questionnaire are represented using bar graphs in Fig. 2. Results from the ‘Ease of Completion’ question indicate that both the baseline and visual questionnaires are easy to complete, with slight preference for the baseline as they maintain the same text answer options for all questions. However the ‘Pleasure’ question indicated that the visual questionnaire was perceived as significantly more fun than the baseline.

4 Conclusion and Future Work

This study indicates that the use of visual media to obtain personality traits from psychometric questionnaires is both accurate and fun to complete - providing the potential to improvement user engagement. Feedback from one of the participants: “I loved this survey so much!! it was really engaging. I think all surveys should look like this.” further highlights its merit.

In the future, we plan to conduct a large scale study and explore if similar results hold with a larger population sample. We will incorporate this method in a commercial application for personalised interventions, explore improvements in user engagement and obtain further user feedback on this medium of answering questionnaires. This method also has the potential to be applied to other psychometric questionnaires, improving the application of psychology to HCI.