1 Introduction

Regular physical exercise is beneficial for a person’s physical as well as psychological well-being [49]. Unfortunately, a great number of people globally do not engage in the recommended levels of physical activity [18]. Researchers are increasingly designing technologies to support people in changing their behavior and reaching their physical activity goals [19]. However, designing and developing effective motivational technology remains challenging. Among the solutions offered in literature are as follows: basing motivational strategies on existing behavior change theory [33], and tailoring the strategies to characteristics of the user [38]. However, what characteristics should one tailor an exercise application’s theory-based motivational strategies to?

Table 1 The processes of change divided in experiential and behavioral processes with a short description
Full size table

There are many theories and models of behavior change, but most are not conceptualized with practically applicable strategies, or with technology as the system delivering the strategies. The transtheoretical model (TTM) from Prochaska et al. [41] is a behavior change model that is arguably well suited for use in a behavior change system, because it provides a framework for the conceptualization of behavior change and for the measurement of change, and strategies to promote change [37]. There are two main concepts in the TTM: the stages of change and the processes of change (see Fig. 1 and Table 1). The stages of change classify people into progressing stages of behavior change. These stages range from long-term inactive (i.e., precontemplation) to long-term active (i.e., maintenance). The processes of change are those strategies that are deemed effective to motivate people to progress through the stages of change, but not every process is thought suitable for every stage (see Fig. 1). Based on a recent meta-analysis of 85 studies [40], it is the most widely used behavior change model in health-related persuasive technologies. Moreover, it has proven to be effective in combination with tailored text messages to influence someone’s physical activity behavior [34]. Therefore, we opt to use the TTM as the basis for designing behavior change strategies.

Fig. 1
figure 1

A simplified linear representation of the association of the stages with the accompanying processes. PC\(~{}=~\)precontemplation, C\(~{}={}~\)contemplation, P\(~{}={}~\)preparation, A\(~{}={}~\)action, and M\(~{}={}~\)maintenance. The length of the bar indicates over which stages to use the processes

Full size image

When tailoring strategies to user characteristics to increase effectiveness, it is important to understand what factors influence someone engaging in physical activity. Research into factors that correlate with physical activity has been numerous (e.g., [5, 13, 46]), with factors found such as age, gender, education, and socioeconomic status (and others). But research into systems that can adapt their interventions or strategies based on these factors is still scarce. Some recent studies investigate how systems could tailor the strategies adopted to people’s individual preferences for certain strategies [24, 29]. One way this can be done is by building profiles of the users that account for factors that are relevant for tailoring strategies [25], for example by selecting strategies which fit their personality [3].

How, then, should such strategies or interventions be operationalized and presented to the user? Mutsuddi and Connelly [34] showed that tailored text messages in combination with a behavior change theory or model to influence someone’s behavior can be effective for physical activity. Rimer and Kreuter [45] showed that tailoring text messages to a theory or model can enhance motivation to attend to and process health information. This paper will focus on text messages as the modality to deliver our strategies. Unfortunately, as pointed out by Latimer et al. [30], studies describing the development of such technology do not yet explain in detail how the researchers designed the motivational messages used. Therefore, there are no best practices available to construct these messages.

Our long-term goal is to design a behavior change system that tailors motivational strategies for physical activity to the user’s characteristics and is well grounded by principles from existing behavior change theories or models.

In previous work [48], we operationalized behavior change strategies by collecting peer-designed motivational messages through a crowdsourcing survey and aligning these messages to the processes of change strategies from the TTM. Our goal was to tailor the selection of processes to a user’s stage of change as described by the TTM. In a subsequent survey [48], we evaluated how motivating people from all the stages of change found the text messages representing the ten processes of change. Our expectation was to find that some processes were perceived more motivating for the earlier stages, and other processes for the later stages. In contrast to our expectations, we found that the processes of change as represented by the text messages were all rated as motivating in the later stages of change, but none in the earlier stages. One interpretation of these results may be that the stage of change someone is in is actually not a very discriminating factor for the use of the processes. Because the ratings of all the processes seem to display a similar trend, they might as well be used the same way for all the stages.

We wonder whether different people need different strategies to motivate them regardless of their stage of change, and therefore, in this paper, we investigate whether other factors, like gender and personality (from data measured in the same study), are important factors for how motivating different behavior change strategies are perceived.

To summarize, in this paper, we investigate whether user characteristics (beyond the stage of change someone is in) influence how motivating text messages are perceived. We examine how ratings of 50 motivational messages (see Appendix) are related to the participants’ gender and personality. We expect that both factors personality and gender influence how motivating people perceive behavior change text messages. By identifying how motivating various theory-based strategies are perceived based on personality and gender, our results inform the development of behavior change systems and contribute to the use and operationalization of theory-based strategies.

In the following sections, we report on related work, the method, results and discussion of the results, design recommendations, and limitations of the current study. We end with an overall conclusion.

2 Background

Review papers indicate that there are many factors influencing physical activity. Dishman et al. [13] reviewed a list of 38 determinants of physical activity which they divided into personal (e.g., age and education), environmental, and activity characteristics. Sherwood and Jeffery [46] classified 13 determinants into two categories, namely personal (e.g., stage of change and motivation) and environmental characteristics. Bauman et al. [5] classified their 61 characteristics differently: demographic and biological factors (e.g., age, education, gender (male only) and socioeconomic status); psychological, cognitive, and emotional factors (e.g., personality); behavioral attributes and skills; social and cultural factors; physical environment factors; and physical activity characteristics.

An interesting factor that influences physical activity is personality [5], while personality has been found to be a determinant in behavior [2], it is not widely adopted to tailor interventions for physical activity to, possibly because most effects found are small to medium correlations [43]. For this paper, we were especially interested in personality as a characteristic to tailor interventions to because personality has been found to be, arguably, a relatively stable personal characteristic in adults [44]. If personality is a stable characteristic and if personality varies across different people, we could tailor motivational messages to a person’s personality profile. Because personality remains constant over time, this would allow for a longer-term personalization. Courneya et al. [10] showed that exercise behavior, motives, barriers, and preferences were correlated to the personality traits of the Big Five. For instance, it was found that Openness was positively correlated to health and stress relief, Extraversion was positively correlated to exercising with other people, Extraversion and Openness were negatively correlated to supervised exercise, and Agreeableness was negatively correlated to competitiveness. These correlations show that people with different personalities have distinct considerations when it comes to exercising. This is consistent with Ingledew and Markland [22], where they found that different personalities have distinct motivations for change in exercise participation, for instance, Openness positively correlated with a health/fitness motivation. These studies indicate that people with different personalities should be motivated in different ways to participate in exercise and encourage personality-tailored interventions for physical activity (e.g., [43, 44]).

Halko et al. [17] explored the relationship between personality (i.e., Big Five) and strategies in the context of health-promotion with mobile applications. Their results showed that people with different personality types had distinct preferences for (mobile) persuasive messages (for healthy living). Hirsh et al. [20] let participants rate the level of persuasiveness of the messages for advertisements that were tailored to each personality type. For example, people with the Extraversion personality type would receive messages like “With XPhone, you’ll always be where the excitement is” [20, p. 579] because extraverts are especially sensitive to rewards and social attention. The results showed a clear benefit of tailoring messages to personality type. Similar results were obtained in a study by Adnan et al. [1] where an application was developed to persuade users to study more using persuasion strategies that were tailored to users’ personalities: different personalities indeed preferred different (persuasive) study behaviors. Similar results were also obtained by Alkış and Temizel [3], where they showed significant relations between personality traits and persuasive strategies.

Kristan and Suffoletto [27] looked at how men and women rated feedback messages to reduce hazardous alcohol consumption and found that overall women responded more favorably to all feedback messages, especially on messages that facilitated a strategy. Yan et al. [50] found that women were not driven by competitive messages but men were. Busch et al. [8] personalized persuasive strategies based on gender (measured on a fine-grained scale from masculine to feminine) and found that gender is a reliable variable for personalization, with femininity being significantly related to seven of the ten strategies. Overall, these studies indicate the potential effectiveness of tailoring to personality or gender when designing motivational strategies to influence participants. Unfortunately, research is scarce on how systems should tailor motivational strategies or interventions for physical activity to preferences emerging from factors like gender and personality.

We argue that tailoring is important and that more factors need to be considered when tailoring behavior change strategies to users. Based on the discussed literature, we expect that both factors, personality and gender, will influence how motivating the text messages representing the certain processes are rated. But because there no existing work evaluating these text messages representing the processes in relation to personality or gender, we have no specific expectations as to how personality and gender will influence the evaluation of the text messages and the text message categories.

3 Method

To investigate how motivating the messages representing the processes of change would be rated, we designed an online survey study. The survey was carried out through Amazon Mechanical TurkFootnote 1 (AMT) on SurveyMonkeyFootnote 2.

3.1 Participants

The sample consisted of 350 respondents. The study was conducted in English. All but five respondents were native English speakers; their results were not found anomalous and were left in the sample.

The minimum age was 20 and the maximum was 71. The average age was 36.53 (SD = 11.83) and the median 34. With respect to education, 106 respondents received some college education, 142 had a college degree, 35 completed a masters, 59 completed high school, 5 obtained a PhD, and 3 had other types of qualifications. To ensure consistency and a high quality of responses, the AMT requirements for the respondents were that they should have already completed > 1000 tasks on AMT, that > 98% of these tasks were approved successfully, and that the participants were located in the USA. These requirements ensured that participants were already familiar with online surveys, that they were serious about filling in the survey, and that they had some proficiency in English. In fact, 342 reported “very good” as self-assessed proficiency and 8 “good”, and none “average”, “bad” or “very bad”.

3.2 Text messages

In an earlier study (described in [48]), we gathered a large corpus of text messages (2886) by asking users from AMT to design motivational messages for different scenarios (based on the stages of change). Afterwards, we coded the messages into categories representing the processes of change (i.e., the strategies for behavior change, see Fig. 1 and Table 1) from the TTM with a codebook. This codebook was iteratively developed by two coders (coder 1 was the main investigator) individually coming up with operationalized definitions of the processes of change following the guidelines of Guest and MacQueen [16, 31]. Whom, in a similar vein, have also used this approach to develop a codebook for the processes of change from the transtheoretical model [16, p. 120] (more details are reported in [48]). The categories for coding were as follows: consciousness raising (CR), dramatic relief (DR), environmental reevaluation (ER), social liberation (SOL), self-reevaluation (SR), self-liberation (SEL), helping relationships (HR), counterconditioning (CC), reinforcement management (RM), and stimulus control (SC). For the following online survey study (the same evaluation is the basis for this paper), coder 1 chose five text messages for each of the ten categories in a way that those five messages would have the least overlap and repetition, while still being a good representation of the individual process (50 text messages in total, see Appendix). We evaluated these messages by having people rate how motivating or demotivating they found them.

3.3 Task

In the survey, people evaluated 50 motivational text messages (five messages for each of the ten strategies, see Appendix). We asked them to rate each message according to how motivating they thought the message was for them (“Please rate how motivating or demotivating you find the following messages for yourself”). All the 50 messages were presented together on one page and the order of the messages was randomized for each participant. The messages were rated on a scale from 1 (“Very demotivating”) to 5 (“Very motivating”) with 3 as the neutral position (“Neither demotivating nor motivating”).

3.4 Measures

At the start of the survey, we asked the participants about their gender, age, native language, understanding of the English language, education level, maternal education level (as an indication of socioeconomic status [15]), and main field of work. After, we presented participants with the specific task of this survey: rating the 50 text messages. At the end of the survey, we asked participants information about when they would prefer to receive messages such as these, how many of such messages they would prefer to receive weekly, and whether they had experience with smartphones and exercise apps. Moreover, we measured people’s susceptibility to persuasion [24], and their current stage of change 3 [39], perceived experiences with processes of change 3 [36], self-efficacy 3 [6], and decisional balance [35] for exerciseFootnote 3. To measure participants’ personality, we used the 50-item IPIP representation of the revised version of Costa and McCrae’s [9] NEO Personality InventoryFootnote 4 which posed 50 statements (e.g., “Make plans and stick to them.”). Participants were asked to answer how descriptive they found these statements of themselves (on a 5-point Likert scale, 1 being “very inaccurate” and 5 being “very accurate”).

The internal reliability of the personality traits and process-messages was overall very good (see Table 2 for personality and Table 3 for process-categories).

Table 2 Averages (M), standard deviations (SD), and Cronbach’s alpha’s (α) for all the scales. Personality scales are 10 averaged items with scoring from 1 to 5. (N = 350)
Full size table
Table 3 Averages (M), standard deviations (SD), and Cronbach’s alpha’s (α) for all the scales
Full size table

3.5 Procedure

The study adheres to the ethical guidelines of the University of Twente, and it was approved by the ethical committee of the Faculty of Electronics, Mathematics and Computer Science. Participants were recruited through AMT. They were informed of their compensation (3 US dollars), the goal of the study (finding out which text messages are motivating), and the estimated time to complete the survey (35 minutes). Participants could then decide to accept or decline the survey and proceed to the SurveyMonkey website were the survey was hosted. On the first page, the goal of this study was repeated and participants were asked to complete a consent form. On the second page, the participants were asked to fill in demographic information. On the third page, instructions and context to rate the messages were given. On the fourth page, they were given 50 motivational messages to rate. On the pages five through sixteen, they were given multiple questionnaires. At the seventeenth and last page, participants were debriefed and given a completion code to fill in on AMT to receive payment.

4 Results: how participants evaluated the motivational messages

In this section, we present the tests we used to analyze the data and the results of how the process-categories, gender, and personality related to the rating of the text messages. The responses of all 350 participants were analyzed.

4.1 Model selection

To test whether personality traits, gender, the text message categories, the interaction between personality traits and the message categories, and the interaction between gender and the message categories influenced the ratings of the text messages, we ran a linear mixed-effects model analysis in R [42] with the lme4 package [4] and the lmerTest package [28] to output significant differences. The contribution of the variables was assessed using the AIC and BIC scores, where a smaller number indicates a better fit. An analysis of variance comparison of the incrementally built models is reported in Table 4 (built with xtable package [12]). We followed the AIC score because we selected a fair number of parameters for the model, and the AIC does not penalize the number of parameters as strongly as BIC [7]. Our final model with the smallest AIC score included the motivational rating of the text messages as the outcome, and the categories of the text messages (processes of change), personality traits, gender, the interaction between personality traits and the categories, and the interaction between gender and the categories as fixed effects, and participants and different messages as random effects.

Table 4 An analysis of variance comparison of incrementally built models
Full size table

4.2 Results of the model

We report on a summary of the model (see Tables 5, 6, and 7) with the consciousness raising (CR) category as reference level (benchmark to which to compare the scores of the other categories relatively). To make the intercept more interpretable, we centered the personality traits by subtracting the mean score of each trait for all scores in each trait. This means that the intercept score is now the score on the reference level while all other factors are zero, meaning a gender of zero (male) and personality scores of zero (meaning sample average).

Table 5 The variance and standard deviation of the random effects of the model: participants and text messages. Number of observations: 17500, number of respondents: 350, number of text messages: 50
Full size table
Table 6 The estimates and standard error of the fixed effects of the model: process-categories, personality traits, and gender
Full size table
Table 7 The estimates and standard error of the fixed interaction effects of the model: process-categories, personality traits, and gender
Full size table

4.2.1 Main effects

For the main effects, participants who scored higher on Openness scored higher on the motivational rating for the CR category (intercept reference level). Participants who scored higher on Agreeableness scored higher on the motivational rating for CR, and females scored significantly lower on the motivational rating for CR category (Table 6).

4.2.2 Interaction effects

For the interaction effects, we see, for example, that there was no statistical evidence that participants who scored higher or lower on Openness also scored higher or lower on the motivational rating for CC messages, compared to that for CR messages (reference level), but the participants who scored higher on Openness did score significantly lower on the motivational rating for all the other message categories compared to that for CR. Also, there was no statistical evidence that participants who scored higher or lower on Conscientiousness also scored higher or lower on the motivational rating of CR messages (from Table 6), but participants who scored higher on Conscientiousness did score significantly lower on the motivational rating for DR messages compared to that for CR messages (see Table 7 for these and more significant interaction effects).

4.3 Model with different reference levels

However, we are interested in the direct relation between all the categories and the personality traits, and all the categories and gender, not in the relations in comparison to the reference level category CR. The summary of the model does report on the relationship between categories and the personality traits and gender through the interaction estimates, because the estimate for a personality trait and a selected category other than the reference category is equal to the estimate of that trait for the reference category plus the estimate for the interaction between that trait and the selected category. However, this does not incorporate the significance of the relation between a trait and a category, only the significance of the relation between a trait and a category compared to the reference level category. To illustrate this, as stated in Section 4.2.2, there was no statistical evidence that participants who scored higher or lower on Conscientiousness also scored higher or lower on the motivational rating of CR messages (Table 6), but participants who scored higher on Conscientiousness did score significantly lower on the motivational rating for DR messages compared to that for CR messages (Table 7). This could perpetrate the idea that there is a significant relation between scoring higher on Conscientiousness and scoring lower on DR messages. However, based on “post-hoc” tests (i.e., changing the reference level), participants who scored higher on Conscientiousness did not score significantly lower on the motivational rating of the DR message category (see column C, row DR in Table 8). Therefore, to see how significant the main effects of personality and gender were on the other nine categories, we ran the same model again with changed reference levels (see Table 8). Note that the first row of Table 8 reports identical estimates to estimates reported for the personality traits and gender in Table 6, where we also report the standard error.

Table 8 The estimates of the main effects of the model with category levels as reference levels and personality (O, C, E, A, and N), and gender (G)
Full size table

4.3.1 Personality main effects

We investigated the relations between the personality trait variables (O, C, E, A, and N) and the process-categories on the motivational rating of the text messages. We conducted “post-hoc” tests (i.e., changing the reference level) because it is not possible to determine from the basic lmer test summary whether significant relations corresponding to interaction effects are simply null effects or effects in the opposite direction to the main effect. In Table 8, the estimate scores are reported for the personality traits (OCEAN) and the process-categories (ten times as reference category). The p values suggest that the personality traits O, E, and A significantly relate to eight of the process-categories. There was a significant positive relation between Openness and the rating of CR and CC, and a significant negative relation between Openness and ER. There was a significant positive relation between Extraversion and the rating of SOL and SC. And there was a significant positive relation between Agreeableness and the rating of CR, ER, SR, SEL, HR, and SC. This shows that the use of these eight process-categories (CR, ER, SOL, SR, SEL, HR, CC, and SC) could be adapted to how people score on the personality traits of O, E, and A.

4.3.2 Gender main effects

We investigated differences in ratings of text message categories between males and females (see Table 8). The reference category for gender is male. CR, DR, ER, and SR messages were more motivating to male participants than female participants, because the estimate is significantly negative. This shows that the use of these four Experiential process-categories (CR, DR, ER, and SR) could be adapted to which gender people identify themselves as (male or female).

5 Discussion

In this section, we discuss the results of our analysis, considerations for behavior change system design and the limitations of our current work.

5.1 Personality

We found a number of significant relations between the ratings of the text message process-categories and personality traits, and in this section, we discuss how to interpret and use these results. Specifically, we found significant relations between process-categories CR, ER (inverse relation), and CC and the personality trait Openness; significant relations between process-categories SOL and SC and the personality trait Extraversion; and significant relations between process-categories CR, ER, SR, SEL, HR, and SC and the personality trait Agreeableness. These significant relations indicate that personality plays a role in how motivational our text messages are perceived. In other words, people scoring high on Openness found CR text messages to be more motivating, ER text messages to be less motivating, and CC text messages to be more motivating. People scoring high on Extraversion found SOL and SC text messages to be more motivating, and people scoring high on Agreeableness found CR, ER, SR, SEL, HR, and SC text messages to be more motivating. These personality trait findings indicate that, when choosing motivational strategies or interventions for a behavior change system for physical activity, it is important to take into account the personality of the user, because strategies can be perceived differently than expected, potentially leading to counterintuitive results.

Overall, Agreeableness was related to the most processes (six in total). This is somewhat similar to [3], where Alkış and Temizel found Agreeableness to be the most susceptible to their six persuasion strategies (i.e., authority, reciprocation, scarcity, liking, commitment, and consensus). On the other hand, they found Openness to Experience to be the least susceptible to these strategies, which is not reflected in our results. Their strategies are also not identical to our processes of change text message strategies. To the best of our knowledge, this is the first effort to translate the processes of change to operationalized text messages; therefore, exact comparison is not possible.

It is interesting to note that these results are different from our previous work [47], where we investigated the relation between personality and the processes of change through a questionnaire measuring the everyday experiences with the processes, in contrast to our current work where we investigate the relation between personality and the processes of change through rating text messages representing the processes on how motivating the messages are. Although the factors we incorporated are not exactly equal, making comparison difficult, in our previous work, we found mostly different relations between the traits and the processes, and Conscientiousness was related to the most processes (six in total).

An explanation could be that our text messages representing the processes are not good representations of the complex processes of change and that is why the results do not match up to the processes of change through a questionnaire. Representing the processes of change through “simple” text messages may not cover the complexity of a process. But, with our rigorous coding (as reported in [48]) following the guidelines of Guest and MacQueen [16, 31], who also used this to develop a codebook for the processes of change from the TTM [16, p. 120], and with decent reliability of the categories (see Table 3), we have tried to mitigate this.

Another explanation for the difference between our current results and our previous results could be that the processes of change questionnaire asks for the participants recent experience with this process (e.g., “I look for information related to exercise.”; 1: Never–5: Repeatedly). This alludes to how relevant the process is or could be for the participant (consciously or subconsciously). This relevance (or irrelevance) in turn could suggest the experimenter that this process could be a good way to try to stimulate the participant. On the other hand, our evaluation of the process of change messages asks the participant to rate how motivating they think this message is for their current situation (e.g., “Exercise will help clear your mind and reduce stress.”; 1: Very demotivating–5: Very motivating), which pushes them to think very consciously about how motivated they would be by this message if they received it. In that sense, it is interesting that in our previous results, Conscientiousness (often associated with following norms and rules, planning, and organizing [23]) played a greater role in terms of correlation to the experience of processes (through the questionnaire), while in this research, it is Agreeableness (often associated with being more cooperative, compliant, and trusting [23]) that plays a greater role in terms of correlation to how motivating the text messages processes are rated. The difference being that one does not have to be motivated by a process to experience it or consider it valuable to experience. You can schedule your exercise time because it is a valuable way to get yourself to exercise, but you do not have to find it motivating. Similarly, you can find messages on scheduling workouts motivating because they are valuable in getting (or reminding) yourself to exercise, while you are not (too) conscious about scheduling exercise yourself.

Our current results are obtained with the same messages, and the messages are in the same form, as how we intend to use them for our system and therefore the relations we observed will be our foundation for tailoring system behavior.

5.2 Gender

We evaluated the relation between gender and the process-categories. Of the ten categories, we found four to be significantly related with gender. All four of these significant differences indicated that male participants found the text messages of these process-categories more motivating (i.e., CR, DR, ER, and SR) than the female participants. It is interesting to note that these were all Experiential processes, meaning that these processes focus on influencing the experiences related to the physical activity behavior change. Overall, these gender findings indicate that when choosing motivational strategies or interventions for a behavior change system for physical activity, it is also important to take into account the gender of the user, because strategies can be perceived differently than expected, specifically strategies that try to motivate people by appealing or referring to experiences.

In comparison to related work, our results are not corroborated. For example, Kristan and Suffoletto [27] found that overall women responded most favorably to all their feedback messages (divided into informational, motivational, or strategy facilitating). Busch et al. [8] found males to be more motivated by comparison and competition strategies, but these do not relate to any of the Experiential strategies we found more motivating for males.

Overall, the differences found in our results suggest that it is important to take into account the gender of the person engaging in behavior change, especially in regard to the (Experiential) process-categories that usually are associated with the earlier stages of behavior change, but we should be careful in generalizing these results (also those related to the personality traits) further than our study, due to the lack of other research with similar findings. In terms of findings for our system, it means that the use of these four process-categories (CR, DR, ER, and SR) will be adapted to the gender people identify themselves with, especially for the process-categories of DR and ER, which are evaluated demotivating by females, but slightly motivating by males.

5.3 System design considerations

We interpret our results as an indication of the need for tailoring (to personality and gender), but, in the context of how long behavior change can take, also of the need for having multiple strategies to tailor to the user. From a different perspective, our results could also be interpreted as needing only a few strategies. If we take into account only how motivating people rated the strategies, we could choose (or customize, according to Kaptein [26]) a strategy for the user based solely on what scored the highest on rated motivation for the user’s features (which implies the highest “return” for a user’s behavior change). However, considering the long-term aspect and the complexity of behavior change (in contrast to a one-time act of compliance), we do not think that the most optimal solution to achieve behavior change is to focus on one (highest rated) strategy. Instead, we would suggest a combination [11] of highest-rated strategies or interventions (for the user’s features), to avoid repetition over longer period of times (in combination with plenty of variation of messages within a strategy), but also to maximize the effectiveness of the message that behavior change has benefits for that user [14]. This could still be modeled, as suggested by Kaptein [26], by making the model more complex, even without focusing on only the highest-rated strategies, but also on which combination of strategies results in the optimal outcome (the most “permanent” behavior change). However, not all (combinations of) strategies or interventions might be or are appreciated by the user, which could result in the technology not being used and the intervention not taking place. For example, in [21], they found that the implemented reminder strategy, in the context of sleep behavior, did not work for people who had a negative attitude towards their intended activity. In that sense, our measurement of how motivating these messages are perceived is input not only for what strategies to definitely use but also for what strategies to avoid for which users to increase the likelihood of this system being used over longer periods of time.

Another point that could be made about investigating all the different factors that might relate to the strategies a system designer wants to use to influence or motivate a behavior is to use an adaptive system. An adaptive system would not need any prior input on how the factors relate, and could find the most optimal model for user by testing all the strategies. However, for the purpose of our system, we feel that this does not fit our research approach in two ways. First, the use of an adaptive system with a wide variety of possible strategies and no prior input has a high risk of being abandoned by users, because on average, the users will be exposed to a number of strategies that might not be motivating or even demotivating them (one of our results of the evaluation), resulting in a higher chance of abandoning the technology. Second, starting from an adaptive system with no prior input might optimize the model in an more effective way compared to a predetermined static model, but it will not help explaining or interpreting why the model works. For example, we could find that a certain group of messages is highly effective for some user features, but if these messages do not explicitly group on a certain theme or underlying construct, it will be hard to interpret and replicate the results. Nevertheless, we feel that eventually it would be preferable for every system to be, at least partially, adaptable, to better accommodate for changes in users that might not have been modeled beforehand (e.g., relapse), especially when the adaptive system can be effectively combined with prior input from an already (moderately) effective model.

Our findings allow us to inform the development of our own behavior change system and might help inform the design of motivational strategies, interventions, or behavior change systems in general. Our main result is that we found significant relations between process-categories and personality traits and between process-categories and gender. The significant relations indicate that based on different personalities, people prefer different motivational strategies. Hence, the use of these eight process-categories (all except DR and RM) could be tailored to how people score on the personality traits of Openness, Extraversion, and Agreeableness. We also found four significant inverse relations between gender (female) and the process-categories CR, DR, ER, and SR, meaning that men found these four process-categories to be more motivating than women. This suggests that the gender of the person being motivated to change their behavior plays a significant role, specifically in regard to the process-categories that are usually associated with the earlier stages of behavior change. Taking these results into account could improve the effectiveness of our motivational strategies and motivational strategies in general.

5.4 Limitations of the current study

There were some limitations to the present study. We gathered our respondents through Amazon Mechanical Turk, which could mean a misrepresentation of the “general” population, although studies have reported that AMT generally has quite a good representation for online survey standards [32]. However, we are aware that there can be limitations for how representative the sample is. Also, because we used a cross-sectional design our results do not provide evidence for causation, only correlation. Moreover, we asked participants to rate how motivational they think the messages are for themselves in their own situation, which will hopefully relate to how they respond to these messages when they receive them in a real-life situation, but we do not know this yet. In our future in-the-wild study, we hope to confirm that these ratings do relate to the real effectiveness of the messages.

6 Conclusion

In this study, we investigated how people evaluated our motivational text messages representing the theory-based processes of change-categories in relation to personality and gender. Significant relations were found for both personality and gender, and the theory-based motivational text messages categories. Our main results are significant relations between process-categories (CR, ER, SOL, SR, SEL, HR, CC, and SC) and personality traits (Openness, Extraversion, and Agreeableness) and between process-categories (CR, DR, ER, and SR) and gender. From an empirical perspective, our findings provide a contribution with the relations found that indicate that personality as well as gender play a role in how motivational our text messages are perceived, specifically in regard to the process-categories that are usually associated with the more early stages of behavior change. From a theoretical perspective, our findings provide a contribution because they indicate that the saliency and impact of theory-based strategies could be improved by tailoring them to personal characteristics, like personality and gender. Overall, our findings suggest that, when choosing strategies to use in motivational technology or coaching systems, it is important to consider the individual differences of the users, like personality and gender, how this influences their preferences for strategies; and to design systems that can tailor to these differences.