Advertisement

Translational Behavioral Medicine

, Volume 7, Issue 2, pp 254–267 | Cite as

Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis

  • Olga Perski
  • Ann Blandford
  • Robert West
  • Susan Michie
Open Access
Systematic Reviews

Abstract

“Engagement” with digital behaviour change interventions (DBCIs) is considered important for their effectiveness. Evaluating engagement is therefore a priority; however, a shared understanding of how to usefully conceptualise engagement is lacking. This review aimed to synthesise literature on engagement to identify key conceptualisations and to develop an integrative conceptual framework involving potential direct and indirect influences on engagement and relationships between engagement and intervention effectiveness. Four electronic databases (Ovid MEDLINE, PsycINFO, ISI Web of Knowledge, ScienceDirect) were searched in November 2015. We identified 117 articles that met the inclusion criteria: studies employing experimental or non-experimental designs with adult participants explicitly or implicitly referring to engagement with DBCIs, digital games or technology. Data were synthesised using principles from critical interpretive synthesis. Engagement with DBCIs is conceptualised in terms of both experiential and behavioural aspects. A conceptual framework is proposed in which engagement with a DBCI is influenced by the DBCI itself (content and delivery), the context (the setting in which the DBCI is used and the population using it) and the behaviour that the DBCI is targeting. The context and “mechanisms of action” may moderate the influence of the DBCI on engagement. Engagement, in turn, moderates the influence of the DBCI on those mechanisms of action. In the research literature, engagement with DBCIs has been conceptualised in terms of both experience and behaviour and sits within a complex system involving the DBCI, the context of use, the mechanisms of action of the DBCI and the target behaviour.

Keywords

Engagement Digital Behaviour change interventions eHealth mHealth Conceptual framework Systematic review 

INTRODUCTION

A substantial number of Internet-connected adults use some forms of digital technology to monitor or modify their health: estimates vary between 20 and 80% [1, 2, 3]. Digital behaviour change interventions (DBCIs), defined as “…a product or service that uses computer technology to promote behaviour change” [4], can, for example, be delivered through computer programs, websites, mobile phones, smartphone applications (apps) or wearable devices. Evidence suggests that DBCIs can help people change a range of different health behaviours, including smoking [5, 6], alcohol consumption [7], weight management [8], physical activity [9] and self-management of chronic conditions [10]. Some form of “engagement” with DBCIs is assumed to be important for their effectiveness [11]. A positive association between engagement and, for example, smoking cessation, weight loss and increased fruit and vegetable intake has been observed [12, 13, 14]. To date, we have not achieved a shared understanding of how to usefully conceptualise and operationalise engagement with DBCIs. This systematic review, which follows the Cochrane Collaboration’s Handbook of Systematic Reviews of Interventions [15], examines how engagement has been construed and measured in the behavioural science, computer science and human-computer interaction (HCI) literatures and uses this to propose an integrative definition and conceptual framework of engagement with DBCIs that can be used to generate predictions and explanations of empirical observations.

The design of DBCIs requires knowledge of intervention content, delivery, interface design and computer programming, which have traditionally been informed by separate scientific disciplines, such as behavioural science, computer science and HCI. Scientific disciplines are characterised by accumulating a body of specialist knowledge and developing a specific terminology concerned with the particular object of research [16]. Due to the multifaceted structure of DBCIs, an interdisciplinary approach, where knowledge from multiple disciplines is harnessed to develop a shared viewpoint, is required to develop a useful conceptualisation of engagement in this context [17].

Engagement has traditionally been conceptualised differently across the behavioural science, computer science and HCI literatures, which might be due to the different epistemologies subscribed to, the differing research contexts and the different objectives pursued. In the computer science and HCI literatures, engagement has traditionally been conceptualised as the subjective experience of flow, a mental state characterised by focused attention and enjoyment [18]. This kind of conceptualisation might have emerged as a result of the focus on entertainment and usability of interactive technology. In the behavioural science literature, engagement has typically been conceptualised as “usage” of DBCIs, focusing on the temporal patterns (e.g. frequency, duration) and depth (e.g. use of specific intervention content) of usage [19, 20]. This kind of conceptualisation has emerged due to the observation that while many download and try DBCIs, sustained usage is typically low [21, 22, 23, 24]. Henceforth, two working definitions of engagement as used in the computer science and HCI literatures (“engagement as flow”) and the behavioural science literature (“engagement as usage”) are used to scope the space within which this review is conducted.

Although existing systematic reviews have assessed whether particular DBCI features (e.g. tailoring, reminders) are associated with higher engagement [25, 26] and whether engagement is associated with intervention effectiveness [11], it is not possible to synthesise results from these reviews or to draw any conclusions regarding the shape of the function (e.g. linear, non-linear) relating engagement with intervention outcomes due to the use of incomparable definitions of engagement [11]. In order to reduce fragmentation of research efforts, it would be useful to develop a shared understanding of how to conceptualise and operationalise engagement with DBCIs.

A conceptual framework can been defined as “a system of concepts, assumptions, and expectations, and the presumed relationships among them” [27]. Previous conceptual frameworks of engagement have proposed multiple interacting factors (e.g. social support, sensory appeal, ease of use) that influence “engagement as flow” or “engagement as usage” [28, 29, 30]; however, these frameworks are either not derived from empirical observations or draw only on literature from one of many interrelated scientific disciplines. For example, the framework proposed by O’Brien and Toms [28], notwithstanding its grounding in empirical observations, drew only on research from the technology literature and focused on “engagement as flow” without any links to behaviour change. Conversely, the framework by Ritterband and colleagues [29] focused on “engagement as usage” and was derived from behavioural science theory only. The model proposed by Short and colleagues [30] attempted to integrate both theoretical predictions and empirical findings from the behavioural science, persuasive design and technology literatures but did not do so in a systematic manner. Although the ontology of behaviour change interventions proposed by West and Michie provides a starting point for organising and representing DBCIs, engagement constitutes one of many important components and is hence not examined in detail [4]. It is therefore not possible to determine whether existing frameworks of engagement sufficiently explain real-world events, or whether important aspects are missing.

The aims of this review are threefold; the second and third build on output from the first:
  1. 1.

    To synthesise past work on engagement, addressing the following research questions:

     
  1. (a)

    How has engagement been defined in the selected literatures?

     
  2. (b)

    How has engagement been measured?

     
  3. (c)

    What factors have been found or hypothesised to influence engagement?

     
  4. (d)

    What are the proposed relationships between engagement and intervention effectiveness?

     
  1. 2.

    To develop an integrative definition of engagement with DBCIs and specify how it can be measured.

     
  2. 3.

    To develop a conceptual framework of the direct and indirect influences on engagement with DBCIs and the proposed relationships between engagement and intervention effectiveness.

     

METHODS

The Cochrane Handbook of Systematic Reviews of Interventions [15] and the Guidance for Undertaking Reviews in Health Care [31] were used to inform the development of the search strategy, identify inclusion criteria, select studies and extract the data. Principles from critical interpretive synthesis (CIS) were used to inform the data synthesis [32]. As CIS is one of the few methods available that affords the synthesis of qualitative and quantitative data, it was deemed to be the most suitable method. CIS is useful when a review seeks to identify a definition of a phenomenon, as it aims to produce a higher-order structure or conceptual framework (“synthesising argument”), which is grounded in the concepts (“synthetic constructs”) identified in the reviewed articles [32]. CIS does not propose a formal method for critically appraising the quality and methodological rigour of included studies but recognises that the critical evaluation and integration of disparate forms of evidence is essentially a product of the “authorial voice” [33]. The evidence is critiqued on the basis of the implicit assumptions underlying the methodological decisions made in the reviewed articles. Hence, the quality of the evidence is considered in the development of the synthetic constructs, with the consideration based on the authors’ judgements. Principles of CIS have previously been employed in reviews of the health literature [34, 35, 36].

Criteria for considering studies for this review

All types of study designs were included except position papers. All types of information sources were included except articles that were not peer-reviewed or not available in English. Studies with adult participants (i.e. aged 18 years or older) were included, as it was expected that different factors might influence engagement in children and adult populations due to different cognitive abilities [37]. Studies specifically targeting participants with cognitive impairment or intellectual disabilities were excluded for the same reason. DBCIs and digital interventions targeting individuals with mental health or chronic physical health conditions were included as no a priori reason suggesting that engagement should be conceptualised differently across the included topic areas could be identified. Interventions were excluded if they did not incorporate any digital component as part of the intervention itself (i.e. face-to-face delivery only) or if the technology was used solely as a tool to deliver measurement surveys. Studies involving recreational or educational digital games or multimedia software (e.g. software involving animations, sound and text) were included providing that engagement was discussed or measured. For the conceptualisation of “engagement as flow”, the games or technology did not need to be related to behaviour change. The primary outcome was definitions of engagement with DBCIs, digital games or multimedia software expressed either implicitly or explicitly. Secondary outcomes included proposed direct and indirect influences on engagement, measures of engagement and associations between engagement and intervention effectiveness expressed either implicitly or explicitly.

Search methods for the identification of studies

Electronic searches

A structured search of the following electronic databases was conducted in November 2015: Ovid MEDLINE (1946—November 2015), PsycINFO (1806—November 2015), ISI Web of Knowledge (1900—November 2015) and ScienceDirect (1900—November 2015). Search terms were piloted and refined to achieve a balance between sensitivity, i.e. retrieving a high proportion of relevant articles, and specificity, i.e. retrieving a low proportion of irrelevant articles [15]. An academic librarian was consulted for the validation of the databases and the final search terms. Terms were searched for in titles and abstracts as free text terms or as index terms (e.g. Medical Subject Headings) where appropriate (see Electronic Supplementary Material 1).

Searching for other resources

Articles from adjacent fields not immediately or obviously relevant to the research questions were identified through expertise within the review team [32]. The Association for Computing Machinery Digital Library (a repository for conference proceedings) and relevant journals (i.e. Journal of Medical Internet Research, Journal of the American Medical Informatics Association, Telemedicine and e-Health) were hand searched, and reference chaining was employed to identify additional articles of interest [15, 32].

Data collection and analysis

Selection of studies

Articles identified through the electronic and hand searches were merged using EndNote X7 [38] to ensure consistency. Duplicate records were removed. Two researchers independently screened (i) titles, (ii) abstracts and (iii) full texts of the identified articles against the pre-defined eligibility criteria [15]. Any disagreements were resolved through discussion and by consulting a third researcher if necessary. Inter-rater reliability was assessed based on two coding categories (i.e. inclusion versus exclusion) after the full text screening phase with the prevalence- and bias-adjusted kappa (PABAK) statistic, which controls for chance agreement [39]. The following cutoffs were used: 0.40–0.59 indicates fair agreement, 0.60–0.74 indicates good agreement and >0.75 indicates high agreement [15].

Data extraction and management

A pro-forma was developed by the first author to extract information about the study setting, participant characteristics, study design, data collection method and study findings [32]. The pro-forma was piloted on a sample of included articles to ensure that relevant information was captured [15]. A second researcher independently checked the pro-forma for accuracy and completeness [31]. Due to limited resources, a single reviewer completed the data extraction.

Quality appraisal

CIS suggests the prioritisation of seemingly relevant articles rather than favouring particular study methodologies [40]. Judgements about the relevance and underlying assumptions of articles were made by the first author and were incorporated into the data synthesis [32].

Data synthesis

Based on the principles from CIS, the data synthesis comprised the following steps:
  1. 1.

    Concepts identified in the full texts of included articles were labelled with codes by the first author. The research questions were used as a top-down coding frame; fragments of text explicitly or implicitly referring to definitions of engagement, measures of engagement, influences on engagement or associations between engagement and intervention effectiveness were coded.

     
  2. 2.

    A subsample of codes was selected through random sequence generation (https://www.random.org/) for validation by an independent researcher to increase rigour [41]. Disagreements were discussed until consensus was reached.

     
  3. 3.

    Synthetic constructs (i.e. concepts that explain similar themes) were developed from the codes, and relationships between synthetic constructs were specified by the first author.

     
  4. 4.

    The synthetic constructs and the proposed relationships between constructs were validated by an independent researcher. Disagreements were discussed until consensus was reached.

     
  5. 5.

    Two synthesising arguments (i.e. an integrative definition and its measurement, and a conceptual framework) were developed based on the synthetic constructs by the first author.

     
  6. 6.

    The synthesising arguments were refined through discussion between all co-authors.

     

Results

Summary of search results

The electronic database search yielded 925 published articles. After removing duplicates, 560 articles remained for screening. A PABAK score of 0.88 was achieved after the full text screening phase, indicating high inter-rater reliability [15]. Due to this reliability score, the additional 31 information sources were screened by a single reviewer. Of the 140 full texts screened, 117 met the inclusion criteria and were included in the data synthesis. Six qualitative studies, 27 reviews, 2 mixed methods studies and 82 quantitative studies were included (see Fig. 1). Characteristics of the included studies are described in Electronic Supplementary Material 2.
Fig 1

PRISMA flow diagram of the study selection process [42]

How has engagement been defined in the literature?

The following two synthetic constructs were developed: “engagement as subjective experience” and “engagement as behaviour”.

Engagement as subjective experience

Engagement has been conceptualised as the subjective experience that emerges in the momentary interaction with a system [18, 28, 43]. This kind of conceptualisation was only identified in the computer science and HCI literatures. Similarities can be found between engagement and the state of “flow”, described as a mental state characterised by focused attention, intrinsic interest and enjoyment, balance between challenge and skill, and temporal dissociation (i.e. losing track of the passage of time) [18, 44, 45, 46, 47]. Similarities can also be found between engagement and the state of “immersion” within digital gaming, characterised by cognitive absorption, the willingness to direct emotions towards an activity and feeling cutoff from reality [43, 48, 49, 50, 51]. As conceptual overlap was observed between these experiential qualities, the authors propose that they can be grouped under the following cognitive and emotional states: attention, interest and affect.

Engagement as behaviour

The majority of articles reviewed from the behavioural science literature conceptualised engagement in behavioural terms, suggesting that it is identical to the usage of a DBCI or its components. Engagement has further been described as the extent of usage over time [19, 52], sometimes referred to as the “dose” obtained by participants or “adherence” to an intervention [25, 53, 54], determined by assessing the following subdimensions: “amount” or “breadth” (i.e. the total length of each intervention contact), “duration” (i.e. the period of time over which participants are exposed to an intervention), “frequency” (i.e. how often contact is made with the intervention over a specified period of time) and “depth” (i.e. variety of content used) [20, 53]. In the computer science and HCI literatures, engagement has been conceptualised as the degree of involvement over a longer period of time [55], sometimes referred to as “stickiness” [56]. A distinction has also been made between “active” and “passive” engagement; while the former involves contributing to the intervention through posting in an online discussion forum, the latter involves reading what others have written without commenting, also known as “lurking” [57]. Engagement has also been conceptualised as a process of linked behaviours, suggesting that users move dynamically between stages of engagement, disengagement and re-engagement [28]. As conceptual overlap was observed between these definitions, the authors propose that engagement involves different levels of usage over time.

Development of an integrative definition of engagement

An integrative definition of engagement with DBCIs was developed through the merging of overlapping conceptualisations as outlined above, in addition to the integration of the two overarching synthetic constructs. The following two-part definition is therefore proposed:

“Engagement with DBCIs is (1) the extent (e.g. amount, frequency, duration, depth) of usage and (2) a subjective experience characterised by attention, interest and affect”.

Engagement is conceptualised as a multidimensional construct: the behavioural dimensions of engagement are underpinned by the user’s subjective experience of what it feels like to be engaged with a DBCI. Engagement is considered to be a dynamic process that is expected to vary both within and across individuals over time.

How has engagement been measured?

The following two synthetic constructs were developed: “subjective measures” and “objective measures”.

Subjective measures

In research settings, self-report questionnaires have frequently been used to measure engagement with digital games and DBCIs [51, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67]. Qualitative approaches, such as interviews or think aloud methodology, have been employed to gain a better understanding of the nature of users’ experiences of engagement with digital games and DBCIs [60, 68, 69].

Objective measures

Automatic tracking of use patterns, including number of logins, time spent online and the amount and type of content used during the intervention period, was the most commonly used measure of engagement in the behavioural science literature [11, 19, 20, 26, 44, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82]. Physiological measures including cardiac activity, respiratory depth [62] and electro-dermal activity [65], and psychophysical measures, such as eye tracking [51], have been used to measure engagement in the computer science and HCI literatures.

Measures relating to the integrated conceptualisation of engagement

Based on the literature synthesis, we suggest that all facets of engagement proposed in the integrative definition of engagement can in principle be measured or inferred through the following: (1) user-reported interaction with the DBCI through self-report questionnaires, interview studies or think aloud studies; (2) automated recording of DBCI use (e.g. logins, page views); and (3) recording of physiological or psychophysical correlates of DBCI interaction.

What factors have been hypothesised or found to influence engagement?

The following two synthetic constructs were developed: “context” and “DBCI”. Context was subdivided into “population” and “setting.” DBCI was subdivided into “content” and “delivery.” Relationships between constructs were specified.

Context

Population

Psychological characteristics

Motivation was found to be positively associated with engagement across many studies, with none indicating a negative association [20, 68, 83, 84, 85, 86, 87]. As the available evidence is correlational in nature, the direction of influence cannot be assumed. It has been hypothesised that the relationship between motivation and engagement might be U-shaped; those who are least and most motivated to, for example, quit smoking, are hypothesised to disengage quickly from DBCIs due to failed and successful behaviour change, respectively [19].

Expectations are thought to be influential in that users are hypothesised to engage more if there is a match between their expectations and the goal of the DBCI [49, 73, 86, 88, 89]. Prior experiences of using other websites or apps, or of having tried face-to-face counselling (which may or may not have worked), might shape users’ expectations of what DBCIs can provide [90].

Mental health, including low mood, anxiety and stress, has been found to be negatively associated with engagement [68, 73, 87, 91, 92, 93, 94, 95, 96]. A negative association with mental health was mainly observed in studies of DBCIs targeting individuals diagnosed with a mental health condition but was also observed in physical activity [68] and weight loss [94] interventions. Similarly, experience of well-being or believing that one does not need to work on certain issues has been found to be negatively associated with engagement [92].

Need for cognition, defined as the tendency to process large amounts of information [11, 30, 57, 88, 97], and self-efficacy to execute a given behaviour [83, 98, 99] were found to be positively associated with engagement.

Personal relevance, which refers to the extent to which a DBCI is perceived to apply to the individual and their particular situation, has been hypothesised to positively influence engagement [69, 78, 100, 101, 102, 103, 104]. Results from interview studies indicate that participants believe that lack of personal relevance is a sufficient reason for dropping out from intervention trials [86, 92, 95, 105].

Demographic characteristics

Age [20, 57, 63, 68, 69, 70, 73, 76, 79, 91, 95, 96, 99, 106, 107, 108, 109, 110, 111], gender [20, 69, 73, 90, 95, 100, 101, 110, 111], education [20, 69, 91, 92, 96, 99, 106, 107, 109, 110, 112], employment [91, 92, 107] and ethnicity [57, 106] were found to be significantly associated with engagement. There was a trend towards a positive association between engagement and older age, higher educational attainment and being a woman; however, as no meta-analysis was conducted, a conclusion about the size and direction of influence cannot be drawn. Computer literacy, or confidence using the Internet, has been found to be positively associated with engagement [11, 20, 98, 99, 106, 108, 113]. However, as none of the included studies adequately measured baseline computer skills in their designs, a firm conclusion cannot be drawn.

Physical characteristics

Physique, including baseline weight and the presence of comorbidities, was found to be negatively associated with engagement [68, 70, 71, 91, 92, 93, 94, 106, 112].

Setting

The social and physical environments in which a DBCI is used, have been hypothesised to influence engagement [4, 29, 30]. The social environment includes culture (e.g. prevailing norms), commercial environment, media and social cues. The physical environment includes financial resources, material resources, time pressure, physical cues, location, the healthcare system and policy. Time [86, 92, 93, 114] and access to hardware or the Internet [30, 115] have been hypothesised to be positively associated with engagement.

DBCI

Content

DBCIs that include particular behaviour change techniques (BCTs), such as action plans [78], goal setting [116], feedback [59] and self-monitoring tools [78], have been found to be associated with higher engagement [78]. Rewards and incentives have been hypothesised [26, 100, 101, 117] or found [118] to positively influence engagement; however, evidence from trials in which the presence of rewards or incentives has been manipulated is scarce.

Social support features, referring to features that facilitate the receipt of social support, were found to positively influence engagement [76, 82, 119, 120, 121, 122, 123, 124]. Features that decrease the feeling of loneliness or that increase motivation through competition with others include online discussion forums, gamification elements such as leaderboards that show users where they rank in a gamified system, and peer-to-peer contact [125, 126]. Evidence indicates that DBCIs that provide access to such features are successful in getting users who report lower social support at baseline to engage [57, 127]; however, participants who reported higher levels of social support at baseline were found to be more likely to engage with the social elements of DBCIs across a few studies [68, 86, 91, 96].

Reminders have been hypothesised [117, 128, 129] or found to positively influence engagement; results from a meta-analysis indicate a positive effect of reminders on engagement [130]. However, receiving too many reminders may have a negative effect on engagement due to “e-mail fatigue” [69].

Delivery

Mode of delivery, which includes face-to-face, telephone, text message, smartphone app, website and mass media delivery, has been hypothesised to influence engagement with DBCIs [4].

Professional support features, which include features that enable remote contact with a clinician via e-mail, telephone or text messages, have been found to positively influence engagement with DBCIs [20, 25, 26, 63, 68, 70, 73, 77, 88, 90, 95, 120, 131, 132, 133, 134]. However, results from a randomised controlled trial (RCT) of a web-based weight loss intervention in which some participants received coaching calls from a nurse indicated that participants in the coaching arm were more likely to drop out around the time of the first coaching session, suggesting a negative influence of professional support features in particular situations [70].

Control features, referring to features that make users feel that they are in control of and are free to make choices about how to interact with a DBCI, have been hypothesised [51, 119] or found [52, 74, 110] to positively influence engagement. For example, results from an RCT in which participants either received content all at once or sequentially over a period of weeks suggest that participants were more likely to disengage when the content was delivered sequentially [110]. Tunnelled interventions (i.e. those that lead users through a number of predetermined steps) have been found to generate more page views compared with self-paced ones [74]. However, this may be an artefact of making users click through a pre-specified number of pages in order to progress through the DBCI.

Novelty, generated by regular content updates, has been found to positively influence engagement through preventing boredom [25, 26]. However, there might be a trade-off between novelty and programme complexity; it has been hypothesised that participants will disengage if the intervention is perceived as too long or overly complicated [26, 68, 73, 88, 131, 135, 136]. It has been hypothesised that the presence of too many features may compromise a DBCI’s ease of use [19], referring to whether or not it feels natural for the user to operate an interactive system. Ease of use has been hypothesised to positively influence engagement [71, 100, 137].

The personalisation or tailoring of content has been hypothesised [26, 52, 68, 72, 80, 103, 106, 110, 113, 119, 120, 138] or found [19, 20, 66] to positively influence engagement. Interactivity, referring to a two-way flow of information between a DBCI and its user, has been hypothesised [28, 48, 50, 66, 78, 100, 139] or found [19] to positively influence engagement.

Message tone, which refers to the terminology and wording used to communicate health messages [92, 101], and narrative [43, 50, 65, 103, 125, 140], referring to the presence of a storyline, have been hypothesised to positively influence engagement. Furthermore, challenge [61, 100, 141], aesthetics and design [120, 139, 142, 143] and credibility features [68, 73], referring to features that inculcate a feeling of trust, familiarity [97, 139, 144], and the provision of guidance or tutorials [68, 106, 145] have been hypothesised to positively influence engagement with DBCIs.

What are the proposed relationships between engagement and the effectiveness of DBCIs?

The following four synthetic constructs were developed to explain the proposed relationships between engagement and the effectiveness of DBCIs: “mechanisms of action”, “unmeasured third variable”, “optimal dose” and “effective features”.

Mechanisms of action

Mechanisms of action proposed to mediate the effect of engagement with DBCIs on intervention effectiveness [4] include increased knowledge, motivation, affect management, cognitive restructuring, skill building [29], comprehension and practice of programme content, and increased self-efficacy [19]. A further distinction has been made between “intervention receipt”, which refers to the extent to which participants understand and can perform the skills taught, and “enactment of intervention skills”, which refers to the extent to which participants use these skills [146, 147]. It has also been hypothesised that mechanisms of action, such as accountability to a healthcare practitioner and relatedness to other individuals, might positively influence engagement with DBCIs [68, 77, 86, 96].

Unmeasured third variable

An unmeasured third variable, such as higher baseline motivation or self-efficacy, may be responsible for the observed association between increased engagement and positive DBCI outcomes. Alternatively, those who engage with DBCIs might simply be more inclined to behave healthily in general [11]. It has also been argued that the target behaviour itself might influence engagement [148]. For example, smokers who relapse might be more likely to stop engaging with the DBCI, while those who successfully manage their cravings might be more likely to continue engaging with the DBCI.

Optimal dose

Optimal dose refers to a pre-defined level of engagement at which specific DBCIs are effective. It has been hypothesised that the receipt of an optimal dose may explain the relationship between engagement and intervention effectiveness but that the optimal dose for particular DBCIs may vary depending on user characteristics [70, 113].

Effective features

The use of specific intervention features has been found to be associated with better DBCI outcomes [70]. It has been suggested that there may be a mismatch between features that participants choose to engage with frequently and effective features that are causally linked to intervention outcomes [104]. For example, although users may enjoy engaging with a particular feature (e.g. filling out a food diary), thus using it frequently, use of a less popular feature (e.g. “getting support” tools) might be more strongly associated with intervention outcomes, such as weight loss [70].

Development of a conceptual framework of engagement with DBCIs

The final aim of the review was to develop a conceptual framework specifying potential direct and indirect influences on engagement and relationships between engagement and intervention effectiveness. As the framework proposed by Ritterband and colleagues [29] and the ontology proposed by West and Michie [4] explicitly linked engagement to behaviour change, we drew on these to structure our conceptual framework, mapping the other existing frameworks onto it. Additional factors identified in the reviewed literature not otherwise specified were also mapped onto the conceptual framework.

We propose a conceptual framework in which engagement with a DBCI influences the target behaviour through specific mechanisms of action; box 4, box 1, box 3 and box 2, respectively. Content has been found to directly influence engagement with DBCIs; box a. Delivery has been hypothesised to directly influence engagement with DBCIs; box b. The context and the target behaviour are hypothesised to directly influence engagement; box 5 and box 3, respectively. Mechanisms of action are hypothesised to indirectly influence engagement; box 2. The population (e.g. demographic, physical and psychological characteristics) has been found to directly influence engagement with DBCIs; box c. The setting has been hypothesised to directly influence engagement; box d. Engagement is hypothesised to be indirectly influenced by the moderating influence of the context on the influence of the DBCI; box 4, box 5 and box 1, respectively. Figure 2 shows this schematically. Hypothesised influences are marked with stars.
Fig 2

Conceptual framework of direct and indirect influences on engagement with DBCIs. Transparent boxes indicate concepts. Concepts can be defined as abstract ideas that are derived from either direct or indirect evidence [149]. Blue boxes indicate attributes of concepts. Attributes can be defined as properties that characterise a concept [150]. Solid black arrows indicate relationships between concepts and attributes. Arrows with transparent heads indicate an influence of a concept.

DISCUSSION

An integrative conceptualisation of engagement with DBCIs has been developed; engagement is defined here as a multidimensional construct which can be measured through self-report questionnaires, verbal reports, automatic recording of DBCI use or recording of psychophysical manifestations. A conceptual framework was developed, which suggests that the context of use influences engagement with DBCIs either directly or indirectly by moderating the influence of the DBCI on engagement. Mechanisms of action might indirectly influence engagement and the target behaviour might directly influence engagement with DBCIs, suggesting the presence of a positive feedback loop. The proposed relationships between engagement and intervention effectiveness are tentative, as these have not been studied extensively.

The suggested behavioural and experiential dimensions of engagement can in principle be measured or inferred in every instance of a DBCI. The content, structure, length and design of specific DBCIs tend to vary, and hence, the relevance of the different dimensions of engagement will vary accordingly. Although the intended frequency, amount, duration and depth of use might be set to “1” in a one-off intervention, the individual parameters are still present and measureable. Thus, the proposed definition of engagement allows for direct comparison across different kinds of DBCIs by including multiple dimensions of engagement at its core. This has been lacking in previous conceptualisations. Evidence of higher engagement coupled with evidence of, for example, enjoyment of using a DBCI is hypothesised to predict greater DBCI effectiveness. If this is the case, the proposed definition of engagement should provide a means of generalising findings from particular DBCIs to other similar DBCIs. It may not be possible to evaluate the usefulness of the proposed definition prior to empirical work [151].

Although some self-report questionnaires designed to measure engagement demonstrate good validity and reliability [64, 152], these typically rely on measuring engagement after, as opposed to during, the event. However, the advent of new technologies allows self-reports of engagement to be measured in real-time rather than through paper-and-pencil questionnaires [153]. Although physiological measures have been used to measure engagement, notably in the HCI literature, associations between physiological and self-reported measures of engagement are weak [65]. The nature of these associations should be investigated further.

Previous conceptual frameworks have been based on theoretical predictions only or have been derived from the literature within one scientific domain [4, 28, 29, 30]. In contrast, our conceptual framework is derived from theoretical predictions and empirical observations within multiple, interrelated disciplines. This endeavour was facilitated by the use of principles from CIS, which allowed the combination of a diverse set of research findings. The proposed conceptual framework of engagement is a synthesis of existing ontologies, frameworks and models and incorporates factors not previously included. The novel components in our framework are as follows: “mental health”, “experience of well-being”, “familiarity”, “guidance” and “narrative”. The negative association between poor mental health and engagement might be explained by the observation that those with poor mental health (e.g. depression) typically experience decreased self-efficacy to, for example, stop smoking or lose weight [154, 155]. Experience of well-being might be negatively associated with engagement due to being related to the belief that one does not need any support. Familiarity with the design of DBCIs and guidance might positively influence engagement because familiar examples, design conventions or stepped how-to-use guides may inculcate feelings of comfort and ease of use. A narrative might draw users in, increasing their interest and enjoyment. Moreover, this review identified a trend towards a positive association between engagement and older age, higher educational attainment and being a woman, which merits further investigation. Although these demographic characteristics have been included in existing frameworks of engagement, the direction of influence has not been previously discussed. Through the use of a systematic, interdisciplinary approach, the proposed conceptual framework offers a comprehensive overview of the factors that may influence engagement with DBCIs and hence provides a starting point for reducing the observed fragmentation of research findings.

LIMITATIONS

The lack of evidence supporting the claim that setting of use (e.g. culture, social norms, physical cues, healthcare pathway) directly influences engagement with DBCIs constitutes a limitation. This might either reflect the search terms used or indicate that this has not been investigated in the literature; we cannot distinguish between these explanations. There was also a lack of evidence in support of the claim that the context of use (i.e. setting and population) may moderate the influence of the DBCI on engagement. For example, the setting of use may vary depending on the mode of delivery (e.g. computer versus mobile phone). Hence, the DBCI might indirectly influence engagement through determining the setting of use; while computers may predominantly be used at home or in a clinic, mobile phones might mainly be used on the go, which may influence the amount or depth of engagement. Future research should test this hypothesis. Another limitation is that no formal quality assessment of the included articles was conducted. However, this was in line with the chosen method, which suggests that the articles should be judged on the basis of their relevance to the research question rather than their methodological rigour. This method was selected due to the conceptual nature of the research questions. A further limitation is that the data extraction and literature synthesis were conducted by a single reviewer, potentially introducing bias. Finally, the end date for the literature search (i.e. November 2015) constitutes a limitation; with the pace of technological advances and the proliferation of digital health research, it is likely that relevant literature has since been published.

IMPLICATIONS AND AVENUES FOR FUTURE RESEARCH

The proposed integrative definition and conceptual framework of engagement with DBCIs have implications for clinical practice: the use of a shared terminology and measurement techniques will ensure more rapid advance in understanding engagement with DBCIs and developing methods to improve it. A shared conceptualisation of engagement can be used to help policymakers and commissioners to set evaluation standards for DBCIs. Moreover, the proposed conceptual framework can be used to generate testable hypotheses about how to improve engagement with DBCIs. For example, according to the conceptual framework, the presence of rewards might influence engagement with a DBCI due to increased motivation. This hypothesised link between rewards, motivation and engagement can be tested using an experimental design. Future avenues for research include the assessment of what dimensions of engagement (e.g. attention, interest, affect,  amount, duration, frequency, depth) are most strongly associated with intervention effectiveness, whether it is possible to establish benchmarks for the optimal dose of engagement across different kinds of DBCIs and whether the context of use influences engagement with DBCIs.

CONCLUSION

Engagement with DBCIs is conceptualised here in terms of both experience and behaviour. Engagement may be influenced by the DBCI itself, the context of use, mechanisms of action of the DBCI and the target behaviour.

Notes

Acknowledgements

The authors would like to thank Jacqui Smith, librarian at University College London, for helping to build the search strategy; Nicola Newhouse for validating the coding frame and commenting on an early draft; and Holly Walton for helping to screen articles. Olga Perski is a Ph.D. candidate funded by a grant from Bupa under its partnership with University College London. Susan Michie is part funded by grants from Cancer Research UK and NIHR’s School for Public Health Research. Robert West is funded by Cancer Research UK. The funders played no role in the design, conduct or analysis of the study nor in the interpretation and reporting of study findings.

Compliance with ethical standards

Statements on human rights, the welfare of animals, informed consent, and the Declaration of Helsinki are not applicable to this manuscript. IRB approval is not applicable.

Ethical responsibilities of authors

All authors have approved the final manuscript and agree with its submission to Translational Behavioural Medicine. All authors have contributed equally to the scientific work and are responsible and accountable for the results. We confirm that this manuscript has not been previously published (partly or in full) and that the manuscript is not being simultaneously submitted elsewhere. We confirm that the data have not been previously reported elsewhere and that no data have been fabricated or manipulated to support our conclusions. No data, text or theories by others are presented as if they were the authors’ own. The authors have full control of all data, which are accessible upon request.

Conflict of interest

OP, SM and AB declare that they have no conflict of interest. RW undertakes research and consultancy and receives fees for speaking from companies that develop and manufacture smoking cessation medications.

Supplementary material

13142_2016_453_MOESM1_ESM.docx (96 kb)
ESM 1 (DOCX 95.9 kb)

References

  1. 1.
    Rock Health. (2015). Digital health consumer adoption: 2015. Retrieved November 4, 2015, from http://rockhealth.com/reports/digital-health-consumer-adoption-2015/.
  2. 2.
    Fox S, Duggan M. Mobile health 2012. Pew Internet & American Life Project. 2012 Retrieved from http://www.pewinternet.org/2012/11/08/mobile-health-2012/.
  3. 3.
    Kontos, E., Blake, K. D., Chou, W.-Y. S., & Prestin, A. (2014). Predictors of eHealth usage: insights on the digital divide from the Health Information National Trends Survey 2012. Journal of Medical Internet Research, 16(7), e172. doi: 10.2196/jmir.3117.PubMedPubMedCentralCrossRefGoogle Scholar
  4. 4.
    West, R., & Michie, S. (2016). A Guide to Development and Evaluation of Digital Interventions in Healthcare. London: Silverback Publishing.Google Scholar
  5. 5.
    Civljak, M., Stead, L. F., Sheikh, A., & Car, J. (2013). Internet-based interventions for smoking cessation. Cochrane Database Syst Rev, 7, CD007078.Google Scholar
  6. 6.
    Whittaker, R., Borland, R., Bullen, C., Rb, L., Mcrobbie, H., & Rodgers, A. (2009). Mobile phone-based interventions for smoking cessation. Cochrane Database Syst Rev, 4.Google Scholar
  7. 7.
    Nair, N. K., Newton, N. C., Shakeshaft, A., Wallace, P., & Teesson, M. (2015). A systematic review of digital and computer-based alcohol intervention programs in primary care. Current Drug Abuse Reviews, 8(2), 111–118.PubMedCrossRefGoogle Scholar
  8. 8.
    Liu, F., Kong, X., Cao, J., Chen, S., Li, C., Huang, J., et al. (2015). Mobile phone intervention and weight loss among overweight and obese adults: a meta-analysis of randomized controlled trials. Am J Epidemiol, 181(5), 337–348. doi: 10.1093/aje/kwu260.PubMedPubMedCentralCrossRefGoogle Scholar
  9. 9.
    Muntaner, A., Vidal-Conti, J., & Palou, P. (2015). Increasing physical activity through mobile device interventions: a systematic review. Health Informatics Journal, 1–19. doi: 10.1177/1460458214567004.
  10. 10.
    Jones, K. R., Lekhak, N., & Kaewluang, N. (2014). Using mobile phones and short message service to deliver self-management interventions for chronic conditions: a meta-review. Worldviews on Evidence-Based Nursing/Sigma Theta Tau International, Honor Society of Nursing, 11(2), 81–88. doi: 10.1111/wvn.12030.CrossRefGoogle Scholar
  11. 11.
    Donkin, L., Christensen, H., Naismith, S. L., Neal, B., Hickie, I. B., & Glozier, N. (2011). A systematic review of the impact of adherence on the effectiveness of e-therapies. Journal of Medical Internet Research, 13(3), e52. doi: 10.2196/jmir.1772.PubMedPubMedCentralCrossRefGoogle Scholar
  12. 12.
    Cobb, N. K., Graham, A. L., Bock, B. C., Papandonatos, G., & Abrams, D. B. (2005). Initial evaluation of a real-world Internet smoking cessation system. Nicotine & Tobacco Research, 7(2), 207–216. doi: 10.1080/14622200500055319.CrossRefGoogle Scholar
  13. 13.
    Tate, D. F., Wing, R. R., & Winett, R. a. (2001). Using Internet technology to deliver a behavioral weight loss program. J Am Med Assoc, 285(9), 1172–1177. doi: 10.1001/jama.285.9.1172.CrossRefGoogle Scholar
  14. 14.
    Alexander, G. L., McClure, J. B., Calvi, J. H., Divine, G. W., Stopponi, M. A., Rolnick, S. J., et al. (2010). A randomized clinical trial evaluating online interventions to improve fruit and vegetable consumption. Am J Public Health, 100(2), 319–326. doi: 10.2105/AJPH.2008.154468.PubMedPubMedCentralCrossRefGoogle Scholar
  15. 15.
    The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. [Updated March 2011]. (J. Higgins & S. Green, Eds.) 2011 Retrieved from www.cochrane-handbook.org.Google Scholar
  16. 16.
    Krishnan A What are academic disciplines? NCRM Working Paper Series: ESRC National Centre for Research Methods (2009).Google Scholar
  17. 17.
    Krishnan A Five strategies for practising interdisciplinarity. NCRM Working Paper Series: ESRC National Centre for Research Methods. (2009). Retrieved from http://eprints.ncrm.ac.uk/782/1/strategies_for_practising_interdisciplinarity.pdf.
  18. 18.
    Csikszentmihalyi, M. (1990). Flow: the Psychology of Optimal Performance. New York: Cambridge University Press.Google Scholar
  19. 19.
    Danaher, B. G., Boles, S. M., Akers, L., Gordon, J. S., & Severson, H. H. (2006). Defining participant exposure measures in web-based health behavior change programs. Journal of Medical Internet Research, 8(3), e15. doi: 10.2196/jmir.8.3.e15.PubMedPubMedCentralCrossRefGoogle Scholar
  20. 20.
    Couper, M. P., Alexander, G. L., Zhang, N., Little, R. J. A., Maddy, N., Nowak, M. A., et al. (2010). Engagement and retention: measuring breadth and depth of participant use of an online intervention. Journal of Medical Internet Research, 12(4), e52. doi: 10.2196/jmir.1430.PubMedPubMedCentralCrossRefGoogle Scholar
  21. 21.
    Eysenbach, G. (2005). The law of attrition. Journal of Medical Internet Research, 7(1), e11. doi: 10.2196/jmir.7.1.e11.PubMedPubMedCentralCrossRefGoogle Scholar
  22. 22.
    Consumer Health Information Corporation. Motivating patients to use smartphone health apps. (2015). Retrieved August 10, 2015, from http://www.prweb.com/releases/2011/04/prweb5268884.htm.
  23. 23.
    Bennett, G. G., & Glasgow, R. E. (2009). The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health, 30, 273–292. doi: 10.1146/annurev.publhealth.031308.100235.PubMedCrossRefGoogle Scholar
  24. 24.
    Brouwer, W., Oenema, A., Raat, H., Crutzen, R., De Nooijer, J., De Vries, N. K., & Brug, J. (2010). Characteristics of visitors and revisitors to an Internet-delivered computer-tailored lifestyle intervention implemented for use by the general public. Health Educ Res, 25(4), 585–595. doi: 10.1093/her/cyp063.PubMedCrossRefGoogle Scholar
  25. 25.
    Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. W. C. (2012). Persuasive system design does matter: a systematic review of adherence to web-based interventions. Journal of Medical Internet Research, 14(6), e152. doi: 10.2196/jmir.2104.PubMedPubMedCentralCrossRefGoogle Scholar
  26. 26.
    Schubart, J. R., Stuckey, H. L., Ganeshamoorthy, A., & Sciamanna, C. N. (2011). Chronic health conditions and internet behavioral interventions: a review of factors to enhance user engagement. Computers, Informatics, Nursing, 29(2), 81–92. doi: 10.1097/NCN.0b013e3182065eed.PubMedCrossRefGoogle Scholar
  27. 27.
    Huberman, M. A., & Miles, M. B. (1994). Data management and analysis methods. In Handbook of Qualitative Research (pp. 428–443). Thousand Oaks: SAGE Publications.Google Scholar
  28. 28.
    O’Brien, H. L., & Toms, E. G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci Technol, 59(6), 938–955.CrossRefGoogle Scholar
  29. 29.
    Ritterband, L. M., Thorndike, F. P., Cox, D. J., Kovatchev, B. P., & Gonder-Frederick, L. a. (2009). A behavior change model for internet interventions. Ann Behav Med, 38, 18–27. doi: 10.1007/s12160-009-9133-4.PubMedPubMedCentralCrossRefGoogle Scholar
  30. 30.
    Short, C. E., Rebar, A. L., Plotnikoff, R. C., & Vandelanotte, C. (2015). Designing engaging online behaviour change interventions: a proposed model of user engagement. The European Health Psychologist, 17(1), 32–38.Google Scholar
  31. 31.
    Centre for Reviews and Dissemination, U. of Y. Systematic reviews: CRD’s guidance for undertaking reviews in healthcare. (K. Khan, G. Ter Riet, J. Glanville, A. Sowden, & J. Kleijnen, Eds.) (2008). Retrieved from http://eprints.whiterose.ac.uk/1139/.
  32. 32.
    Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., et al. (2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol, 6, 35. doi: 10.1186/1471-2288-6-35.PubMedPubMedCentralCrossRefGoogle Scholar
  33. 33.
    Dixon-Woods, M., Bonas, S., & Booth, A. (2006). How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res, 6(1), 27–44. doi: 10.1177/1468794106058867.CrossRefGoogle Scholar
  34. 34.
    Entwistle, V., Firnigl, D., Ryan, M., Francis, J., & Kinghorn, P. (2012). Which experiences of health care delivery matter to service users and why? A critical interpretive synthesis and conceptual map. Journal of Health Services Research & Policy, 17(2), 70–78. doi: 10.1258/jhsrp.2011.011029.CrossRefGoogle Scholar
  35. 35.
    Kazimierczak, K. A., Skea, Z. C., Dixon-Woods, M., Entwistle, V. A., Feldman-Stewart, D., N’Dow, J. M. O., & MacLennan, S. J. (2013). Provision of cancer information as a “support for navigating the knowledge landscape”: findings from a critical interpretive literature synthesis. Eur J Oncol Nurs, 17(3), 360–369. doi: 10.1016/j.ejon.2012.10.002.PubMedCrossRefGoogle Scholar
  36. 36.
    Morrison, L., Yardley, L., Powell, J., & Michie, S. (2012). What design features are used in effective e-health interventions? A review using techniques from critical interpretive synthesis. Telemedicine and e-Health, 18(2), 137–144. doi: 10.1089/tmj.2011.0062.PubMedCrossRefGoogle Scholar
  37. 37.
    Anderson, P. J. (2002). Assessment and development of executive function (EF) during childhood. Child Neuropsychology, 8(2), 71–82. doi: 10.1076/chin.8.2.71.8724.PubMedCrossRefGoogle Scholar
  38. 38.
    Thomson Reuters. EndNote X7. Philadelphia, USA 2013.Google Scholar
  39. 39.
    Byrt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. J Clin Epidemiol, 46(5), 423–429. doi: 10.1016/0895-4356(93)90018-V.PubMedCrossRefGoogle Scholar
  40. 40.
    Dixon-Woods, M., Sutton, A., Shaw, R., Miller, T., Smith, J., Young, B., et al. (2007). Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. Journal of Health Services Research & Policy, 12(1), 42–47. doi: 10.1258/135581907779497486.CrossRefGoogle Scholar
  41. 41.
    Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? Br Med J, 322, 1115–1117. doi: 10.1136/bmj.322.7294.1115.CrossRefGoogle Scholar
  42. 42.
    Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med, 6(7), e1000097. doi: 10.1371/journal.pmed.1000097.PubMedPubMedCentralCrossRefGoogle Scholar
  43. 43.
    Brown E, Cairns P. A grounded investigation of game immersion. In CHI ‘04 Extended Abstracts on Human Factors in Computing Systems. (2004) 1297–1300. ACM. doi: 10.1145/985921.986048.
  44. 44.
    Bianchi-Berthouze N, Kim WW, Patel D. Does body movement engage you more in digital game play? and why? In Proceedings of the International Conference on Affective Computing and Intelligent Interaction. 2007: 102–113.Google Scholar
  45. 45.
    Chou JC, Hung C, Hung Y. Design factors of mobile games for increasing gamers’ flow experiences. In Proceedings of the 2014 I.E. ICMIT. 2014:137–139.Google Scholar
  46. 46.
    Sharek, D., & Wiebe, E. (2014). Measuring video game engagement through the cognitive and affective dimensions. Simulation & Gaming, 45, 569–592. doi: 10.1177/1046878114554176.CrossRefGoogle Scholar
  47. 47.
    Zhou, T. (2013). Understanding the effect of flow on user adoption of mobile games. Personal & Ubiquitous Computing, 17, 741–748. doi: 10.1007/s00779-012-0613-3.CrossRefGoogle Scholar
  48. 48.
    Oh, J., & Sundar, S. S. (2015). How does interactivity persuade? An experimental test of interactivity on cognitive absorption, elaboration, and attitudes. J Commun, 65, 213–236. doi: 10.1111/jcom.12147.CrossRefGoogle Scholar
  49. 49.
    Bouvier, P., Lavoue, E., & Sehaba, K. (2014). Defining engagement and characterizing engaged-behaviors in digital gaming. Simulation & Gaming, 45(4–5), 491–507. doi: 10.1177/1046878114553571.CrossRefGoogle Scholar
  50. 50.
    Schønau-Fog, H., & Bjørner, T. (2012). “Sure, I would like to continue”: a method for mapping the experience of engagement in video games. Bull Sci Technol Soc, 32(5), 405–412. doi: 10.1177/0270467612469068.CrossRefGoogle Scholar
  51. 51.
    Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., & Walton, A. (2008). Measuring and defining the experience of immersion in games. International Journal of Human-Computer Studies, 66(9), 641–661.CrossRefGoogle Scholar
  52. 52.
    McClure, J. B., Shortreed, S. M., Bogart, A., Derry, H., Riggs, K., St John, J., et al. (2013). The effect of program design on engagement with an internet-based smoking intervention: randomized factorial trial. Journal of Medical Internet Research, 15(3), e69. doi: 10.2196/jmir.2508.PubMedPubMedCentralCrossRefGoogle Scholar
  53. 53.
    Voils, C. I., King, H. A., Maciejewski, M. L., Allen, K. D., Yancy Jr., W. S., & Shaffer, J. A. (2014). Approaches for informing optimal dose of behavioral interventions. Ann Behav Med, 48, 392–401. doi: 10.1007/s12160-014-9618-7.PubMedPubMedCentralCrossRefGoogle Scholar
  54. 54.
    Wang, J., Sereika, S. M., Chasens, E. R., Ewing, L. J., Matthews, J. T., & Burke, L. E. (2012). Effect of adherence to self-monitoring of diet and physical activity on weight loss in a technology-supported behavioral intervention. Patient Preference and Adherence, 6, 221–226.PubMedPubMedCentralCrossRefGoogle Scholar
  55. 55.
    Calleja, G. (2007). Digital game involvement. Games & Culture, 2(3), 236–260.CrossRefGoogle Scholar
  56. 56.
    Lin, J. C.-C. (2007). Online stickiness: its antecedents and effect on purchasing intention. Behav Inform Technol, 26(6), 507–516. doi: 10.1080/01449290600740843.CrossRefGoogle Scholar
  57. 57.
    Han, J. Y., Kim, J.-H., Yoon, H. J., Shim, M., McTavish, F. M., & Gustafson, D. H. (2012). Social and psychological determinants of levels of engagement with an online breast cancer support group: posters, lurkers, and non-users. J Health Commun, 17(3), 356–371. doi: 10.1080/10810730.2011.585696.Social.PubMedCrossRefGoogle Scholar
  58. 58.
    Burns, C. G., & Fairclough, S. H. (2015). Use of auditory event-related potentials to measure immersion during a computer game. Int J Hum Comput Stud, 73, 107–114. doi: 10.1016/j.ijhcs.2014.09.002.CrossRefGoogle Scholar
  59. 59.
    Chiang, Y.-T., Lin, S. S. J., Cheng, C.-Y., & Liu, E. Z.-F. (2011). Exploring online game players’ flow experiences and positive affect. The Turkish Online Journal of Educational Technology, 10(1), 106–114.Google Scholar
  60. 60.
    Chung, J., & Gardner, H. J. (2012). Temporal presence variation in immersive computer games. International Journal of Human-Computer Interaction, 28(8), 511–529. doi: 10.1080/10447318.2011.627298.CrossRefGoogle Scholar
  61. 61.
    Fang, X., Zhang, J., & Chan, S. S. (2013). Development of an instrument for studying flow in computer game play. International Journal of Human-Computer Interaction, 29(7), 456–470. doi: 10.1080/10447318.2012.715991.CrossRefGoogle Scholar
  62. 62.
    Harmat, L., Manzano, Ö. D., Theorell, T., Högman, L., Fischer, H., & Ullén, F. (2015). Physiological correlates of the flow experience during computer game playing. Int J Psychophysiol, 97, 1–7. doi: 10.1016/j.ijpsycho.2015.05.001.PubMedCrossRefGoogle Scholar
  63. 63.
    Hilvert-Bruce, Z., Rossouw, P. J., Wong, N., Sunderland, M., & Andrews, G. (2012). Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther, 50(7–8), 463–468. doi: 10.1016/j.brat.2012.04.001.PubMedCrossRefGoogle Scholar
  64. 64.
    Lefebvre, R. C., Tada, Y., Hilfiker, S. W., & Baur, C. (2010). The assessment of user engagement with eHealth content: the eHealth engagement scale. J Comput-Mediat Commun, 15, 666–681. doi: 10.1111/j.1083-6101.2009.01514.x.CrossRefGoogle Scholar
  65. 65.
    Martey, R. M., Kenski, K., Folkestad, J., Feldman, L., Gordis, E., Shaw, A., et al. (2014). Measuring game engagement: multiple methods and construct complexity. Simulation & Gaming, 45, 528–547. doi: 10.1177/1046878114553575.CrossRefGoogle Scholar
  66. 66.
    Morrison, L., Moss-Morris, R., Michie, S., & Yardley, L. (2014). Optimizing engagement with Internet-based health behaviour change interventions: comparison of self-assessment with and without tailored feedback using a mixed methods approach. Br J Health Psychol, 19, 839–855. doi: 10.1111/bjhp.12083.PubMedCrossRefGoogle Scholar
  67. 67.
    O’Brien, H. L., & Toms, E. G. (2010). The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science & Technology, 61(1), 50–69. doi: 10.1002/asi.CrossRefGoogle Scholar
  68. 68.
    Bossen, D., Buskermolen, M., Veenhof, C., de Bakker, D., & Dekker, J. (2013). Adherence to a web-based physical activity intervention for patients with knee and/or hip osteoarthritis: a mixed method study. Journal of Medical Internet Research, 15(10), e223. doi: 10.2196/jmir.2742.PubMedPubMedCentralCrossRefGoogle Scholar
  69. 69.
    Geraghty, A. W. A., Torres, L. D., Leykin, Y. A. N., & Mun, R. F. (2012). Understanding attrition from international internet health interventions: a step towards global eHealth. Health Promot Int, 28(3), 442–452. doi: 10.1093/heapro/das029.PubMedPubMedCentralCrossRefGoogle Scholar
  70. 70.
    Arden-Close, E. J., Smith, E., Bradbury, K., Morrison, L., Dennison, L., Michaelides, D., & Yardley, L. (2015). A visualization tool to analyse usage of web-based interventions: the example of positive online weight reduction (POWeR). Journal of Medical Internet Research, 2(1), e8. doi: 10.2196/humanfactors.4310.Google Scholar
  71. 71.
    Carter, M. C., Burley, V. J., Nykjaer, C., & Cade, J. E. (2013). Adherence to a smartphone application for weight loss compared to website and paper diary: pilot randomized controlled trial. Journal of Medical Internet Research, 15(4), e32. doi: 10.2196/jmir.2283.PubMedPubMedCentralCrossRefGoogle Scholar
  72. 72.
    Chen, Z., Koh, P. W., Ritter, P. L., Lorig, K., Bantum, E. O. C., & Saria, S. (2015). Dissecting an online intervention for cancer survivors: four exploratory analyses of internet engagement and its effects on health status and health behaviors. Health Educ Behav, 42(1), 32–45. doi: 10.1177/1090198114550822.PubMedCrossRefGoogle Scholar
  73. 73.
    Christensen, H., Griffiths, K. M., & Farrer, L. (2009). Adherence in internet interventions for anxiety and depression. Journal of Medical Internet Research, 11(2), e13. doi: 10.2196/jmir.1194.PubMedPubMedCentralCrossRefGoogle Scholar
  74. 74.
    Crutzen, R., Cyr, D., & de Vries, N. K. (2012). The role of user control in adherence to and knowledge gained from a website: randomized comparison between a tunneled version and a freedom-of-choice version. Journal of Medical Internet Research, 14(2), e45. doi: 10.2196/jmir.1922.PubMedPubMedCentralCrossRefGoogle Scholar
  75. 75.
    Cussler, E. C., Teixeira, P. J., Going, S. B., Houtkooper, L. B., Metcalfe, L. L., Blew, R. M., et al. (2008). Maintenance of weight loss in overweight middle-aged women through the internet. Obesity, 16(5), 1052–1060. doi: 10.1038/oby.2008.19.PubMedCrossRefGoogle Scholar
  76. 76.
    Davies, C., Corry, K., Van Itallie, A., Vandelanotte, C., Caperchione, C., & Mummery, W. K. (2012). Prospective associations between intervention components and website engagement in a publicly available physical activity website: the case of 10,000 steps Australia. Journal of Medical Internet Research, 14(1), e4. doi: 10.2196/jmir.1792.PubMedPubMedCentralCrossRefGoogle Scholar
  77. 77.
    Dennison, L., Morrison, L., Lloyd, S., Phillips, D., Stuart, B., Williams, S., et al. (2014). Does brief telephone support improve engagement with a web-based weight management intervention? Randomized controlled trial. Journal of Medical Internet Research, 16(3), e95. doi: 10.2196/jmir.3199.PubMedPubMedCentralCrossRefGoogle Scholar
  78. 78.
    Glasgow, R. E., Christiansen, S. M., Kurz, D., King, D. K., Woolley, T., Faber, A. J., et al. (2011). Engagement in a diabetes self-management website: usage patterns and generalizability of program use. Journal of Medical Internet Research, 13(1), e9. doi: 10.2196/jmir.1391.PubMedPubMedCentralCrossRefGoogle Scholar
  79. 79.
    Manwaring, J. L., Bryson, S. W., Goldschmidt, A. B., Winzelberg, A. J., Luce, K. H., Wilfley, D. E., & Taylor, C. B. (2008). Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol, 76(2), 341–346. doi: 10.1037/0022-006X.76.2.341.PubMedCrossRefGoogle Scholar
  80. 80.
    Morrison, C., & Doherty, G. (2014). Analyzing engagement in a web-based intervention platform through visualizing log-data. Journal of Medical Internet Research, 16(11), e252. doi: 10.2196/jmir.3575.PubMedPubMedCentralCrossRefGoogle Scholar
  81. 81.
    Murray, E., White, I. R., Varagunam, M., Godfrey, C., Khadjesari, Z., & McCambridge, J. (2013). Attrition revisited: adherence and retention in a web-based alcohol trial. Journal of Medical Internet Research, 15(8), e162. doi: 10.2196/jmir.2336.PubMedPubMedCentralCrossRefGoogle Scholar
  82. 82.
    Poirier, J., & Cobb, N. K. (2012). Social influence as a driver of engagement in a web-based health intervention. Journal of Medical Internet Research, 14(1), e36. doi: 10.2196/jmir.1957.PubMedPubMedCentralCrossRefGoogle Scholar
  83. 83.
    Cugelman, B., Thelwall, M., & Dawes, P. (2011). Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors. Journal of Medical Internet Research, 13(1), e17. doi: 10.2196/jmir.1367.PubMedPubMedCentralCrossRefGoogle Scholar
  84. 84.
    Henshaw, H., McCormack, A., & Ferguson, M. A. (2015). Intrinsic and extrinsic motivation is associated with computer-based auditory training uptake, engagement, and adherence for people with hearing loss. Front Psychol, 6, 1–13. doi: 10.3389/fpsyg.2015.01067.CrossRefGoogle Scholar
  85. 85.
    Hsu, C.-L., & Lu, H.-P. (2004). Why do people play on-line games? An extended TAM with social influences and flow experience. Information & Management, 41, 853–868. doi: 10.1016/j.im.2003.08.014.CrossRefGoogle Scholar
  86. 86.
    McCabe, M. P., & Price, E. (2009). Attrition from an internet-based psychological intervention for erectile dysfunction: who is likely to drop out? Journal of Sex & Marital Therapy, 35(5), 391–401. doi: 10.1080/00926230903065963.CrossRefGoogle Scholar
  87. 87.
    Postel, M. G., de Haan, H. A., ter Huurne, E. D., van der Palen, J., Becker, E. S., & de Jong, C. A. J. (2011). Attrition in web-based treatment for problem drinkers. Journal of Medical Internet Research, 13(4), e117. doi: 10.2196/jmir.1811.PubMedPubMedCentralCrossRefGoogle Scholar
  88. 88.
    Johansson, O., Michel, T., Andersson, G., & Paxling, B. (2015). Experiences of non-adherence to internet-delivered cognitive behavior therapy: a qualitative study. Internet Interventions, 2, 137–142. doi: 10.1016/j.invent.2015.02.006.CrossRefGoogle Scholar
  89. 89.
    Sainsbury, K., Mullan, B., & Sharpe, L. (2015). Dissemination of an online theory-based intervention to improve gluten-free diet adherence in coeliac disease: the relationship between acceptability, effectiveness, and attrition. International Journal of Behavioral Medicine, 22, 356–364. doi: 10.1007/s12529-014-9416-4.PubMedCrossRefGoogle Scholar
  90. 90.
    VanDeMark, N. R., Burrell, N. R., Lamendola, W. F., Hoich, C. A., Berg, N. P., & Medina, E. (2010). An exploratory study of engagement in a technology-supported substance abuse intervention. Substance Abuse Treatment, Prevention, and Policy, 5(10), 1–14.Google Scholar
  91. 91.
    Al-Asadi, A. M., Klein, B., & Meyer, D. (2014). Pretreatment attrition and formal withdrawal during treatment and their predictors: an exploratory study of the anxiety online data. Journal of Medical Internet Research, 16(6), e152. doi: 10.2196/jmir.2989.PubMedPubMedCentralCrossRefGoogle Scholar
  92. 92.
    Habibović, M., Cuijpers, P., Alings, M., van der Voort, P., Theuns, D., Bouwels, L., et al. (2014). Attrition and adherence in a WEB-based distress management program for implantable cardioverter defibrillator patients (WEBCARE): randomized controlled trial. Journal of Medical Internet Research, 16(2), e52. doi: 10.2196/jmir.2809.PubMedPubMedCentralCrossRefGoogle Scholar
  93. 93.
    Hebert, E. A., Vincent, N., Lewycky, S., & Walsh, K. (2010). Attrition and adherence in the online treatment of chronic insomnia. Behavioral Sleep Medicine, 8(3), 141–150. doi: 10.1080/15402002.2010.487457.PubMedCrossRefGoogle Scholar
  94. 94.
    Neve, M. J., Collins, C. E., & Morgan, P. J. (2010). Dropout, nonusage attrition, and pretreatment predictors of nonusage attrition in a commercial web-based weight loss program. Journal of Medical Internet Research, 12(4), e69. doi: 10.2196/jmir.1640.PubMedPubMedCentralCrossRefGoogle Scholar
  95. 95.
    Nicholas, J., Proudfoot, J., Parker, G., Gillis, I., Burckhardt, R., Manicavasagar, V., & Smith, M. (2010). The ins and outs of an online bipolar education program: a study of program attrition. Journal of Medical Internet Research, 12(5), e57. doi: 10.2196/jmir.1450.PubMedPubMedCentralCrossRefGoogle Scholar
  96. 96.
    Richardson, A., Graham, A. L., Cobb, N., Xiao, H., Mushro, A., Abrams, D., & Vallone, D. (2013). Engagement promotes abstinence in a web-based cessation intervention: cohort study. Journal of Medical Internet Research, 15(1), e14. doi: 10.2196/jmir.2277.PubMedPubMedCentralCrossRefGoogle Scholar
  97. 97.
    Oinas-Kukkonen, H., & Harjumaa, M. (2009). Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst, 24(28), 486–501.Google Scholar
  98. 98.
    Hong, J.-C., Chiu, P.-Y., Shih, H.-F., & Lin, P.-S. (2012). Computer self-efficacy, competitive anxiety and flow state: escaping from firing online game. The Turkish Online Journal of Educational Technology, 11(3), 70–76.Google Scholar
  99. 99.
    Meischke, H., Lozano, P., Zhou, C., Garrison, M. M., & Christakis, D. (2011). Engagement in “my child’s asthma”, an interactive web-based pediatric asthma management intervention. Int J Med Inform, 80(11), 765–774. doi: 10.1016/j.ijmedinf.2011.08.002.Engagement.PubMedPubMedCentralCrossRefGoogle Scholar
  100. 100.
    Boyle, E. A., Connolly, T. M., Hainey, T., & Boyle, J. M. (2012). Engagement in digital entertainment games: a systematic review. Comput Hum Behav, 28(3), 771–780. doi: 10.1016/j.chb.2011.11.020.CrossRefGoogle Scholar
  101. 101.
    Haines-Saah, R. J., Kelly, M. T., Oliffe, J. L., & Bottorff, J. L. (2015). Picture Me Smokefree: a qualitative study using social media and digital photography to engage young adults in tobacco reduction and cessation. Journal of Medical Internet Research, 17(1), e27. doi: 10.2196/jmir.4061.PubMedPubMedCentralCrossRefGoogle Scholar
  102. 102.
    Kim, Y. H., Kim, D. J., & Wachter, K. (2013). A study of mobile user engagement (MoEN): engagement motivations, perceived value, satisfaction, and continued engagement intention. Decis Support Syst, 56, 361–370. doi: 10.1016/j.dss.2013.07.002.CrossRefGoogle Scholar
  103. 103.
    Ludden, G. D., van Rompay, T. J., Kelders, S. M., & van Gemert-Pijnen, J. E. (2015). How to increase reach and adherence of web-based interventions: a design research viewpoint. Journal of Medical Internet Research, 17(7), e172. doi: 10.2196/jmir.4201.PubMedPubMedCentralCrossRefGoogle Scholar
  104. 104.
    Parks, A. C. (2014). A case for the advancement of the design and study of online positive psychological interventions. J Posit Psychol, 9(6), 502–508. doi: 10.1080/17439760.2014.936969.CrossRefGoogle Scholar
  105. 105.
    Horsch, C., Lancee, J., Beun, R. J., Neerincx, M. A., & Brinkman, W.-P. (2015). Adherence to technology-mediated insomnia treatment: a meta-analysis, interviews, and focus groups. Journal of Medical Internet Research, 17(9), e214. doi: 10.2196/jmir.4115.PubMedPubMedCentralCrossRefGoogle Scholar
  106. 106.
    Funk, K. L., Stevens, V. J., Appel, L. J., Bauck, A., Brantley, P. J., Champagne, C. M., et al. (2010). Associations of internet website use with weight change in a long-term weight loss maintenance program. Journal of Medical Internet Research, 12(3), e29. doi: 10.2196/jmir.1504.PubMedPubMedCentralCrossRefGoogle Scholar
  107. 107.
    Graham, A. L., Cha, S., Cobb, N. K., Fang, Y., Niaura, R. S., & Mushro, A. (2013). Impact of seasonality on recruitment, retention, adherence, and outcomes in a web-based smoking cessation intervention: randomized controlled trial. Journal of Medical Internet Research, 15(11), e249. doi: 10.2196/jmir.2880.PubMedPubMedCentralCrossRefGoogle Scholar
  108. 108.
    Peels, D. A., Bolman, C., Golsteijn, R. H. J., De Vries, H., Mudde, A. N., van Stralen, M. M., & Lechner, L. (2012). Differences in reach and attrition between web-based and print-delivered tailored interventions among adults over 50 years of age: clustered randomized trial. Journal of Medical Internet Research, 14(6), e179. doi: 10.2196/jmir.2229.PubMedPubMedCentralCrossRefGoogle Scholar
  109. 109.
    Steinberg, D. M., Levine, E. L., Lane, I., Askew, S., Foley, P. B., Puleo, E., & Bennett, G. G. (2014). Adherence to self-monitoring via interactive voice response technology in an eHealth intervention targeting weight gain prevention among black women: randomized controlled trial. Journal of Medical Internet Research, 16(4), e114. doi: 10.2196/jmir.2996.PubMedPubMedCentralCrossRefGoogle Scholar
  110. 110.
    Strecher, V. J., McClure, J., Alexander, G., Chakraborty, B., Nair, V., Konkel, J., et al. (2008). The role of engagement in a tailored web-based smoking cessation program: randomized controlled trial. Journal of Medical Internet Research, 10(5), e36. doi: 10.2196/jmir.1002.PubMedPubMedCentralCrossRefGoogle Scholar
  111. 111.
    Wanner, M., Martin-Diener, E., Bauer, G., Braun-Fahrländer, C., & Martin, B. W. (2010). Comparison of trial participants and open access users of a web-based physical activity intervention regarding adherence, attrition, and repeated participation. Journal of Medical Internet Research, 12(1), e3. doi: 10.2196/jmir.1361.PubMedPubMedCentralCrossRefGoogle Scholar
  112. 112.
    Jahangiry, L., Shojaeizadeh, D., Montazeri, A., & Najafi, M. (2014). Adherence and attrition in a web-based lifestyle intervention for people with metabolic syndrome. Iranian Journal of Public Health, 43(9), 1248–1258.PubMedPubMedCentralGoogle Scholar
  113. 113.
    Kuijpers, W., Groen, W. G., Aaronson, N. K., & van Harten, W. H. (2013). A systematic review of web-based interventions for patient empowerment and physical activity in chronic diseases: relevance for cancer survivors. Journal of Medical Internet Research, 15(2), e37. doi: 10.2196/jmir.2281.PubMedPubMedCentralCrossRefGoogle Scholar
  114. 114.
    Mahmassani, H. S., Chen, R. B., Huang, Y., Williams, D., & Contractor, N. (2010). Time to play? Activity engagement in multiplayer online role-playing games. Transportation Research Record: Journal of the Transportation Research Board, 2157, 129–137. doi: 10.3141/2157-16.CrossRefGoogle Scholar
  115. 115.
    Ferguson, M. A., & Henshaw, H. (2015). Computer and internet interventions to optimize listening and learning for people with hearing loss: accessibility, use, and adherence. Am J Audiol, 24, 338–343. doi: 10.1044/2015.PubMedCrossRefGoogle Scholar
  116. 116.
    Weston A, Morrison L, Yardley L, Van Kleek M, Weal M. Measurements of engagement in mobile behavioural interventions? In Digital Health. 2015:1–8.Google Scholar
  117. 117.
    Donovan, E., Mahapatra, P. D., Green, T. C., Chiauzzi, E., Mchugh, K., Hemm, A., et al. (2015). Efficacy of an online intervention to reduce alcohol-related risks among community college students. Addiction Research & Theory, 23(5), 437–447. doi: 10.3109/16066359.2015.1043625.CrossRefGoogle Scholar
  118. 118.
    Khadjesari, Z., Murray, E., Kalaitzaki, E., White, I. R., McCambridge, J., Thompson, S. G., et al. (2011). Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. Journal of Medical Internet Research, 13(1), e26. doi: 10.2196/jmir.1523.PubMedPubMedCentralCrossRefGoogle Scholar
  119. 119.
    An, L. C., Perry, C. L., Lein, E. B., Klatt, C., Farley, D. M., Bliss, R. L., et al. (2006). Strategies for increasing adherence to an online smoking cessation intervention for college students. Nicotine & Tobacco Research, 8(December), S7–S12. doi: 10.1080/14622200601039881.CrossRefGoogle Scholar
  120. 120.
    Brouwer, W., Kroeze, W., Crutzen, R., de Nooijer, J., de Vries, N. K., Brug, J., & Oenema, A. (2011). Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. Journal of Medical Internet Research, 13(1), e2. doi: 10.2196/jmir.1639.PubMedPubMedCentralCrossRefGoogle Scholar
  121. 121.
    Cairns, P., Cox, A. L., Day, M., Martin, H., & Perryman, T. (2013). Who but not where: the effect of social play on immersion in digital games. Int J Hum Comput Stud, 71, 1069–1077. doi: 10.1016/j.ijhcs.2013.08.015.CrossRefGoogle Scholar
  122. 122.
    Morris, R. R., Schueller, S. M., & Picard, R. W. (2015). Efficacy of a web-based, crowdsourced peer-to-peer cognitive reappraisal platform for depression: randomized controlled trial. Journal of Medical Internet Research, 17(3), e72. doi: 10.2196/jmir.4167.PubMedPubMedCentralCrossRefGoogle Scholar
  123. 123.
    Crutzen, R., Cyr, D., Larios, H., Ruiter, R. A. C., & De Vries, N. K. (2013). Social presence and use of internet-delivered interventions: a multi-method approach. PLoS One, 8(2), e57067. doi: 10.1371/journal.pone.0057067.PubMedPubMedCentralCrossRefGoogle Scholar
  124. 124.
    Ben-Zeev, D., Kaiser, S. M., & Krzos, I. (2014). Remote “hovering” with individuals with psychotic disorders and substance use: feasibility, engagement, and therapeutic alliance with a text-messaging mobile interventionist. Journal of Dual Diagnosis, 10(4), 197–203. doi: 10.1080/15504263.2014.962336.Remote.PubMedPubMedCentralCrossRefGoogle Scholar
  125. 125.
    Miller, A. S., Cafazzo, J. A., & Seto, E. (2014). A game plan: gamification design principles in mHealth applications for chronic disease management. Health Informatics Journal, 1–10. doi: 10.1177/1460458214537511.
  126. 126.
    Brigham, T. J. (2015). An introduction to gamification: adding game elements for engagement. Medical Reference Services Quarterly, 34(4), 471–480. doi: 10.1080/02763869.2015.1082385.PubMedCrossRefGoogle Scholar
  127. 127.
    Richardson, C. R., Buis, L. R., Janney, A. W., Goodrich, D. E., Sen, A., Hess, M. L., et al. (2010). An online community improves adherence in an internet-mediated walking program. Part 1: results of a randomized controlled trial. Journal of Medical Internet Research, 12(4), e71. doi: 10.2196/jmir.1338.PubMedPubMedCentralCrossRefGoogle Scholar
  128. 128.
    Leslie, E., Marshall, A. L., Owen, N., & Bauman, A. (2005). Engagement and retention of participants in a physical activity website. Preventive, 40, 54–59. doi: 10.1016/j.ypmed.2004.05.002.Google Scholar
  129. 129.
    Irvine, A. B., Russell, H., Manocchia, M., Mino, D. E., Cox Glassen, T., Morgan, R., et al. (2015). Mobile-web app to self-manage low back pain: randomized controlled trial. Journal of Medical Internet Research, 17(1), e1. doi: 10.2196/jmir.3130.PubMedPubMedCentralCrossRefGoogle Scholar
  130. 130.
    Lin, H., & Wu, X. (2014). Intervention strategies for improving patient adherence to follow-up in the era of mobile information technology: a systematic review and meta-analysis. PLoS One, 9(8), e104266. doi: 10.1371/journal.pone.0104266.PubMedPubMedCentralCrossRefGoogle Scholar
  131. 131.
    Kok, G., Bockting, C., Burger, H., Smit, F., & Riper, H. (2014). Mobile cognitive therapy: adherence and acceptability of an online intervention in remitted recurrently depressed patients. Internet Interventions, 1, 65–73. doi: 10.1016/j.invent.2014.05.002.CrossRefGoogle Scholar
  132. 132.
    van den Berg, M. H., Ronday, H. K., Peeters, A. J., Voogt-van der Harst, E. M., Munneke, M., Breedveld, F. C., & Vliet Vlieland, T. P. M. (2007). Engagement and satisfaction with an internet-based physical activity intervention in patients with rheumatoid arthritis. Rheumatology, 46(3), 545–552. doi: 10.1093/rheumatology/kel341.PubMedCrossRefGoogle Scholar
  133. 133.
    Stark, S., Snetselaar, L., Piraino, B., Stone, A., Kim, S., Hall, B., & Burke, L. E. (2011). PDA self-monitoring adherence rates in two dialysis dietary intervention pilot studies: BalanceWise-HD and BalanceWise-PD. J Ren Nutr, 21(6), 492–498. doi: 10.1053/j.jrn.2010.10.026.PDA.PubMedPubMedCentralCrossRefGoogle Scholar
  134. 134.
    Mohr, D. C., Duffecy, J., Ho, J., Kwasny, M., Cai, X., Burns, M. N., & Begale, M. (2013). A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One, 8(8), e70086. doi: 10.1371/journal.pone.0070086.PubMedPubMedCentralCrossRefGoogle Scholar
  135. 135.
    Klein, M., Mogles, N., & Wissen, A. V. (2014). Intelligent mobile support for therapy adherence and behavior change. J Biomed Inform, 51, 137–151. doi: 10.1016/j.jbi.2014.05.005.PubMedCrossRefGoogle Scholar
  136. 136.
    McCambridge, J., Kalaitzaki, E., White, I. R., Khadjesari, Z., Murray, E., Linke, S., et al. (2011). Impact of length or relevance of questionnaires on attrition in online trials: randomized controlled trial. Journal of Medical Internet Research, 13(4), e96. doi: 10.2196/jmir.1733.PubMedPubMedCentralCrossRefGoogle Scholar
  137. 137.
    Helander, E., Kaipainen, K., Korhonen, I., & Wansink, B. (2014). Factors related to sustained use of a free mobile app for dietary self-monitoring with photography and peer feedback: retrospective cohort study. Journal of Medical Internet Research, 16(4), e109. doi: 10.2196/jmir.3084.PubMedPubMedCentralCrossRefGoogle Scholar
  138. 138.
    Whiteside, U., Lungu, A., Richards, J., Simon, G. E., Clingan, S., Siler, J., et al. (2014). Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. Journal of Medical Internet Research, 16(2), e42. doi: 10.2196/jmir.3173.PubMedPubMedCentralCrossRefGoogle Scholar
  139. 139.
    Jennings, M. (2000). Theory and models for creating engaging and immersive e-commerce websites. In Proceedings of the 2000 ACM SIGCPR Conference on Computer Personnel Research (pp. 77–85). New York: ACM.CrossRefGoogle Scholar
  140. 140.
    Park, N., Min, K., Jin, S. A., & Kang, S. (2010). Effects of pre-game stories on feelings of presence and evaluation of computer games. Int J Hum Comput Stud, 68, 822–833. doi: 10.1016/j.ijhcs.2010.07.002.CrossRefGoogle Scholar
  141. 141.
    Hwang, M.-Y., Hong, J.-C., Hao, Y.-W., & Jong, J.-T. (2011). Elders’ usability, dependability, and flow experiences on embodied interactive video games. Educ Gerontol, 37(8), 715–731. doi: 10.1080/03601271003723636.CrossRefGoogle Scholar
  142. 142.
    Chapman P, Selvarajah S, Webster J. Engagement in multimedia training systems. In Proceedings of the 32nd Hawaii International Conference on System Sciences 1999; 0: 1–9. Washington, DC: IEEE. doi: 10.1109/HICSS.1999.772808.
  143. 143.
    Liu, S., Liao, H., & Pratt, J. A. (2009). Impact of media richness and flow on e-learning technology acceptance. Comput Educ, 52, 599–607. doi: 10.1016/j.compedu.2008.11.002.CrossRefGoogle Scholar
  144. 144.
    Miller, A. S., Cafazzo, J. A., & Seto, E. (2014). A game plan: gamification design principles in mHealth applications for chronic disease management. Health Informatics Journal. doi: 10.1177/1460458214537511.PubMedGoogle Scholar
  145. 145.
    Lieberman, D. Z. (2006). Effects of a personified guide on adherence to an online program for alcohol abusers. Cyberpsychology & Behavior, 9(5), 603–607.CrossRefGoogle Scholar
  146. 146.
    Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., et al. (2004). Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol, 23(5), 443–451. doi: 10.1037/0278-6133.23.5.443.PubMedCrossRefGoogle Scholar
  147. 147.
    Borrelli, B. (2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent, 71, S52–S63. doi: 10.1111/j.1752-7325.2011.00233.x.PubMedCentralCrossRefGoogle Scholar
  148. 148.
    Ubhi, H. K., Michie, S., Kotz, D., Wong, W. C., & West, R. (2015). A mobile app to aid smoking cessation: preliminary evaluation of SmokeFree28. Journal of Medical Internet Research, 17(1), e17. doi: 10.2196/jmir.3479.PubMedPubMedCentralCrossRefGoogle Scholar
  149. 149.
    Chinn, P. L., & Kramer, M. K. (1991). Theory and nursing: a systematic approach. St. Louis: Mosby-Year Book.Google Scholar
  150. 150.
    Fiannaca A, La Rosa M, Rizzo R, Urso A, Gaglio S. An ontology design methodology for Knowledge-Based systems with application to bioinformatics. In Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), 2012 I.E. Symposium. 2012:85–91.Google Scholar
  151. 151.
    Weber, R. (2012). Evaluating and developing theories in the information systems discipline. J Assoc Inf Syst, 13(1), 1–30.Google Scholar
  152. 152.
    O’Brien, H. L., & Toms, E. G. (2010). The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science & Technology, 61(1), 50–69. doi: 10.1002/asi.21229.CrossRefGoogle Scholar
  153. 153.
    Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavorial medicine. Ann Behav Med, 16(3), 199–202.Google Scholar
  154. 154.
    Haukkala, A., Uutela, A., Vartiainen, E., Mcalister, A., & Knekt, P. (2000). Depression and smoking cessation: the role of motivation and self-efficacy. Addict Behav, 25. doi: 10.1016/S0306-4603(98)00125-7.
  155. 155.
    Linde, J. A., Jeffery, R. W., Levy, R. L., Sherwood, N. E., Utter, J., Pronk, N. P., & Boyle, R. G. (2004). Binge eating disorder, weight control self-efficacy, and depression in overweight men and women. Int J Obes, 28(3), 418–425. doi: 10.1038/sj.ijo.0802570.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2016

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Clinical, Educational and Health PsychologyUniversity College LondonLondonUK
  2. 2.UCL Interaction CentreUniversity College LondonLondonUK
  3. 3.Cancer Research UK, Health Behaviour Research Centre, Department of Epidemiology and Public HealthUniversity College LondonLondonUK

Personalised recommendations