In conclusion, the short answer to all four research (sub-)questions in this paper was yes, and we will discuss the answers to the research questions in relation to the evaluation framework of Fig. 1. First, the mobile health quiz provides added value (usefulness, fun, positive triggers) with low barriers (ease of use, limited time burdens). Second, according to the participants the mobile health quiz is well integrated in the overall service mix. Hence, the ‘ICT effectiveness’ factor of Fig. 1 appears to be sufficiently addressed. The mobile health quiz also increases the ‘coaching effectiveness’ factor of Fig. 1 by providing regular triggers for health awareness, coping strategies and useful health behaviours. In answer to the third research sub-question, health readiness (awareness, motivation, plans and actions) and competence (health perceptions, everyday choices, coping and goal achievement, growth and development) are improved. Fourth, the various health behaviours are improved as measured with the Dutch BRAVO survey (on physical activity, smoking, alcohol, food and energy/recuperation).
The answers to the third and fourth research sub-questions demonstrate the contribution to the ‘health effectiveness’ factor of Fig. 1, both of the overall hybrid service mix and of the mobile micro-learning health quiz within that mix. Besides explicit participant feedback, we have the indications from the course completion rates of the mobile health quiz, which were 66 %: well above the Eysenbach [7] ‘law’. Given the fact that these courses were not mandatory for these participants, but only additional support for health self-management, and given the fact that each course easily takes 20 min to complete (in a context of time scarcity, [3]) we regard course completion rates as a sensible proxy for perceived usefulness.
Health behaviours not only improved at 1 and 3 months, but also 10 months after the start. The latter finding is relatively special, in the sense that most healthy lifestyle interventions, whether at work sites [29] or not [17], tend to generate only short term results (3 or 6 months), after which people generally fall back into their old patterns.
These long term effects might have been promoted by a positive peer support effect from other group members [15] or management (see ‘Care Provider’ case in Table 5), even though this was not mentioned as a strong factor by most participants: see last item of Table 4.
Theory
Our contributions to theory are threefold. Through this empirical test of our proof of concept with employer organizations we see a tentative confirmation of the three design research propositions on which we have built our design research. First, a mobile micro-learning health quiz appears useful for fulfilling the key design requirements from Fig. 1 when designing ICT-supported healthy lifestyle interventions: health-, coaching- and ICT effectiveness.
Second, as contributions on these design requirements increased, we indeed empirically observed improved health readiness, behaviours and competence. Thus, our second proposition is tentatively confirmed that the design requirements from Fig. 1 may contribute to ICT-enabled health intervention success.
Third, long term health behaviours appear to benefit from a grounding in health competences (health perceptions, everyday choices, coping and goal achievement, growth, health identity and self-evaluation), see Fig. 2. This is largely a qualitative observation, based on participant feedback and sensitized by the work of Kahneman [12] and Seligman [18] on daily routines, decisions, growth, and the way happiness and life satisfaction are grounded in identity and self-evaluation, see also the theory section. Long term health impacts seem to lie not only in health readiness (awareness, intentions, plans, actions – which have a relatively operational focus and are aimed at next week rather than next year, see the HAPA and i-change models from theory). Rather, longer term health competences can be trained and developed: from health perceptions, via everyday choices, strategies for coping and goal achievement, growth, to health identity and self-norms. This is also what several study participants indicated: their health views had changes, as well as their preferences, choices, health goals, self-management and -evaluation. They indicated this is what helped them deal with the changing dynamics of life and health in the longer run. We have planned additional research in order to more rigorously measure the direct contributions of eCoaching to health competence development.
Limitations and practical implications
Regarding practical implications, the relevant design question is what made these results come about and how can we improve even further? After discussing study limitations we conduct a design evaluation, using the framework from our theory section.
This study has several limitations. First, it is largely qualitative, evaluating effects across three case organizations. In the survey, users mostly agree on issues of micro-learning perceived usefulness and perceived ease of use: both antecedents of the TAM model [4]. Statistical techniques like explorative regression analysis were not possible due to small sample size and low variance. Second, our survey results are likely subject to self-selection biases: our survey respondents were self-selected consisting of largely of users who completed most or all of the micro-learning courses. They are the ones most likely to be biased positively. Third, the context is different in each case organization and in our study design we cannot control for confounding factors. On the other hand, the strong evaluation- and effect agreement between participants across organizations does hint at the robustness and cross-case validity of the findings.
Table 6 summarizes results from our study and discusses methods for improvement for our design. Regarding factor 1 of Fig. 1, health effectiveness, more objective improvement measures could be advantageous. Especially for self-management of people with health issues (e.g., diabetes-2, irregular high blood pressure or heart arrhythmia, or impaired renal function) 24 × 7 monitoring of effects of lifestyle improvement can be very beneficial. For the second factor, coaching effectiveness, there is a challenge of automatically integrating context- and health information. For example, if we notice that someone has been sitting most of the day, when will reminders/triggers be appreciated to get up and move about, and when not? For example, if I’m very busy with finishing a report or having urgent meetings, it can be a conscious and preferred strategy to continue a sedentary work activity for the time being. Reminders and triggers can also become a nuisance. But at other times, when falling in my coach potato trap in the evening, I may very well appreciate more persistent triggers to go play sports with a buddy. Finally, ICT value adding (factor 3) could be improved via at least two routes. First, improving automated logging of health (behaviour) data and integrating this into coach processes. Second, designing more intelligent, interactive coach processes, which incorporate user preferences and plans, contextual/situational priorities and health data consequences.
Table 6 Design evaluation (authors’ opinions, 5-point scale from - - to ++)
In summary, given the relatively static content of the micro-learning health quiz, it served its health support goals well, thanks to the other service mix elements and the overall service concept. eCoach improvement opportunities for the future abound, of which we identified several.