Procedure and participants
In May 2020, during the first lockdown, 1282 adolescents (cohort 1: mean age 16.67 years, SD = 3.07, 68% girls) played the Grow It! app for 6 weeks (Fig. 1). In December 2020, during the second lockdown, another 1,871 adolescents (cohort 2: mean age 18.66 years, SD = 3.70, 81% girls) played the app for 3 weeks. To participate, adolescents needed to be able to read and write Dutch, live in the Netherlands, and own a smartphone. Non-probability convenience sampling took place via advertisements and a promotion video on (social) media and through online announcements by schoolteachers. All study information was published on our website, where it was also possible to contact the team of researchers, and sign the informed consent on a secure webpage for participants and also for parents if the age of the child was < 16 years. After the online baseline questionnaire (codebook), participants received an SMS with a unique login code to activate the app. Table 1 shows sample characteristics, Table 2 shows descriptive statistics of all study variables.
The Grow It! study was conducted in accordance with the guidelines proposed in the World Medical Association Declaration of Helsinki and has been approved by the Medical Ethical Committee of the Erasmus Medical Centre (registration number: MEC2020-0287).
Description of the Grow It! app
The Grow It! application is a multiplayer serious gaming smartphone tool (Android/iOS) aimed at promoting well-being (Fig. 2). Participants are randomly allocated to a team of four to six players. Within the app, each team collaborates in nurturing and embellishing their virtual tree, which grows each time the team reaches a predefined number of points (‘spurt’). Individual players earn points by filling out ESM questionnaires five random times per day between 9:00 and 21:00 and by completing daily challenges aimed at promoting adaptive coping including: social support, acceptance, problem-solving, and distraction. For instance, take a picture of a red car (photo challenge: aimed at distraction), ask someone what they like about you and write it down (assignment: aimed at social support), or ask someone whether he/she is having difficulties with and come up with a solution together. Write it down (assignment: aimed at problem solving)! Every day, adolescents can choose one out of three challenges. To demonstrate the successful accomplishment of a challenge, adolescents can either upload pictures, answer a free text form, or select the right answer option. Team members can communicate and motivate each other by means of positive stickers (chat function). Its privacy and security were approved by the privacy and security office of Erasmus MC, and the app complies with the Dutch General Data Protection Regulation (GDPR) and NEN-norm 7510:2017 (Dutch standard of information security management systems in healthcare).
As primary outcome, well-being was assessed at baseline and follow-up using two items . The first item refers to affective well-being: “How happy did you feel in the past week?” rated on a seven-point scale ((1) not at all to (7) very much). Previous work shows good convergent validity of this single items measurement of affective well-being . The second item assessed cognitive well-being: “How satisfied were you with your life in the past week?” and was rated on a ten-point scale ((1) not at all to (10) very much). The single item measure of cognitive well-being has been demonstrated to have good reliabilities that are comparable to multiple-item scales .
Depressive symptoms were measured at baseline and follow-up with the short 12-item Children’s Depression Inventory questionnaire (CDI ). Each item has three statements from which the adolescent could choose the one that would best describe him/her over the last week. For example: ‘I am sad sometimes’ (0), ‘I am often sad’ (1), and ‘I am sad all the time’ (2). Total scores ranged from 0 to 24, with higher scores indicating more depressive symptoms. Cronbach’s α was good in cohort 1 and cohort 2 (Baseline: 0.84, 0.84 Follow-up: 0.84, 0.83) and in previous work (α = 0.80) .
Anxiety was measured at baseline and follow-up with the nine items of the Generalized Anxiety Disorder scale from the Screen for Child Anxiety Related Disorders (SCARED child version ). Participants were asked to choose one response that would best describe their feelings in the last 2 weeks. Example items are: “In the last 2 weeks I was nervous”, “In the last 2 weeks, I was worried whether other people would like me”. The response scale runs from “not at all” (0), “a little bit or sometimes” (1) to “definitely or often” (2). High scores on the SCARED indicate more anxiety. Cronbach’s α was good in cohort 1 and cohort 2 (Baseline: 0.88, 0.85 Follow-up: 0.89, 0.86) and comparable to previous research (α = 0.90) .
Impact of COVID-19
To measure the impact of COVID-19, we used three items from the CoronaVirus Health Impact Survey Questionnaire  tapping into difficulties with cancellations of (important) events, financial problems, and optimism about the future. Items are scored on a five-point Likert scale (1 = not at all, 3 = quite, 5 = very much). Moreover, to assess the atmosphere at home since COVID-19 participants were asked to rate the atmosphere at home from 1 = uncomfortable/not pleasant at all to 4 = a little bit pleasant and 7 = very comfortable/pleasant. As an indication of governmental restrictions related to COVID-19, we calculated the COVID-19 stringency index (; 0–100).
User information of Grow It! app
Following the guidelines to measure user experience , evaluation of the app was measured at follow-up by asking participants what grade they would give to the Grow It! app, with an answer scale from 1 to 10. Also, participants were asked whether they would recommend the Grow It! app to their friends and what kind of effect the app had on their behavior and feelings. App engagement was operationalized in terms of the percentage of completed daily challenges in the Grow It! app (0–100%) and the compliance of the ESM is the percentage ESM questionnaires that were completed in the Grow It! app (0–100%) .
One item in the baseline questionnaire assessed adolescent education level (primary school, low = (preparatory school for) technical and vocational training, middle = (preparatory school for) professional education, high = (preparatory school for) university).
Before testing our hypotheses, we ran tests for cohort -differences (Appendix A1) and attrition (Appendix A2). Because participants between cohort 1 and cohort 2 differed on sex, mean age, cultural identity and COVID-19 stringency index, hypotheses were tested separately for cohort 1 and cohort 2 and sensitivity analyses were added to assess the effects of attrition (Appendix A3). To assess our first hypothesis (1), we conducted paired samples t tests to compare the affective and cognitive well-being from baseline to follow-up. To determine (2) how many adolescents increased in well-being, we followed a method by Grice and colleagues on persons and effect sizes . We considered an individual change of 0.2 SD (small effects size according to Cohen’s D) as a meaningful increase or decrease [32,29,30,31,32]. To understand (3) how background characteristics and (4) user experience and app engagement (challenges and compliance) would be associated with changes in affective and cognitive well-being after playing the Grow It! app, we had preregistered ANOVA and correlations with change rates. Here, we improved our analytical plan and conducted repeated measure models for each predictor (between * within effect). To correct for multiple testing, for each research question separately, the false discovery rate (FDR) was applied . All statistical calculations were performed using IBM SPSS Statistics (version 25).