Weekly Self-Ratings of Treatment Involvement and Their Relation to Symptom Reduction in Internet Cognitive Behavioral Therapy for Insomnia

Treatment involvement, including involvement with written material as well as homework assignments, has previously been associated with better outcomes in cognitive behavioral therapy, but there is a lack of knowledge on which specific aspects of involvement that are the most promising for use as predictors of treatment effects. The objective of this study was to use a battery of weekly self-rated questions regarding different aspects of participant involvement during 8 weeks of either guided internet cognitive behavioral therapy for insomnia (ICBT-i, n = 73), or an active internet-delivered control treatment also including homework (ICBT-ctrl, n = 75), to predict reductions in insomnia severity after treatment. The markers of involvement were single questions on amount of time spent on treatment, amount of text read, amount of subjective knowledge gained and index scores of homework quantity (based on ratings of how many times specific techniques were used) and homework quality (based on ratings of the understanding, used as intended, and helpfulness of each specific technique). While none of the markers predicted reductions in insomnia severity for participants in the ICBT-ctrl group, text read, subjective knowledge gain, and homework quality predicted larger reductions in insomnia severity for participants in ICBT-i. Amount of subjective knowledge gained was a particularly useful marker in the ICBT-i group, since weekly ratings from the first half of treatment sufficed to explain a rather large proportion of variance in insomnia severity post treatment (14%). The results strengthen subjective knowledge gain as a candidate for use as a predictor of treatment effects in ICBT-i.

participant is able to do homework as intended and benefit from it. An example of high homework quantity and low homework quality would be a participant who every night works with sleep restriction without understanding why, not restricting time in bed enough, and not experience any benefit from it. Another way to categorize involvement in CBT is to separate receipt (if the methods are understood by the participant) from enactment (if the methods are properly rehearsed by the participant) (Lichstein et al. 1994).
CBT for insomnia (CBT-i) can be an effective treatment option also when delivered as self-help (Ho et al. 2015) or via the internet as Internet delivered CBT-i (ICBT-i) (Ye et al. 2016;Zachariae et al. 2016). Studies have found preliminary evidence for compliance to specific techniques such as sleep restriction and stimulus control to be important for reductions in insomnia severity in both a specific insomnia intervention (Kaldo. et al. 2015b) and an individually tailored intervention with an insomnia component (Kraepelien et al. 2019). In a previous study by our group, we used data during the first treatment weeks, including insomnia symptom scores, depression scores and ratings of knowledge gain and homework quantity and quality, to make individual predictions of the success (treatment response or remission) of the participants' treatment (Forsell et al. 2019a). The study showed that prediction during treatment can be successfully used in an adaptive treatment strategy, i.e. to guide individual adaptations of a treatment, such as the introduction of additional therapistsupport or telephone session with a sleep expert, in order to avoid treatment failures.
The current study is based on data from a previously published randomized controlled trial of therapist-guided ICBTi compared to an active control treatment. In the trial, ICBT-i showed superior effects on insomnia severity compared to the active control treatment (ICBT-ctrl) up to 6 months after treatment (Kaldo et al. 2015a), but differences in insomnia severity were not significant between groups after 12 and 36 months (Blom et al. 2016). This was possibly due to a higher usage of insomnia related interventions, which was reported by the participants in the control treatment during the follow-up period (Blom et al. 2016). Although some differences in the amount of text read between groups were reported in the original paper (Kaldo et al. 2015a), additional weekly self-rated data on treatment involvement, including homework quality, have not previously been reported.
The aim of the current study was to use the different markers of patient treatment involvement, based on weekly self-ratings during both ICBT-i and ICBT-ctrl, to examine which of these markers predict reductions in insomnia severity after each respective treatment. To evaluate the possible clinical usefulness of these markers as predictors, their predictive ability, both early and later in the treatment, was explored.

Method
The current study is a longitudinal and predictive analysis of a previously published randomized controlled trial comparing ICBT-i to ICBT-ctrl (Kaldo;et al. 2015a). In the current study we decided to explore each arm separately, since there were several conceptual differences between the two interventions (content, brevity, and support) which would make a direct comparison of the therapeutic processes difficult to interpret. The trial was pre-registered at Clinical-Trials.gov, registration ID: NCT01256099 and approved by the regional ethics review board in Stockholm, Sweden (2009/1810. For additional details on the trial, such as the participant flow chart, please see the main article by Kaldo et al. (2015a).

Participants
The participants were 148 adult individuals with a primary diagnosis of insomnia who were randomized on a 1:1 allocation ratio to receive either ICBT-i (n = 73) or ICBT-ctrl (n = 75). The participants were initially recruited via advertisements and articles in newspapers in Sweden, as well as via two Swedish homepages for individuals seeking internet delivered treatment. Exclusion criteria included comorbid sleep apnea or narcolepsy, working night shift, or fulfilling diagnostic criteria for major depression (Blom et al. 2015). Use of sleep medicine was unrestricted. Mean age (standard deviation; range) in the full sample (n = 148) was 47.7 years (14.8; 19-75), and 78% (116/148) were female. For a complete list of inclusion-/exclusion criteria as well as additional participant characteristics, please see the method section of the original study (Kaldo et al. 2015a).

Interventions
Both interventions were 8 weeks long, consisted of eight weekly treatment modules and used a secure internet platform for questionnaires, self-help texts and information about sleep and insomnia, control questions on how much was learned from the texts, continuous weekly sleep diary which automatically calculated sleep efficiency and other sleep measures, and homework assignments with interactive worksheets where the participant could get the individualized feedback on homework from their assigned therapist. Participants also had access to a discussion forum including the participants in the same intervention group and were monitored by a clinician for suicidal ideation. Both interventions were presented to the participant as based on CBT. ICBT-i was the high intensity insomnia treatment modality, including therapist-guidance on a weekly schedule, and all 1 3 well-established CBT-components for insomnia including the two most important components sleep restriction and stimulus control. ICBT-ctrl was an active control treatment made to be perceived as a credible insomnia treatment to avoid any nocebo effects, but all included components were scaled down to be more low intensity. The guidance was on-demand technical support only, and the presumed most effective components such as sleep restriction and stimulus control were also missing from the program. Please see Table 1 for the content of each treatment modality respectively. Another difference was that while the modules in ICBT-ctrl was presented to the participant in a fixed order, the module order in ICBT-i was tailored to the individual, with sleep restriction and stimulus control prioritized to appear early in treatment for those who supposedly would benefit from these components.

Insomnia Severity Index
The main outcome in both the parent trial and in this analysis was the Insomnia Severity Index (Bastien et al. 2001). The ISI consists of 7 items rated from 0 to 4 on different aspects of insomnia severity. The ISI is suitable as an outcome measure also when given in a web-based format since it is sensitive to change (Thorndike et al. 2011). The time points of ISI used in this analysis was screening, pretreatment and posttreatment. There was no missing data at the screening and pretreatment time points. At posttreatment 6 of 73 participants (8%) had missing ISI data in the ICBTi group and 12 of 75 participants (16%) in the ICBT-ctrl group.

Treatment Involvement Ratings
Since there were no previously validated involvement measures specifically for internet-based CBT, and for the current insomnia treatment specifically, a number of self-rated questions were constructed for this study. The self-ratings of involvement were distributed after each treatment week and the questions focused on the work done by the participant the antecedent week. When rating their homework, the participants were provided with a complete list of the techniques used in their intervention (List of techniques is provided in the Supplementary Materials). If a participant did not log in during a full treatment week, that week's treatment involvement rating was considered as missing and the participant got to rate the following week's work instead. The number of missing weekly homework ratings could therefore vary between 0 and 8 for each participant.
Each week, the participant was asked to rate their involvement with the following questions:  (1), Some (2), Much (3), Very much or all (4). The marker Text read was calculated from the total point sum of this item from all available weekly ratings. (3), Very much or all (4). The marker Subjective knowledge gain was calculated from the total point sum of this item from all available weekly ratings. 4. Mark how many of the past seven (7) (0), To a small degree as described (1), To a fairly large degree as described (2), Almost as described (3), Completely as described (4). 5c How much benefit have you had from this technique? Answers (points): None at all (0), A little benefit (1), Some benefit (2), Much benefit (3), Very much benefit (4).

How much has it affected your knowledge of and your view on sleep and sleep problems? Answers:
All three questions capture different aspects of quality: understanding the specific technique, use the technique as intended in the treatment program, and perceived helpfulness of the technique. The index score Homework quality was calculated as a weekly mean of items 5a-c from all available weekly ratings. We deem the actual performance and the result of the use of the techniques as more important than the understanding, thus the difference scoring of 5a-c.

Statistical Analysis
Statistical analyses were conducted in SPSS version 25 (IBM Corp.) Treatment involvement rating index scores from both groups were presented with means and confidence intervals. A hierarchical stepwise regression model was used once per group and per treatment involvement marker. Post-treatment ISI score was used as the dependent variable, and in step 1 of the regression, the pre-treatment ISI score was used to control for different insomnia severity starting points. In step 2 the different markers were added separately. The difference in explained variance (ΔR 2 ) from step 1 to step 2 were presented for each predictor and treatment group.

Treatment Involvement in the First and Second Half of Treatment
Since a promising way to utilize predictions of treatment effect in insomnia treatment is to do the predictions during treatment and adapt the treatment for individuals at risk of treatment failure (Forsell et al. 2019b), separate markers of involvement were also created using only data from the first four (of eight) weekly treatment involvement ratings. This represented the midpoint of treatment where there would still be four weeks left to adapt treatment for individuals risking treatment failure. Another set of markers were calculated for the weekly treatment involvement ratings from the second half of treatment (weeks 5 to 8) to explore what aspects of treatment involvement from the remaining last four treatment weeks that was associated with decreased insomnia severity. All regressions were then applied again but for the first and last four weekly treatment involvement markers separately.

Sensitivity Analysis
We also performed a sensitivity analysis where amount of completed weekly measurements for an individual participant, was added to step 1 of all regressions to be used as a control for artefacts in the relationship between markers of treatment involvement and outcome due to missing data or general adherence to the intervention. Completed weekly measures was seen as a proxy for attendance in the treatment platform. Again, the difference in explained variance (ΔR 2 ) from step 1 to step 2 were presented for each predictor and treatment group.

Additional Analysis of Homework Quality Items 5a-c Individually
To explore how the individual items in the aggregate index score representing homework quality performed as predictors of outcome on their own, an additional analysis was performed similar to the main regression analysis, but with each item 5a-c entered individually as the second step, and also a second step with all items entered simultaneously in step 2.

Insomnia Severity and Markers of Treatment Involvement
Descriptive statistics of measurements in both intervention groups are presented in Table 2 and the weekly average scores are illustrated in Fig. 1. Insomnia severity was lower in ICBT-i compared to ICBT-ctrl post treatment, mirroring the reported results of the original study (Kaldo et al. 2015a). Participants in ICBT-i rated spending on average almost 40 h on their treatment compared to around 20 h on average rated by participants in ICBT-ctrl.

Prediction of Insomnia Severity
The results of the regression analyses with all weekly measurements as predictors are presented in Table 3. For ICBTctrl pre-treatment insomnia severity was the only significant predictor across all analyses. In ICBT-i, using all weekly measurements, the amount of text read could explain 7% of the variance in insomnia severity post-treatment when controlling for pre-treatment insomnia severity, the amount of subjective knowledge gained could similarly explain 11%, and self-rated homework quality could explain 12% of the variance in outcome. When only using the markers in the ICBT-i group from week 1 to 4 (Table 4), subjective knowledge gain could explain 14%, and homework quality could explain 6% of the variance in outcome, controlled for pre-treatment insomnia severity. The amount of text read from week 1 to 4 was not a significant predictor of outcome, as it was when using all weekly measurements. On the other hand, when using the treatment involvement markers from week 5 to 8 (Table 5), Subjective knowledge gain was no longer a significant predictor of outcome, but Homework quality from the second half of treatment could explain 21% of outcome on insomnia severity in this model.

Sensitivity Analysis
The mean number of missing weekly assessments (standard deviation; 95% confidence interval) were 1.07 (1.99; 0.60-1.53) in the ICBT-i-group and 2.13 (2.75; 1.50-2.77) in the ICBT-ctrl-group. For ICBT-i, the percentage of missing ratings for a given week varied between 3 and 23% during the 8 weeks; for ICB-ctrl it varied between 11 and 41%.
Number of missing weekly measures was not a significant predictor of outcome on its own, as before controlling for pretreatment insomnia severity, with 2% explained variance of outcome (not significant) for ICBT-i and 0% explained variance (not significant) for ICBT-ctrl (data not shown). All three predictors from analyses using all weekly measurements were still significant when controlling for the amount of completed weekly assessments in the first step (Table S1 in online Supplementary Material). When using measurements from week 1 to 4 only, homework quality in the ICBTi group was no longer a significant predictor of outcome, but Subjective knowledge gain still was (Table S2 in online Supplementary Material). On the contrary, when looking at week 5 to 8, only homework quality remained a significant predictor of outcome.

Additional Analysis of Homework Quality Items 5a-c Individually
The results of the additional analysis are presented in the supplementary material (Table S4). Only item 5a (understanding techniques) and 5c (helpfulness of techniques) were significant predictors of post treatment insomnia severity when looking at each item separately for the ICBT-i-group, explaining 13% and 16% respectively of the variance in outcome. 5b (use techniques as intended) was not a significant predictor of outcome on its own.

Discussion
The purpose of this study was to explore the predictive potential of a number of self-rated markers of treatment involvement administered as weekly measurements during either internet CBT for insomnia or an active control intervention. The results suggest that in the full ICBT-i intervention, the amount of text read, subjective knowledge gained, and the self-rated quality of performed homework predicts 7-12% of post-treatment insomnia severity, also when controlling for pre-treatment insomnia severity. The amount of outcome variance these markers explained only dropped marginally to 6-10% when controlled for a proxy of general platform attendance. No markers predicted post-treatment insomnia severity in the active control intervention, possibly due to an overall lower rate of symptom change creating less variance to predict, or due to less specific therapeutic effects of the techniques included in the control interventions, or both. Also, there was an overall higher involvement in ICBT-i compared to control, driven by the factors time spent, quality of homework, and knowledge gained, and this Weekly measures ICBT-i ICBT-ctrl also created a bit more variability in the predictors which also increase the predictive power. Although time spent on treatment did not predict outcome in the separate analyses of ICBT-i and ICBT-ctrl, the large difference between treatment groups of 16.5 h is striking. Unfortunately, there was no data to construct a breakdown of what participants in the two treatment groups spent their time on. The finding that the measure of homework quality, and not homework quantity, was associated with outcome highlights the importance of assessing homework quality, even as rated by the participant him/herself. This has earlier been argued for by for example Kazantzis et al. (2017). One aspect of this is that it might be easier for participants to self-report quality more accurately than quantity: many techniques are not something you do during delimited, well-defined timeperiods, instead it might be easier to assess whether you have understood, followed instructions, and gained some help from a technique or not. On the other hand, the additional analysis of the homework quality items can be interpreted as an indication that it is not helpful with self-ratings of the degree the participant has followed instructions as intended. It would not be unlikely if participants are inclined to answer overly optimistic in regards to using the techniques as described in the intervention rendering this item weak as a predictor. Additionally, estimation quantity as number of days a technique was used might not have been optimal for all techniques.
The results suggested that self-rated amount of knowledge gained could work as a useful predictor of outcome during treatment, since these measurements from week 1 to 4 in the 8-week intervention could explain 14% of the variance in outcome, controlled for baseline insomnia severity. This was actually higher than when markers from all weeks were used and indicate that the predictive potential of subjectively acquired knowledge stems from the early part of treatment. This result highlights subjective knowledge gain as a particularly useful predictor, that could be used for example in adaptive treatment strategies where prediction usually is done around or slightly before the mid-treatment time point (Forsell et al. 2019a, b). Adaptations that facilitate treatment success, such as extra therapist-support, can then be made to the second half of treatment (Forsell et al. 2019b). In apparent contrast to this finding, an earlier study of the role of pretreatment knowledge in ICBT found that pretreatment knowledge worked as a negative predictor of outcome (Berg et al. 2019). That study was however not focused on adult insomnia but on adolescent depression, measured actual knowledge via a knowledge test, rather than the subjective perception of having gained new knowledge, and measured knowledge before treatment and not how much they have gained knowledge in the early phase of treatment. One possibility is that both high pretreatment knowledge, and low early treatment knowledge gain, is associated with other unexplored possible predictors of outcome such as previous experiences of CBT-i. It could also be argued that subjectively gained knowledge is not a direct measure of treatment adherence, but it is a very likely consequence of that and fits well into the somewhat broader concept of treatment involvement that we have chosen for this study.
Self-rated homework quality during the second half of ICBT-i showed the opposite pattern as subjective knowledge gain, by being a specifically strong predictor during the latter part of the treatment. This is probably affected by a combination of things: that important techniques such as sleep restriction and stimulus control do not take place during the first two modules, that sustained engagement in homework is beneficial, and that the later weeks more clearly separate those participants who are active, from those who lose their engagement and stop filling out weekly measures (as can be seen in Fig. 1g). This finding, together with that on subjective knowledge gain, may simply describe that early on in an intervention it is important to gain knowledge and understanding (receipt), and later also to practice skills and be persistent (enactment) (Lichstein et al. 1994).
This study has to be interpreted carefully when considering some of its limitations. The explorative and correlational analyses in this study demand caution and cannot be used as evidence for any causal hypothesis. There is a large possibility of other variables or constructs influencing both involvement measures and outcome, e.g., stressful life events, lack of time, disagreeing with the rational for the treatment, ongoing other psychological treatments or concurrent hypnotics use. Nevertheless, the results suggest that an involvement Fig. 1 a Weekly change in the treatment involvement marker Time spent in internet cognitive behavioral therapy for insomnia (ICBT-i) and control treatment (ICBT-ctrl) groups. The Y-axis represents hours spent on treatment. b Weekly change in the treatment involvement marker Text read in Internet Cognitive Behavioral Therapy for insomnia (ICBT-i) and control treatment (ICBT-ctrl) groups. c Weekly change in the treatment involvement marker Subjective knowledge gain in Internet Cognitive Behavioral Therapy for insomnia (ICBTi) and control treatment (ICBT-ctrl) groups. d Weekly change in the treatment involvement marker Homework quantity in Internet Cognitive Behavioral Therapy for insomnia (ICBT-i) and control treatment (ICBT-ctrl) groups. The Y-axis represents total numbers of times a specific homework assignment was reportedly used. e Weekly change in the treatment involvement marker Homework quantity in Internet Cognitive Behavioral Therapy for insomnia (ICBT-i) and control treatment (ICBT-ctrl) groups. The Y-axis represents total numbers of times a specific homework assignment was reportedly used divided by number of available homework assignments. f Weekly change in the treatment involvement marker Homework quality in Internet Cognitive Behavioral Therapy for insomnia (ICBT-i) and control treatment (ICBT-ctrl) groups. g Weekly change in mean amount of missing weekly measures in Internet Cognitive Behavioral Therapy for insomnia (ICBT-i) and control treatment (ICBT-ctrl) groups. The Y-axis represents the proportion of missing weekly measurements. A lower number represents a lower amount of missing weekly measures and a higher platform attendance ◂ measure such as subjective knowledge gain can be useful as a predictor of outcome in clinical settings regardless of causality. As in many trials of psychological interventions, the number of participants was rather small, which can result in a statistical power too low to detect signals that would have been of a clinically meaningful effect. For all these reasons, the results need replication and should be regarded as preliminary. It was not in the scope of this paper to examine which specific homework assignments that were most important for outcome, something that would have been an interesting additional analysis considering previous research on the importance of for example sleep restriction and stimulus control (Kaldo et al. 2015b). The version of ICBT-i used in this study focused a lot on sleep restriction and stimulus control as the preferred techniques. A strength of the analyses of this study was that we controlled for pre-treatment insomnia severity, and in a sensitivity analysis the effect of participants' amount of platform attendance, which could otherwise potentially have confounded the results. A lesson learned from this study was that the weekly measures of treatment involvement were quite cumbersome for patients to fill out. Future studies might benefit from simplifying this procedure for participants. For instance, instead of questions around a list of 29 or 8 techniques every week, one could ask for involvement ratings concerning just the latest module and homework. Also, a question that might be interesting to add is whether patients have understood and agree with the model and rationale for treatment. It would also be worth considering whether a more generic, psychometrically sound questionnaire to evaluate treatment-involvement could be developed.

Conclusion
Weekly self-ratings of treatment involvement are associated with outcome on insomnia severity in therapistguided ICBT-i, but not in an active control treatment. More specifically, measurements of the amount of text read, subjective knowledge gained and homework quality seem important for outcome. Subjective knowledge gain during the first 4 weeks, and self-rated homework quality during the last 4 weeks in treatment, were strongly associated to outcome, and especially the former predictor is thus promising to use in clinical practice, to early on identify individuals who will possibly have an unsuccessful treatment. Homework quality during the second half of treatment was associated with good outcome, and future research could examine if a strategy to specifically  focus on knowledge gain and understanding the first part of treatment and then switch to encourage homework quality during the part of treatment could be a fruitful way to boost effects.