Advertisement

Cognitive Therapy and Research

, Volume 42, Issue 2, pp 184–192 | Cite as

Reports of Therapy Skill Use and Their Efficacy in Daily Life in the Short-Term Treatment of Depression

  • Ariana C. Hoet
  • Chris J. Burgin
  • Kari M. Eddington
  • Paul J. Silvia
Original Article

Abstract

Previous studies have shown that the use of therapy skills in between sessions is an important mechanism of symptom improvement. The current study expands this line of research by using a diary approach to examine the use of therapy skills in daily life. A sample of 39 depressed adults (85% female, mean age 38) were signaled twice per week throughout the course of either cognitive-behavioral therapy (CBT) or self-system therapy (SST). Results showed that, on days when participants reported greater use of therapy skills, they reported better mood and functioning in almost all domains. Additionally, participants in CBT reported greater use of cognitive skills while participants in SST reported greater use of self-regulatory skills. This study demonstrates that repeated assessments of daily events and experiences, which prevent retrospective reporting biases, further confirm the importance of therapy skill use as a mechanism of action in psychotherapy.

Introduction

Encouraging clients’ use of therapy skills and engagement with homework assignments between therapy sessions is an integral component of cognitive behavior therapy (CBT) and other manualized treatments (Detweiler and Whisman 1999; Gaynor et al. 2006; Kazantzis et al. 2000; Rees et al. 2005). Many theorize that in order to succeed in therapy, clients must practice the skills they learn in the therapy session (Coon and Thompson 2003; Detweiler and Whisman 1999). According to Beck et al. (1985), homework allows for clients to experiment and learn from their thoughts, behaviors, and feelings in real-life situations. In CBT, these everyday personal experiences are crucial to changing belief structures as they lead to increased reinforcing events that challenge negative thought patterns (Addis and Jacobson 2000; Detweiler and Whisman 1999). Kazantzis and Deane (1999), reported that 98% of therapists surveyed incorporated homework into their treatment. Homework assignments are often used by therapists to encourage the use of therapy skills in between therapy sessions, although it should be noted that homework compliance and skill use are not the same thing. For example, a client can be “compliant” with homework by filling out worksheets in a cursory manner the night before a session without ever using the skills from therapy in daily life.

The completion of homework assigned in therapy also increases the clients’ ability to generalize therapy skills throughout their lives and increases their sense of self-efficacy (Burns and Spangler 2000; Gaynor et al. 2006; Detweiler and Whisman 1999). Additionally, homework compliance demonstrates client openness, engagement in therapy, and ownership of their improvement (Addis and Jacobson 2000; Beck et al. 1985; Detweiler and Whisman 1999). Unfortunately, the impact and possible benefits of homework compliance in therapeutic outcomes are often not prioritized in research studies (Burns and Spangler 2000; Detweiler and Whisman 1999; Rees et al. 2005).

A meta-analysis of 27 studies indicated that therapy that utilized homework lead to better treatment outcomes than therapy without homework assignments (Kazantzis et al. 2000). Further, homework compliance has been found to be a significant predictor of early and post-treatment outcomes, such as decreased depression and anxiety scores (Addis and Jacobson 2000; Coons and; Thompson 2003; Kazantzis et al. 2000; Rees et al. 2005). Specifically, engaging in behavioral tasks was found to lower depressive symptoms and increase quality of life, and completion of CBT thought records significantly predicted reduced anxiety scores (Rees et al. 2005). A study using structural equation modeling demonstrated the causal impact of homework on changes in depression, controlling for demographics, therapist identities, therapeutic empathy, motivation, and other variables influencing outcome over and above homework compliance (Burns and Spangler 2000; Kazantzis et al. 2001).

Unfortunately, methods for measuring homework completion vary widely (Gaynor et al. 2006; Kazantzis et al. 2004), and as noted previously, homework compliance is not necessarily synonymous with therapy skill use. The majority of research studies have relied on therapist or client reports of homework completion after therapy was completed, instead of session by session measurement (Detweiler and Whisman 1999; Kazantzis et al. 2004). For example, Burns and Spangler’s (2000) study on causality relied on retrospective therapist and patient report of compliance at 12 weeks. A small handful of studies have measured homework compliance more frequently, for example through the use of therapist rating at each session (Addis and Jacobson 2000; Coon and Thompson 2003). Studies involving daily records from clients may provide an even more accurate assessment of skill use. For example one study used diary card reports in dialectical behavior therapy (e.g., Lindenboim et al. 2007) and another used a measure in which participants provided daily records of homework completion (a “yes/no” client report of skill use; Rees et al. 2005). While these more frequent assessments are an improvement over measures that may be more prone to retrospective biases, paper-and-pencil approaches are typically not time- and date-stamped, so the possibility of retrospective reporting cannot be ruled out.

Methodological Considerations

Previous studies seem to clearly suggest that homework completion is important in therapy, and there is an assumption that clients who are more adherent with homework are using the skills they are learning in therapy to help them cope with problems in daily life. However, we know very little about what clients actually do—what skills they use—in their daily lives in order to cope with their difficulties. Most of the studies examining homework completion have not been designed to look at specific skill use in daily life. Kazantzis et al. (2000), reported that four studies discussed relaxation, three exposure, two assertiveness skills, one biofeedback, and one video homework. Only a few studies, to our knowledge, described the type of homework assignments in their methods. For example, some studies focused on thought records, pleasurable activities, and behavioral activation (Busch et al. 2010; Coon and Thompson 2003; Rees et al. 2005). However, none of these studies were designed to measure skill use in participants’ daily life.

Experience sampling and diary methods, in which data are collected frequently over the course of therapy in the client’s natural environment, are well-suited for examining skill use in daily life. When data are collected using tablets, phones, or computers, responses are time- and date-stamped, allowing the researcher to verify the timing of responses (Bolger and Laurenceau 2013; Conner et al. 2009; Hektner et al. 2007). Surprisingly, despite their obvious advantages, experience sampling and diary methods have rarely been used to look at skill use among clients undergoing psychotherapy. The primary goal of the current study was to use quasi-daily diary methods to look at the impact of the use of skills learned in therapy on daily functioning and mood using data from a small randomized controlled trial comparing two forms of psychotherapy for major depression: CBT and self-system therapy (SST). Although these two treatments are similar in some ways, for example both are short-term psychotherapies focused on developing skills through in-session discovery and between session homework, the underlying theories are quite different. Whereas CBT focuses on altering distorted beliefs, SST focuses on self-regulatory skills, for example by teaching people how to more effectively set and pursue goals and evaluate goal progress. Both CBT and SST have been shown to be effective treatments for depression (Eddington et al. 2015; Strauman et al. 2006). We chose to monitor skill use rather than homework compliance because, while a specific homework assignment in a given week may not necessarily involve daily work, both therapies emphasize building a set of skills (cognitive or self-regulatory) over the course of therapy and integrating those skills into daily life.

Participants in this study were instructed to complete a phone-based survey two times per week throughout the course of short-term therapy using an interactive voice response (IVR) system. Each survey included questions about the use of therapy skills that day as well as ratings of mood, activities, and other aspects of daily functioning. In addition, each survey included an open-ended question in which participants were asked to describe the skills they had actually used that day. We opted for an open-ended format in order to obtain participants’ own descriptions of what they did, rather than relying on checklists, and coded responses into five categories (cognitive skills, self-regulatory skills, relaxation, other, or no skill use).

In this study, we hypothesized that, on the days participants report higher levels of skill use, they would report higher emotional well-being and better overall functioning. Additionally, we hypothesized that participants in the CBT condition would report using more cognitive skills, whereas participants in the SST condition would report the use of more self-regulation skills. Therefore, given that previous studies have focused on the overall benefit of therapy skill use and homework completion on therapeutic outcome, our study aims to expand on existing literature by demonstrating the impact of specific daily skill use on well-being.

Methods

Participants

For a detailed description of recruitment procedures, inclusion criteria, treatment condition assignment, and participant flow, see Eddington et al. (2015). Of the 56 adults with primary major depressive disorder or dysthymia who did not meet study exclusionary criteria (which included current use of antidepressant medications) and were assigned to either SST (n = 27) or CBT (n = 29), six failed to return for any therapy sessions and four did not respond to any of the diary surveys. The final sample, therefore, included 46 participants randomized to SST (n = 22) or CBT (n = 24). Furthermore, to be included in current analyses, participants had to have completed greater than six intermittent calls, have more than four therapy sessions completed, and have <50% of calls relative to number of sessions completed. Thirty-nine people met these criteria and thus were included in the intermittent call analysis (22 in CBT and 17 in SST). Sample descriptives are shown in Table 1. Of note, two participants of the thirty-nine did not provide any verbal responses to the open-ended question, and were therefore not included in the skills coding analyses (final sample of 21 in CBT and 16 in SST).

Table 1

Descriptive information of participants

Age

M = 38.10, SD = 12.81

Sex

6 Male, 33 Female

Highest level of education

 High school or below

17.9%

 Some college or college degree

61.5%

 Some graduate school or advanced degree

20.5%

Employment status

67% employed

Marital status

 Married

30.8%

 Divorced

15.4%

 Separated

5.1%

 Never married

48.7%

Diagnostic status

 Primary MDD

94.9%

 Primary dysthymia

5.1%

 Secondary anxiety disordera

75%

Treatment history

 Past therapy

87.2%

 Past medications

35.9%

 Past hospitalization

23.1%

BDI-II score at baseline (M, SD)

M = 34.71, SD = 9.15

BAI score at baseline (M, SD)

M = 23.05, SD = 15.29

aThis category also includes PTSD

Procedures

Participant eligibility was determined during an initial laboratory assessment during which participants completed the Beck depression inventory-II (BDI-II) and Beck anxiety inventory (BAI). If the BDI-II score was above a 14, participants underwent semi-structured diagnostic interviews; participants were eligible for enrollment if they met DSM-IV-TR diagnostic criteria for major depressive disorder or dysthymia (see Eddington et al. 2015, for enrollment details) and did not have any exclusionary conditions (no current antidepressant medications, no history of mania, no current substance dependence, no history of psychotic symptoms, no diagnosis of antisocial or borderline personality disorders, and no active suicidal intent). Once enrolled in the study, participants began the intermittent diary surveys during the week of their first therapy session. A phone-based survey system (Telesage 2009) called each participant on 2 random days per week for the entire course of therapy with a reminder to complete the diary survey any time on that day. Responses were all time- and date-stamped in order to confirm completion with the sampling procedure.

Details about the two therapy conditions (including therapist training and adherence outcomes) can be found in Eddington et al. (2015). Briefly, both treatments involved 16 sessions of therapy, and both treatments were structured and focused on teaching skills for coping with current life situations through in-session work as well as homework assignments. In CBT, the focus is on cognitive restructuring skills and modifying core beliefs; in SST, the focus is on skills supporting effective self-regulation. Note that, due to scheduling issues, therapy sessions did not always occur weekly (although one session per week was the aim). Therefore, the number of weeks of data collection often exceeded 16.

Measures

Beck Depression Inventory-II

The BDI-II (Beck et al. 1996) is a widely used self-report measure of depressive symptom severity during the past 2 weeks. The scale contains 21 items and each is rated on a 4-point scale (0–3), with total scores ranging from 0 to 63. The BDI-II has excellent psychometric properties (Dozois and Covin 2004), and the internal consistency in this study was good (Cronbach’s alpha = 0.87).

Beck Anxiety Inventory

The BAI (Beck et al. 1988) is a widely-used self-report measure of anxiety symptom severity during the past 2 weeks. The scale consists of 21 items and each is rated on a 4-point scale (0–3) scale, with total scores ranging from 0 to 63. The BAI has good psychometric properties (Fydrich et al. 1992); in this study, internal consistency was very good.

Diary Methodology—Items

See Table 2 for the full list of the diary survey items. Each item was rated by the participant using a phone number pad on a scale from one (“not at all”) to seven (“very much”) with a midpoint of four (“somewhat”).

Table 2

Intermittent diary items

Mood (items randomized within this group)

 I felt happy today

I felt proud today

 I felt sad today

I felt discouraged today

 I felt enthusiastic today

I felt anxious today

 I felt satisfied today

I felt like a failure today

 I felt relieved today

I felt confident today

 I felt irritable today

I felt in control of my emotions today

 I felt guilty today

 

Physical functioning (items randomized within this group)

 I felt tired today

I didn’t feel well today

 I had a lot of energy today

I had a lot of body aches and pains today

 My sleep was restful last night

I slept more today than I should have

Social functioning (items randomized within this group)

 I enjoyed being around other people today

 I was concerned about what other people thought of me today

 How much did you interact with friends or family today by phone, text, or online?

 How much did you interact with friends or family today in person?

Activities/stressors

 I followed through with my obligations today

I made good progress toward my goals today

 I avoided things that I needed to do today

I had to push myself to get anything done today

Cognitive (items randomized within this group)

 I had trouble concentrating today

I compared myself to other people today

 I didn’t worry about things today

I was able to keep my mind focused today

 I thought a lot about my accomplishments today

I had negative or upsetting thoughts that just wouldn’t go away today

Therapy-related

 Today I tried using some skills I’ve learned in therapy

 How helpful were those skills?

 I believe that my depression symptoms will improve

Positive affect aggregate is the mean of the responses to the following items: I felt happy today, I felt enthusiastic today, I felt in control of my emotions today, I felt satisfied today, I felt relieved today, I felt proud today, I felt confident today. Negative affect aggregate is the mean of the responses to the following items: I felt sad today, I felt guilty today, I felt anxious today, I felt like a failure today, I felt irritable today, I felt discouraged today. Negative somatic symptoms is the mean of the responses to the following items: I felt tired today, I didn’t feel well today, I had a lot of energy today (reverse coded), I had a lot of body aches and pains today, my sleep was restful last night (reverse coded), I slept more today than I should have

Skill Use Coding

During the phone survey, participants were asked to verbally respond to the question, “What skills did you use today?” In addition, they were asked to rate how helpful the skills were for them, using the same one to seven scale described above. Audio recorded answers were coded for the types of skills used; coders were unaware of the assigned treatment condition for the participants.1 Responses were coded (1 = skill category used or 0 = skill category not used) by a trained coder, AH, based on five possible categories: cognitive, self-regulation, relaxation/mindfulness, other, or did not use any skills. A written description of each category was developed and was used as a reference throughout the coding process. The cognitive category included mentions of rethinking, beliefs, or thinking positively. For example, “I wrote down my thought process” or “I came up with alternative/realistic thoughts.” Self-regulation included doing things for pleasure, focusing on goals, making plans, or staying busy, such as, “I accomplished even small things on my list,” “I focused on promotion goals,” or “I created an action plan.” The relaxation/mindfulness category included mentions of relaxing, deep breathing, progressive muscle relaxation, or living in the present moment. These included statements such as, “I relaxed after work,” or “I practiced deep breathing.” Skills that did not fit into these categories were coded as “other.” Occasionally, participants reported using multiple skills in a single response that fell into different categories (15 calls out of 634, 2.4%). In those cases, all relevant categories were coded for that response; the maximum number of categories used in a single response was two.

In order to assess the reliability of the coding, a random selection of 20% of the first 220 audio files was coded by a second coder (KE). There was only one minor disagreement concerning a response that involved two categories; the coders agreed on one of the categories but differed on the other, and a consensus was quickly reached. Because the reliability was excellent (98%), the remainder were coded by a single coder.

Data Analytic Approach

Ratings made in daily life (within-person level) are nested within participants (between-person level) therefore, multilevel modeling is an appropriate statistical technique for analyzing this type of data (Affleck et al. 1999). Multilevel modeling is an extension of the more commonly used multiple regression analyses (Hox 2002), and is standard practice for the analysis of repeated, longitudinal data. Our current analyses focused on examining the relationship between participants’ responses to the question “Today I tried using some skills I’ve learned in therapy” with various other measures. As such, the coefficients produced by these analyses are comparable to unstandardized regression coefficients. For the current study, all analyses were conducted using Mplus 7 (Muthén and Muthén, 1998–2010). In addition, within-person predictors were centered at each person’s own mean (Luke 2004) and we calculated parameter estimates using maximum likelihood estimation with robust standard errors. Finally, the analyses were run as a random effects model.2

Aggregate Construction

Given participant time constraints, constructs measured in daily life are typically assessed using only one or two questions (Fisher and To 2012). However, past research has shown that the mean of several daily life items can be calculated to form an aggregate score to be used as a more robust assessment of the construct (e.g., Burgin et al. 2013). Consistent with past research, the current study calculated three aggregate mean scores to better measure daily life experiences. The first, positive affect aggregate was computed by calculating the mean of responses to the following items: I felt happy today, I felt enthusiastic today, I felt in control of my emotions today, I felt satisfied today, I felt relieved today, I felt proud today, I felt confident today. The second, negative affect aggregate, was calculated by taking the mean of responses to the following items: I felt sad today, I felt guilty today, I felt anxious today, I felt like a failure today, I felt irritable today, I felt discouraged today. Finally, negative somatic symptoms was computed by taking the mean of the responses to the following items: I felt tired today, I didn’t feel well today, I had a lot of energy today (reverse coded), I had a lot of body aches and pains today, my sleep was restful last night (reverse coded), I slept more today than I should have.3

Results

Daily Life Results

Results from Table 3 demonstrate that responses to the daily life question “Today I tried using some skills I’ve learned in therapy (hitherto referred to as the skills question) related positively with a positive affect aggregate that was created by calculating the mean of the responses to seven other daily life questions that tapped into positive affect. Conversely, responses to this question were negatively associated with a negative affect aggregate that consists of six other daily life questions which tapped into negative affect. With regard to physical functioning, responses to the skills question were negatively associated with a six item aggregate measuring negative somatic symptoms.

Table 3

Relationship of responses to the question “Today I tried using some skills I’ve learned in therapy” with other daily life criterion

Level 1 (daily life) criterion

Level 1 (daily life) predictor

“Today I tried using some skills I’ve learned in therapy”

Estimates

Standard error

Affect in the moment

 Positive affect aggregate

0.120**

0.043

 Negative affect aggregate

−0.086*

0.035

Physical functioning

 Negative somatic symptoms aggregate

−0.064*

0.032

Social functioning

 I enjoyed being around other people today

0.143***

0.040

 How much did you interact with friends or family today in person?

0.096**

0.035

 How much did you interact with friends or family today by phone, text, or online?

0.117***

0.033

 I was concerned about what other people thought of me today

−0.015

0.034

Activities/stressors

 I followed through with my obligations today (going to classes, work appointments, etc)

0.104**

0.032

 I made good progress toward my goals today

0.110***

0.053

 I had to really push myself to get anything done today

−0.114*

0.045

 I avoided things that I needed to do today

−0.090**

0.034

 The best thing I did today was (open ended response)

  (Follow up) how pleasant or enjoyable was it?

0.105*

0.052

 The worst thing I did today was (open ended response)

  (Follow up) how unpleasant was it?

−0.027

0.047

 The most stressful thing that happened to me today was

  How stressful was it?

0.022

0.052

Cognitive

 I had trouble concentrating today

−0.097*

0.045

 I compared myself to other people today

0.019

0.033

 I thought a lot about my accomplishments today

0.107*

0.047

 I had negative or upsetting thoughts that just wouldn’t go away today

−0.103*

0.051

 I didn’t worry about things today

0.024

0.039

 I was able to keep my mind focused today

0.072

0.038

Therapy-related

 What skills did you use? (open ended response)

  (Follow up) how helpful were those skills?

0.621***

0.055

  (Follow up) I believe that my depression symptoms will improve

0.160***

0.034

All items rated from one (not at all) to seven (very much). N = 39

*p < .05; **p < .01; ***p < .001

Next, we examined social functioning (see results in Table 3). Scores on the skills question positively predicted enjoyment of being around others, interacting with friends and family in person and by other means (e.g., phone, text, online), but was unrelated to how concerned participants were about how others thought of them that day. Responses to the skills question were also positively related to questions pertaining to positive activities. Specifically, responses to the skills question positively predicted follow through with daily obligations and feelings of making progress towards goals. Conversely, it was negatively associated with feelings of having to push to get anything done and avoiding things that needed to be done. After giving an open-ended response to the question “The best thing I did today was”, participants then rated how pleasant or enjoyable the activity was. The skills question scores positively predicted responses to this question. However, responses to the skills question did not predict responses to follow up question “How unpleasant was it” which was a follow up question to the open-ended question “The worst thing I did today was”. Furthermore, responses to the skills question also did not significantly predict responses to the question, “How stressful was it” which was a follow up question to the open-ended question “The most stressful thing that happened to me today was”.

Next, we examined the relationship between responses to the skills question with questions that measured cognitive functioning in daily life (see Table 3). Results demonstrate that skills question scores were negatively associated with having trouble concentrating in daily life and having negative or upsetting thoughts that wouldn’t go away. Conversely, responses to the skills question positively predicted thinking about daily life accomplishments. However, scores to the skills question were unrelated to participants stating that they compared themselves to other people, did not worry about things, and keeping the mind focused. Finally, responses to the skills question positively predicted therapy-related questions. Specifically, regarding skills learned in therapy, participants’ skills question scores positively predicted feeling that the skills learned during therapy were helpful and the belief that their depression symptoms would improve.

Open-Ended Reports of Skill Use

For the intermittent phone question regarding the report of skills used that day, 634 audio answers were obtained from 37 participants. Audio answers were coded into the cognitive (n = 149), self-regulation (n = 117), relaxation/mindfulness (n = 45), and other (n = 159) skills categories. The other category included skills such as “being with family,” “removed myself from the situation when I was angry,” or “help others.” Additionally, the other category included the mention of “doing the homework assigned during therapy” without providing further detail. Of the 634 audio calls recorded, 180 (28%) described not using skills due to only having one or two therapy sessions, lack of motivation, or not needing skills that day.

Additionally, an independent-samples t test was conducted to compare the categories of skills used by participants in the SST and CBT conditions. Consistent with our predictions, results indicated participants in the CBT condition reported using more skills coded into the cognitive category and participants in the SST condition reported using more skills coded into the self-regulation category (see Table 4). No significant differences between conditions were found for the relaxation/mindfulness, other, and did not use any skills. Across conditions, participants on average reported that they found the use of therapy skills to be moderately helpful (M = 3.92; SD = 1.26).

Table 4

Mean comparisons and descriptives for participants in CBT and SST on categories of skills used

 

CBT (n = 21)

SST (n = 16)

t(35)

p

M

SD

M

SD

Cognitive

0.368

0.249

0.078

0.099

4.393

<0.001

Self-regulation

0.109

0.183

0.324

0.270

−2.891

0.007

Relaxation/mindfulness

0.061

0.108

0.067

0.141

−0.149

0.882

Other

0.211

0.245

0.290

0.263

−0.950

0.349

Did not use any skills

0.276

0.226

0.275

0.239

0.013

0.990

Means represent the proportion of coded audio responses in which each category of response occurred. Because more than one category could be used for a single response, means do not add up to 1.00

Discussions

The use of therapy skills and completion of between-session assignments has been demonstrated to be a key component of positive therapeutic outcomes (Addis and Jacobson 2000; Coon and Thompson 2003; Kazantzis et al. 2000; Kazantzis and Lampropoulos 2002; Rees et al. 2005). Unfortunately, methods of measuring homework compliance have often relied on therapist or client reports after therapy was completed (Burns and Spangler 2000; Detweiler and Whisman 1999; Kazantzis et al. 2004). While some studies have used more frequent assessments of skill use such as diary cards (e.g., Lindenboim et al. 2007), there is typically no assurance that these written self-reports are actually completed on the day and time recorded by the respondent. This study is the first, to our knowledge, to use a phone-based diary method, which provided time- and date-stamped responses, to assess the relation between the use of skills learned in therapy and daily functioning and mood.

Results indicated that, on days that participants reported greater use of skills learned in therapy, they also reported higher positive affect, lower negative affect, better cognitive functioning, and more effective and satisfying activities (e.g., more enjoyment interacting with others, less avoidance of obligations, and greater progress on personal goals). These results underscore findings from previous studies that highlight the benefit of between-session homework completion during therapy. However, while many previous studies have assessed skill use by monitoring or inquiring about homework completion, our diary approach did not ask about homework per se. Rather, participants were asked about skills learned in therapy that they had used on the day of each survey; those skills may or may not have been associated with a homework assignment for that week. Additionally, this study expanded on previous knowledge by demonstrating the connection that can exists between daily skill use and mood and functioning in between sessions. In other words, while previous studies reported on benefits to well-being at the end of therapy, this study can speak to daily changes that occur throughout therapy.

As expected, coding of audio recorded answers demonstrated that participants in the CBT condition tended to use more cognitive skills, whereas participants in the SST condition reported using more self-regulation skills. This finding suggests that participants were able to integrate these skills in their daily life. This differentiation between the two therapy conditions is not surprising, but to our knowledge no previous study has inquired about skill use in an open-ended way, allowing participants to describe in their own words the strategies they had used. An interesting expansion of this finding would be to determine the extent to which those self-reported strategies correspond directly to homework assignments or session-related material. In other words, do clients experiment with new strategies on pace with progress in therapy, or do they “latch on” to one strategy early on and stick with that?

Of note, the current results are correlational. Therefore, conclusions cannot be made about whether participants demonstrated improved mood because of skill use or if those participants who were feeling better were more likely to use the skills or more likely to report the use of skills. The issue of causality in homework completion research has been raised in previous studies. For example, Burns and Spangler (2000) examined direction of causality in two large samples of depressed patients undergoing CBT; they concluded that homework compliance has a causal effect on reduction in depression and not vice versa. An additional limitation of this study is the generalizability of the results due to the small sample size. Further, the potential therapeutic effects of experience sampling and diary methods have been noted (e.g., Telford et al. 2012), and frequent questions about mood may enhance awareness of subtle changes. The lack of a control condition limits our ability to determine the extent to which the observed changes in daily functioning are attributable to therapy itself versus the diary survey process. Finally, this study focused on skill use of participants in either CBT or SST. Given that acquiring new skills is central to both modes of therapy, future studies focusing on other styles of therapy may be beneficial in order to identify different patterns.

In conclusion, the use of diary methods in this study allowed us to add to the current literature by not only understanding what changes are occurring in therapy but how those changes may be occurring. This methodology demonstrated what clients actually do in their daily lives to cope with their difficulties and how the use of specific skills learned in therapy are related to well-being. The growing popularity of methods that allow for context-sensitive assessment suggests that these methods will soon become standard components of treatment outcome studies. Thus, it is important to establish both their feasibility and validity in various diagnostic groups.

Footnotes

  1. 1.

    Note that responses from participants often made it rather obvious which condition they were assigned to, so, while the actual assignments were not revealed until the coding was complete, it was not possible to remain completely “blind”.

  2. 2.

    Note that we allowed both the slopes and intercepts to be random due to the number of estimates obtained in the daily sample. The entire model was random.

  3. 3.

    The aggregates were created based upon past experience sampling research and also on the convergent validity of the items. A reliability analysis conducted after the fact reveals that all aggregates had high levels of agreement. Cronbach’s alpha for Positive Affect (Alpha = 0.918), Negative Affect (Alpha = 0.859), negative somatic symptoms (Alpha = 0.757).

Notes

Acknowledgements

This study was funded by NIMH (Award No. MH090414-02).

Compliance with Ethical Standards

Conflict of interest

Ariana C. Hoet, Chris J. Burgin, Kari M. Eddington, and Paul J. Silvia declared that they have no conflict of interests.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Addis, M. E., & Jacobson, N. S. (2000). A closer look at the treatment rationale and homework compliance in cognitive-behavioral therapy for depression. Cognitive Therapy and Research, 24(3), 313–326.CrossRefGoogle Scholar
  2. Affleck, G., Zautra, A., Tennen, H., & Armeli, S. (1999). Multilevel daily process designs for consulting and clinical psychology: A preface for the perplexed. Journal of Consulting and Clinical Psychology, 67, 746–754.CrossRefPubMedGoogle Scholar
  3. Beck, A. T., Emery, G., & Greenberg, R. L. (1985). Anxiety disorders and phobias: A cognitive approach. Basic, New York, b58.Google Scholar
  4. Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893–897. doi:  10.1037/0022-006X.56.6.893.CrossRefPubMedGoogle Scholar
  5. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory–II. San Antonio, TX: Psychological Corporation.Google Scholar
  6. Bolger, N., & Laurenceau, J. P. Intensive longitudinal methods: an introduction to diary and experience sampling research. 2013. New York: Guilford .Google Scholar
  7. Burgin, C. J., Silvia, P. J., Eddington, K. M., & Kwapil, T. R. (2013). Palm or cell? Comparing personal digital assistants and cell phones for experience sampling research. Social Science Computer Review, 31(2), 244–251. doi: 10.1177/0894439312441577.CrossRefGoogle Scholar
  8. Burns, D. D., & Spangler, D. L. (2000). Does psychotherapy homework lead to improvements in depression in cognitive–behavioral therapy or does improvement lead to increased homework compliance? Journal of consulting and clinical psychology, 68(1), 46.CrossRefPubMedGoogle Scholar
  9. Busch, A. M., Uebelacker, L. A., Kalibatseva, Z., & Miller, I. W. (2010). Measuring homework completion in behavioral activation. Behavior modification, 34(4), 310–329.CrossRefPubMedPubMedCentralGoogle Scholar
  10. Conner, T. S., Tennen, H., Fleeson, W., & Barrett, L. F. (2009). Experience sampling methods: A modern idiographic approach to personality research. Social and personality psychology compass, 3(3), 292–313.CrossRefPubMedPubMedCentralGoogle Scholar
  11. Coon, D. W., & Thompson, L. W. (2003). The relationship between homework compliance and treatment outcomes among older adult outpatients with mild-to-moderate depression. The American Journal of Geriatric Psychiatry, 11(1), 53–61.CrossRefPubMedGoogle Scholar
  12. Detweiler, J. B., & Whisman, M. A. (1999). The role of homework assignments in cognitive therapy for depression: Potential methods for enhancing adherence. Clinical Psychology: Science and Practice, 6(3), 267–282.Google Scholar
  13. Dozois, D. J. A., & Covin, R. (2004). The Beck Depression Inventory-II (BDI-II), Beck Hopelessness Scale (BHS), and Beck Scale for Suicide Ideation (BSS). In M. J. Hilsenroth & D. L. Segal (Eds.), Comprehensive handbook of psychological assessment, Vol. 2: Personality assessment (pp. 50–69). Hoboken, NJ: Wiley.Google Scholar
  14. Eddington, K. M., Silvia, P. J., Foxworth, T. E., Hoet, A., & Kwapil, T. R. (2015). Motivational deficits differentially predict improvement in a randomized trial of self-system therapy for depression. Journal of Consulting and Clinical Psychology, 83(3), 602.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Fisher, C., & To, M. (2012). Using experience sampling methodology in organizational behavior. Journal of Organizational Behavior, 33(7), 865–877.CrossRefGoogle Scholar
  16. Fydrich, T., Dowdall, D., & Chambless, D. L. (1992). Reliability and validity of the Beck anxiety inventory. Journal of Anxiety Disorders, 6, 55–61. doi:  10.1016/0887-6185(92)90026-4.CrossRefGoogle Scholar
  17. Gaynor, S. T., Lawrence, P. S., & Nelson-Gray, R. O. (2006). Measuring homework compliance in cognitive-behavioral therapy for adolescent depression review, preliminary findings, and implications for theory and practice. Behavior Modification, 30(5), 647–672.CrossRefPubMedGoogle Scholar
  18. Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life. Sage.Google Scholar
  19. Hox, J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  20. Kazantzis, N., & Deane, F. P. (1999). Psychologists’ use of homework assignments in clinical practice. Professional Psychology: Research and Practice, 30(6), 581.CrossRefGoogle Scholar
  21. Kazantzis, N., Deane, F. P., & Ronan, K. R. (2000). Homework assignments in Cognitive and Behavioral Therapy: A meta-analysis. Clinical Psychology: Science and Practice, 7(2), 189–202.Google Scholar
  22. Kazantzis, N., Deane, F. P., & Ronan, K. R. (2004). Assessing compliance with homework assignments: Review and recommendations for clinical practice. Journal of Clinical Psychology, 60(6), 627–641.CrossRefPubMedGoogle Scholar
  23. Kazantzis, N., & Lampropoulos, G. K. (2002).The use of homework in psychotherapy: An introduction. Journal of Clinical Psychology, 58, 487–488.Google Scholar
  24. Kazantzis, N., Ronan, K. R., & Deane, F. P. (2001). Concluding causation from correlation: Comment on burns and spangler (2000). 1079Google Scholar
  25. Lindenboim, N., Comtois, K. A., & Linehan, M. M. (2007). Skills practice in dialectical behavior therapy for suicidal women meeting criteria for borderline personality disorder. Cognitive and Behavioral Practice, 14, 147–156.CrossRefGoogle Scholar
  26. Luke, D. A. (2004). Multilevel modeling. Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
  27. Muthén, L. K., & Muthén, B. O. (1998–2010). Mplus user’s guide (6th edn.). Los Angeles, CA: Muthén and Muthén.Google Scholar
  28. Rees, C. S., McEvoy, P., & Nathan, P. R. (2005). Relationship between homework completion and outcome in cognitive behaviour therapy. Cognitive Behaviour Therapy, 34(4), 242–247.CrossRefPubMedGoogle Scholar
  29. Strauman, T. J., Vieth, A. Z., Merrill, K. A., Kolden, G. G., Woods, T. E., Klein, M. H., ...Kwapil, L. (2006). Self-system therapy as an intervention for self-regulatory dysfunction in depression: a randomized comparison with cognitive therapy. Journal of Consulting and Clinical Psychology, 74(2), 367.CrossRefPubMedGoogle Scholar
  30. Telesage (2009). SmartQ (Version 5.2.48) [Computer software]. Chapel Hill, NC: Telesage.Google Scholar
  31. Telford, C., McCarthy-Jones, S., Corcoran, R., & Rowse, G. (2012). Experience sampling methodology studies of depression: the state of the art. Psychological Medicine, 42(6), 1119–1129.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Ariana C. Hoet
    • 1
  • Chris J. Burgin
    • 2
  • Kari M. Eddington
    • 1
  • Paul J. Silvia
    • 1
  1. 1.UNCG Department of PsychologyUniversity of North Carolina at GreensboroGreensboroUSA
  2. 2.Tennessee Technological UniversityCookevilleUSA

Personalised recommendations