Advertisement

Cognitive Therapy and Research

, Volume 42, Issue 2, pp 204–211 | Cite as

Methods of Delivering Progress Feedback to Optimise Patient Outcomes: The Value of Expected Treatment Trajectories

  • Geoffrey R. Hooke
  • Adelln A. H. Sng
  • Nadia K. Cunningham
  • Andrew C. Page
Original Article

Abstract

Whilst feedback is demonstrated to improve therapy outcomes, little attention has been given to the relative benefits of the form in which feedback is given. The present study aimed to compare patients’ perceptions of feedback graphs with and without expected treatment response trajectories. In a counter-balanced design, patients in 2-week CBT programs were shown feedback graphs with and without expected symptom trajectories; and were asked to complete questionnaire regarding the appeal after viewing the first graphs. Patients (n = 42) viewed feedback graphs and preferred those with trajectories present and perceived the additional detail helpful to both themselves and their therapists. The present findings support the appeal for and potential usefulness of richer feedback for facilitating discussion and positive outcomes in therapy.

Keywords

Patient Routine Outcomes Measurement Outcomes monitoring Outcome management Feedback, cognitive behavior therapy 

Introduction

The ability to measure pre to post-treatment change is essential to accountable psychological practice (Page and Stritzke 2014). However, monitoring client progress between pre- and post-treatment confers benefit to an understanding of therapeutic process by being able to respond in real-time to deviations from an expected course of recovery (Dyer et al. 2016; Lambert et al. 2001; Newnham et al. 2010b). The development of a patient-focussed research paradigm (Delgadillo et al. 2016; Howard et al. 1996; Newnham et al. 2007) provided a basis for continuous evaluation of progress during the course of treatment (Restifo et al. 2015). Using the knowledge that expected treatment outcomes follow a negatively accelerating trajectory, it became possible to identify patients who were “not on track” to achieve a good outcome (Lambert et al. 2005), to predict potential adverse clinical events (Kashyap et al. 2015), and manage treatment (Page et al. 2016).

By developing criteria for an alarm, when a patient’s progress deviates from an expected pathway to recovery, therapists, and patients could be alerted to reflect upon progress to date and change future plans if required. Objective feedback is advantageous because therapists are not good predictors of deterioration (Bar-Kalifa et al. 2016; Hannan et al. 2005) and do not always identify deviations as reasons for changing the course of therapy (Schulte and Eifert 2002). Therapy outcomes are improved when therapists (and often patients) are provided with feedback about patients who are not on track for a good outcome (Dyer et al. 2014; Newnham et al. 2010b; Shimokawa et al. 2010). The use of computerised methods such as touchscreen technology has further streamlined and automated large scale patient monitoring (Newnham et al. 2012). Such technologies also make it easier to adapt and test different methods of delivery, which allows for further refinement of feedback and monitoring.

Interestingly a recent meta-analysis by Kendrick et al. 2016 appeared to call into question the effectiveness of patient reported outcome measures (i.e., feedback) in improving treatment outcomes, but arguably its findings are more supportive than at first glance. The negative conclusion seems to arise because the authors implicitly assumed that the effects of feedback would be equally evident among all patients. However, the logic of feedback is that analogous to providing information to a pilot about the plane’s trajectory on a flight path, only information about a deviation will cause a change in behaviour. Hence, the provision of feedback that a person is on track to a positive outcome will not change therapeutic practice or outcomes and probably this difference potentially explains the apparent difference between the conclusions of the two literature reviews (Kendrick et al. 2016; Shimokawa et al. 2010). To the extent that the effectiveness of successful feedback is dependent on providing feedback to Not On Track patients, this highlights the importance of information that includes detail about expected improvement.

Hence, what needs further exploration is the relative benefits of the format of the feedback to patients and therapists. Feedback needs to be acceptable to patients to be effective, and certain forms of feedback to patients may well be associated with better outcomes. Several different feedback delivery systems have been developed. Initial approaches to monitoring during treatment focused on providing feedback to off track cases (Lambert et al. 2001; Lutz 2003; Rubel et al. 2015), and involved a “traffic light” style of information delivery (i.e., a coloured alert is provided indicating whether the patient is on or off track with an accompanying verbal description). Over time more complex feedback delivery systems have been developed (Bickman et al. 2011). One style of feedback delivery system incorporates graphical, highlighted areas in addition to a “traffic light” presentation to inform the patient and therapist of progress and areas of concern. For instance, the feedback using the OQ-Analyst package (that accompanies the 45-item Outcome Questionnaire; OQ45; Lambert and Finch 1999) is based on calculations involved in estimating clinical significance (Jacobson and Truax 1991) (Ronk et al. 2016, 2013). The threshold for movement from the “unhealthy” into the “healthy” range is identified and the session-to-session change can be reliably and validly compared against the pre-treatment score (Ronk et al. 2012, 2016). The OQ-Analyst graphical feedback indicates the patient is on track, off-track (unchanged or deteriorated), or recovered (including a “dramatic” response). A line (in a therapist report) indicates an expected treatment response relative to actual progress and there is a written message interpreting progress and predicting outcome (although, for proprietary reasons, the expected trajectories for the alerts are not communicated). Other groups have taken somewhat different approaches to the delivery of feedback. For example, Lutz et al. described a system where expected trajectories are developed based on algorithms that match a given patient to the most similar patients seen within the service (Lutz et al. 2005), and therapists are more formally shown an expected pathway through treatment and the patient’s actual progress relative to the expectation.

Another system, developed at Perth Clinic (acute private psychiatric facility) in Australia, uses an initial level of severity to identify expected trajectory of improvement (like the OQ Analyst system), but also includes graphical outputs (like the German system) which plot the patient’s actual progress against a band representing the expected course of improvement in both therapist and patient feedback. Given evidence that early treatment response is indicative of final outcome (Howard et al. 1996), Newnham et al. (2010b) developed an expected treatment response trajectory that was initially anchored by the pre-treatment score. The data yielded up to five different trajectories. Straying from this trajectory to worsening scores indicated an “off track” condition. The trajectory depicts upper and lower bounds around a mean level of improvement, so that reliable deviations from improvement can be distinguished from variability arising from measurement error. Thus, while existing feedback processes shame much in common, they vary on the degree to which the expected trajectories are emphasized in the feedback to patients.

In terms of outcomes, it is apparent that there is evidence for the effectiveness of feedback systems which use the expected trajectories of improvement and those that do not (Newnham and Page 2010). That is, not on track patients have better outcomes when feedback about progress is provided to therapists and patients. However, there may be subtle differences in outcomes associated with delivery methods because of their potential to feed into the therapeutic process. For example, provision of trajectories may confer additional benefit to patients compared to feedback delivery methods lacking this information. The use of technology in monitoring and delivering feedback provided an opportunity to examine the acceptability among patients for a feedback system that included either a trajectory or did not. Since the aim of providing progress feedback to both therapists and patients is to foster a conversation about recovery, the present study set out to examine whether the trajectories offered any further advantage to patients in viewing their graphs. Thus, the aim of the current study was to compare patients’ perceptions of the feedback graphs with and without expected treatment response trajectories.

Method

Participants

The sample comprised 42 patients across six consecutive cognitive-behaviour therapy (CBT) groups conducted at a private psychiatric hospital in Western Australia. Patients’ ages ranged from 18 to 69 years (M = 40, SD = 14), and 69% of the sample were female. Each patient was diagnosed by their treating psychiatrist according to ICD-10-AM criteria (National Centre for Classification in Health Publications, 2002), and primary diagnoses consisted mostly of mood (52%), anxiety (40%), substance use (4%), and other (4%) disorders. The current investigation is part of an ongoing quality assurance initiative conducted by the hospital; patients provided informed consent as part of routine admission procedures.

Materials

Outcome Monitoring Measures

The WHO-5 Wellbeing Index (Bech et al. 1996) was used to measure changes in wellbeing. It has high internal consistency and strong validity in this setting (Newnham et al. 2010a). The modified version measures wellbeing over the previous 24 h. High scores indicate more positive rating of wellbeing.

The Five Item Daily Symptom Index (DI-5; Dyer et al. 2014) was used to measure change in patients’ psychological distress. The DI-5 is a self-report measure comprising five items designed to assess patient’s affective psychological distress, including depression, anxiety, worthlessness, not coping, and suicidal ideation. The DI-5 consists of five items rated using a six point Likert-type scale measuring frequency (‘at no time’, ‘some of the time’, ‘less than half the time’, ‘more than half the time’, ‘most of the time’, ‘all of the time’) scored from 0 (‘at no time’) to 5 (‘all of the time’).. Patients endorse the appropriate option for the previous 24 h. The DI-5 has demonstrated high internal consistency and test–retest reliability in clinical samples (Dyer et al. 2014); and sound construct validity, exhibiting high correlations with other mental health measures in clinical samples.

Feedback Graphs

Patients received an individualized feedback graph that showed their symptom and wellbeing scores on two separate charts (Fig. 1). The patient’s wellbeing scores (thick lines in top chart in Fig. 1) were mapped against two thinner lines representing the upper and lower boundaries of an expected treatment response curve (Newnham et al. 2010b). To generate these curves, a cohort of previous patients undertaking the program was divided into five equal-sized groups based on their wellbeing scores on Day 1. The best-fitting functions (i.e., log-linear curves) were then plotted for each of the five levels of severity. For each function, upper and lower boundaries of half a standard deviation either side of the mean score of each measurement point were then calculated to produce the final trajectories of expected treatment response. A similar procedure was used to develop the symptom trajectories (bottom charts in Fig. 1), but only four groups were necessary to model expected treatment responses across the different levels of severity (Dyer et al. 2014).

Fig. 1

Feedback graph displaying a (hypothetical) patient’s wellbeing (top) and symptom (bottom) scores, with and without overlaid expected treatment response trajectories

The distinction between on-track and not-on-track progress was subsequently made based on comparing the patient’s actual scores to the expected trajectories (Dyer et al. 2014; Newnham et al. 2010b). Detailed investigation of the impact of patients’ progress status on their perceptions of the feedback graphs was beyond the scope of this exploratory study. Given that the aim of the current study was to compare patients’ perceptions of the feedback graphs with and without expected treatment response trajectories, a no-trajectories version of the feedback graph was also produced which only depicted the patient’s scores (Fig. 1).

Feedback Trajectories Questionnaire

A questionnaire was developed for the current study to survey patients’ responses to the feedback graphs with versus without trajectories. The questionnaire was divided into two sections, each administered at different time points. The first section was given to patients after they viewed the first feedback graph, in the first 5 days of a 10 day closed group. This section comprised nine items assessing patient’s perceptions of the feedback graph (e.g. “The feedback graph helped me to monitor my progress in therapy”), and seven items assessing perceptions of the ensuing feedback discussion (e.g., “The feedback session was useful in identifying what I can do to get well”). These items were rated on a seven-point Likert scale (1–7) from “Strongly Disagree” to “Strongly Agree”. The second section of the questionnaire was given to patients after they viewed the second feedback graph. Patients were first asked to indicate which graph they preferred on a five-point scale (with or without trajectories), then answer four items which compared the feedback graphs (e.g. “Which graph did you prefer for comparing how you are doing now to how you were doing before?”).

Procedure

Monitoring and Feedback Design: Patients completed the manualised CBT group program over 10 consecutive working days. The groups comprised 6–8 members, led by two co-therapists who were either clinical psychologists or occupational therapists. Every morning, patients were invited to complete two monitoring measures (five items measuring positive wellbeing and five items measuring symptoms) via touch-screens located in the therapy rooms. Patients who missed completing the measures that day were provided with a reminder, but participation was entirely voluntary.

Patients then received feedback graphs on Day 5 (mid-way) and Day 10 (end) of the program. Therapists were provided with some guidelines on delivering feedback, but no formal structure was defined. Generally, the therapist explained what the separate graphs indicated, then provided patients with the opportunity to discuss their progress within the group. Common prompts used by therapists included encouraging patients to identify patterns in their scores, and how the changes may have been associated with different events or actions over the past week. Patients may also have been encouraged to discuss areas for potential improvement and generate ideas with their therapist (e.g. assertiveness training; scheduling pleasant activities; homework tasks). A description of the monitoring and feedback procedures undertaken at the hospital can be found in Newnham et al. (2012).

Presentation of feedback graphs: All patients received both versions of the feedback graphs (i.e., with trajectories and without trajectories). Order of presentation of the graphs was counter-balanced, such that for three CBT groups (n = 19 patients), patients viewed the feedback graphs with trajectories first, followed by the graphs without trajectories. The remaining three groups viewed the feedback graphs in reverse order (n = 23 patients). Four therapists collected the data. Therapists first introduced the rationale for monitoring and feedback, and then handed out the first feedback graph. The therapist explained the elements depicted in the feedback graph, and patients had the opportunity to ask questions and discuss their feedback within the group. Patients then completed the first section of the Feedback Trajectories Questionnaire, before they received the second graph. The therapist again explained the various elements depicted in the second graph, and patients were given some time to ask additional questions and discuss the graph, before they complete the second section of the questionnaire.

Results

In general, patients preferred the graphs with trajectories. That is, only 10% of patients preferred the graphs without trajectories (and an additional 12% had no preference). Mean scores on the Feedback Trajectories Questionnaire were compared for patients who viewed trajectories first compared to those who did not view trajectories first (see Table 1). Patients who viewed trajectories first had higher ratings when asked if the feedback graphs helped them monitor their progress in therapy and they commented that they thought it would help their therapists monitor progress in therapy. Interestingly, the effect sizes suggest that the perceived benefits of the trajectories were seen to be greater for the patients themselves (Cohen’s d = 0.75) than the for their therapists (d = 0.63). Despite seeing greater value for themselves, patients indicated that the graphs with the trajectories provided them with many more opportunities for discussing progress with therapist (d = 1.5). Patients who viewed graphs with trajectories also had higher ratings when asked whether feedback graphs were useful for comparing current progress to how they were previously (see Table 2).

Table 1

Mean (SD in brackets) agreement ratings (1–7) relating to questions about the feedback graphs for patients who had graphs with or without expected trajectories of improvement

 

No trajectories (n = 23)

Trajectories (n = 19)

The feedback graph helped me to monitor my progress in therapy

5.5 (1.1)

6.1 (0.5)*

The feedback graph helped my therapists to monitor my progress in therapy

5.4 (1.0)

6.0 (0.9)*

I feel encouraged after viewing my feedback graph

4.9 (1.4)

5.3 (1.7)

I feel discouraged after viewing my feedback graph

2.7 (1.3)

3.3 (1.6)

I look forward to viewing my next feedback graph

5.6 (1.6)

5.9 (1.1)

The feedback graph was helpful in comparing how I am doing now to how I was doing before

4.9 (1.4)

6.0 (0.5)*

The feedback graph was helpful for comparing how I am doing now to how other patients are doing

3.9 (1.7)

4.2 (1.9)

The feedback graph gave me more opportunities to discuss how I am doing with my therapist

4.5 (1.2)

6.0 (0.8)*

Overall, I am satisfied with the feedback graph that I received

5.4 (1.0)

5.6 (1.4)

*p < .05

Table 2

Mean (SD in brackets) agreement ratings (1–7) relating to questions about the impact on group discussion about feedback graphs for patients who had graphs with or without expected trajectories of improvement

 

No trajectories (n = 23)

Trajectories (n = 19)

I felt comfortable discussing my feedback graph within the group

4.9 (1.5)

5.9 (0.8)*

I would prefer more information about how to interpret my feedback graph

3.6 (1.8)

3.6 (1.7)

I would prefer to receive my feedback graph more regularly than once a week

2.9 (1.5)

3.2 (1.8)

The discussion of my feedback graph helped me feel better about myself

4.6 (1.3)

5.1 (1.2)

I think that receiving the feedback graph and discussing it with my therapist can help to improve my symptoms and wellbeing

5.0 (1.6)

5.8 (0.9)

The feedback session was useful in identifying what I can do to get well

4.5 (1.7)

5.3 (1.1)

Overall, I am satisfied with the group feedback sessions

5.4 (1.3)

5.8 (1.3)

*p < .05

The other items were not rated differently. That is, patients found that the trajectories and no-trajectories graphs had equal reactions of encouragement and discouragement; were equally looking forward to receiving their graph; did not differ in preference for more guidelines or more frequent feedback. Both groups found the discussion helpful in identifying what they can do, how they could improve their symptoms and aid with goal setting and both groups were satisfied with the feedback graph and discussion overall. It is also noteworthy that the responses to the question about the benefits of the feedback graph for comparing how the patient was doing to how other patients are doing was at the mid-point of the scale (i.e., neither agree nor disagree), suggesting that the patients were not claiming to use the feedback graphs in a comparative manner within the group.

In terms of questions about how the trajectories might impact upon group discussion, there were mostly no differences between those who viewed graphs with trajectories and those without. Where there was a difference, was in response to the questions about feeling comfortable discussing the graph within a group treatment context. Both groups indicated high levels of comfort, but those who had trajectories felt more comfortable with the discussion.

Discussion

The current study aimed to compare patients’ perceptions of feedback graphs with and without expected treatment response trajectories. There was a clear preference for the trajectories being present on the graphs. Overall patients appeared to believe that the additional detail of the trajectories was helpful to both themselves and to their therapists. This may point to the perceived value of feedback to the patients. That is, they may see the value of progress information as not only providing relevant details about the progress from the start of treatment, but also by providing them with a normative comparison. This value placed on the normative information stands in contrast to the observation that the patients seemed to place no value on the information relative to the other group members. Rather, the value seemed to relate to the trajectory that related to their starting point.

Overall, patients found the trajectory graphs of more benefit in facilitating discussions with their therapists, and this suggests a value for cognitive therapists. Behavioural experiments are a valuable tool in cognitive therapy, as therapists encourage the testing of predictions against actual outcomes. The use of feedback graphs are a potentially valuable tool when viewing engagement in therapy as a behavioural experiment. Patients appeared to find the trajectories valuable for facilitating therapeutic discussions and this implies that there is an opportunity to use the trajectories to facilitate communication between therapist and patient about how treatment is unfolding relative to that predicted.

Considering the absolute values of the responses, it appears that both types of feedback were considered useful. For instance, the responses to the item, “The feedback graph gave me more opportunities to discuss how I am doing with my therapist” (6.0 and 4.5 for graphs with and without trajectories, respectively) were both above the mid-point indicating that respondents agreed the graphs were useful. However, the rating given to graphs without the trajectories was only marginally above the neutral mid-point. In contrast, the graphs with the trajectories were rated highly and were more likely to reflect strong support for the ability of the information that enriched the opportunities for therapeutic conversations. This difference raises the question, what was it about the graphs that participants believed to be of potential benefit? The inclusion of trajectories could provide a foundation for a richer conversation as the topic can move beyond the individual’s change relative to their starting point and can consider normatively-based expectations. Although therapists may have a set of implicit expectations of (both typical and atypical) progress, the patient will have less experience upon which to base their predictions. Without an expected trajectory, the patient may not have a clear idea of what benefits are expected from treatment and may not know what change is “normal.” From the perspective of cognitive therapy, it has long been advised that therapists use progress graphs in therapy (e.g. depressed patients testing hopelessness beliefs by comparing mood on days with greater and less physical activity). The addition of expected trajectories has the potential to have a comparative discussion about the extent of any change considering that of most other patients. The trajectories can highlight when the pattern is atypical and unexpected, thereby prompting a conversation about why the change is greater or less than expected. The outcomes can be included in reflections on behavioural experiments, as a client who has been diligent in completing homework tasks may find that their improvement is greater than “expected.”

Another intriguing pattern in the individual items is the response to the statement, “The feedback graph was helpful in comparing how I am doing now to how I was doing before.” The participants preferred the graphs with the expected trajectories when they were reflecting upon their change relative to their own performance. The surprising group difference suggests that the patients found the normative data informative when making these otherwise ideographic comparisons. Thus, it seems that when the additional information is available, patients are not only considering the direction and magnitude of any symptom change, but they are also interpreting these attributes with respect to normative values. From the perspective of cognitive therapy, therapists might draw upon both the individual and the group reference points. However, there may be potential risks, such as if a client has improved relative to their own starting point, but not changed as much as expected from the group comparison. In these circumstances a therapist may need to discuss possible disappointment, but it provides an opportunity to talk about the active steps the patient has taken to bring about the positive change and how these can be steps can be continued and developed.

Patients also found the process of receiving the graphs and the discussion of progress and goals quite positive, and were overall quite satisfied with the frequency of receiving feedback. Thus, this information can allay the potential anxiety among therapists that the provision of graphical feedback is going to be harmful to the treatment process or damaging to the therapeutic relationship. Whilst the trajectories graph offered more detail, patients clearly preferred this graph and benefitted from the information. This also supports a positive reaction to how the graphs were presented in the sessions by the therapists.

Thus, there do seem to be advantages in informing the patient, not just the therapist, of progress. Patients appreciate being involved in the discussions and value information that highlights changes, suggests direction, and potentially serves as a motivator. The very nature of regularly providing this feedback encourages the therapist to have the discussion. For therapists, the use of objective feedback and the sharing of this information as a basis for a collaborative conversation about progress, can ensure less reliance on therapists’ own judgements and foster a shared understanding of the patient’s present state, their goals, and progress towards achieving these aims.

What is not apparent from the current study is whether these results would generalise to all psychotherapies and client groups. That is, the patients in the present study were selected to be involved in a CBT program that mainly focussed on emotional symptoms. There is a clear expectation among patients, and shared by therapists, that the expected trajectory is one of improvement. Patients arrive at treatment unwell, learn cognitive and behavioural skills that they refine and practice, with the aim of completing the program better equipped to manage their symptoms. Not all patients present for treatment with this expectation and not all psychopathologies can be approached with a similar confidence about short-term improvement. With some patients and disorders, a therapeutic goal may be containment and the prevention of a symptomatic deterioration. Hence, the expected trajectory may be one of stability, rather than improvement. Likewise, in an inpatient context, a hospital admission will often be a function of the responsiveness to treatment, and so the trajectory of improvement by a specific day of treatment may be less clear (Page et al. 2005). Thus, the extent to which the current results extend to these situations remains an open question.

Another limitation is that it is not clear the extent to which the present results are attributable to preferences among the therapists. That is, it is possible that patients may have been neutral (or even negative) about the expected trajectories, but influenced by their therapists (who preferred one format of the graphs). Since we did not measure or control for therapist preferences it is not possible to exclude this explanation of the present findings. Future research could examine the degree to which therapist preference influences the patient ratings, but the value of the present study is that it investigated the patients’ immediate responses to graphical feedback in the context of their own treatment.

In conclusion, one potential benefit of the inclusion of technology in cognitive-behaviour therapies is the timely and detailed normative feedback about progress. From the patients’ perspective the use of detailed feedback that clearly articulates the expected trajectory of improvement is welcomed. Future research can investigate the degree to which this patient preference is associated with improved treatment outcomes and enhanced motivation for and engagement with therapy.

Notes

Acknowledgements

The researchers acknowledge the support of Moira Munro.

Funding

The research was funded in part by the Australian Research Council grant LP150100503, received by Andrew C. Page and Geoffrey R. Hooke.

Compliance with Ethical Standards

Conflict of Interest

Geoffrey R. Hooke, Adelln A. H. Sng, Nadia K. Cunningham and Andrew C. Page declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Bar-Kalifa, E., Atzil-Slonim, D., Rafaeli, E., Peri, T., Rubel, J., & Lutz, W. (2016). Therapist-client agreement in assessments of clients’ functioning. Journal of Consulting and Clinical Psychology, 84(12), 1127–1134. doi: 10.1037/ccp0000157.CrossRefPubMedGoogle Scholar
  2. Bech, P., Gudex, C., & Johansen, K. S. (1996). The WHO (Ten) Well-Being Index: Validation in diabetes. Psychotherapy and Psychosomatics, 65, 183–190. doi: 10.1159/000289073.CrossRefPubMedGoogle Scholar
  3. Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429. doi: 10.1176/appi.ps.002052011.CrossRefPubMedGoogle Scholar
  4. Delgadillo, J., Moreea, O., & Lutz, W. (2016). Different people respond differently to therapy: A demonstration using patient profiling and risk stratification. Behaviour Research and Therapy, 79, 15–22. doi: 10.1016/j.brat.2016.02.003.CrossRefPubMedGoogle Scholar
  5. Dyer, K., Hooke, G. R., & Page, A. C. (2014). Development and psychometrics of the five item daily index in a psychiatric sample. Journal of Affective Disorders, 152–154, 409–415. doi: 10.1016/j.jad.2013.10.003.CrossRefPubMedGoogle Scholar
  6. Dyer, K., Hooke, G. R., & Page, A. C. (2016). Effects of providing domain specific progress monitoring and feedback to therapists and patients on outcome. Psychotherapy Research, 26(3), 297–306. doi: 10.1080/10503307.2014.983207.CrossRefPubMedGoogle Scholar
  7. Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A Lab Test and Algorithms for Identifying Clients at Risk for Treatment Failure. Journal of Clinical Psychology, 61(2), 155–163.CrossRefPubMedGoogle Scholar
  8. Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy efficacy, effectiveness, and patient progress. American Psychologist, 51, 1059–1064. doi: 10.1037//0003-066X.51.10.1059.CrossRefPubMedGoogle Scholar
  9. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19.CrossRefPubMedGoogle Scholar
  10. Kashyap, S., Hooke, G. R., & Page, A. C. (2015). Identifying risk of deliberate self-harm through longitudinal monitoring of psychological distress in an inpatient psychiatric population. BMC Psychiatry, 15. doi: 10.1186/s12888-015-0464-3.PubMedPubMedCentralGoogle Scholar
  11. Kendrick, T., El-Gohary, M., Stuart, B., Gilbody, S., Churchill, R., Aiken, L., ... Moore, M (2016). Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. Cochrane Database of Systematic Reviews, 7, CD011119.PubMedGoogle Scholar
  12. Lambert, M. J., & Finch, A. E. (1999). The outcome questionnaire. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcomes assessment (2nd edn., pp. 831–869). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.Google Scholar
  13. Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159–172.CrossRefPubMedGoogle Scholar
  14. Lambert, M. J., Harmon, C., Slade, K. L., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174. doi: 10.1002/jclp.20113.CrossRefPubMedGoogle Scholar
  15. Lutz, W. (2003). Efficacy, effectiveness, and expected treatment response in psychotherapy. Journal of Clinical Psychology, 59(7), 745–750.CrossRefPubMedGoogle Scholar
  16. Lutz, W., Leach, C., Barkham, M., Lucock, M., Stiles, W. B., Evans, C., ... Iveson, S (2005). Predicting change for individual psychotherapy clients on the basis of their nearest neighbors. Journal of Consulting and Clinical Psychology, 73(5), 904–913.CrossRefPubMedGoogle Scholar
  17. Newnham, E. A., Doyle, E. L., Sng, A. A., Hooke, G. R., & Page, A. C. (2012). Improving clinical outcomes in psychiatric care with touch-screen technology. Psychological Services, 9(2), 221–223. doi: 10.1037/a0026143.CrossRefPubMedGoogle Scholar
  18. Newnham, E. A., Harwood, K. E., & Page, A. C. (2007). Evaluating the clinical significance of responses by psychiatric inpatients to the mental health subscales of the SF-36. Journal of Affective Disorders, 98(1–2), 91–97. doi: 10.1016/j.jad.2006.07.001.CrossRefPubMedGoogle Scholar
  19. Newnham, E. A., Hooke, G. R., & Page, A. C. (2010a). Monitoring treatment response and outcomes using the World Health Organization’s Wellbeing Index in psychiatric care. Journal of Affective Disorders, 122(1–2), 133–138. doi: 10.1016/j.jad.2009.06.005.CrossRefPubMedGoogle Scholar
  20. Newnham, E. A., Hooke, G. R., & Page, A. C. (2010b). Progress monitoring and feedback in psychiatric care reduces depressive symptoms. Journal of Affective Disorders, 127(1–3), 139–146. doi: 10.1016/j.jad.2010.05.003.CrossRefPubMedGoogle Scholar
  21. Newnham, E. A., & Page, A. C. (2010). Bridging the gap between best evidence and best practice in mental health. Clinical Psychology Review, 30(1), 127–142. doi: 10.1016/j.cpr.2009.10.004.CrossRefPubMedGoogle Scholar
  22. Page, A. C., Cunningham, N. K., & Hooke, G. R. (2016). Using daily monitoring of psychiatric symptoms to evaluate hospital length of stay. British Journal of Psychiatry Open, 2(6), 341–345.CrossRefPubMedPubMedCentralGoogle Scholar
  23. Page, A. C., Hooke, G. R., & Rampono, J. (2005). A methodology for timing reviews of inpatient hospital stay. Australian and New Zealand Journal of Psychiatry, 39(3), 198–201.CrossRefPubMedGoogle Scholar
  24. Page, A. C., & Stritzke, W. G. K. (2014). Clinical psychology for trainees: Foundations of science-informed practice (2nd edn.). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  25. Restifo, E., Kashyap, S., Hooke, G. R., & Page, A. C. (2015). Daily monitoring of temporal trajectories of suicidal ideation predict self-injury: a novel application of patient progress monitoring. Psychother Research, doi: 10.1080/10503307.2015.1006707.Google Scholar
  26. Ronk, F. R., Hooke, G. R., & Page, A. C. (2012). How consistent are clinical significance classifications when calculation methods and outcome measures differ? Clinical Psychology: Science and Practice, 19(2), 167–179. doi: 10.1111/j.1468-2850.2012.01281.x.Google Scholar
  27. Ronk, F. R., Hooke, G. R., & Page, A. C. (2016). Validity of clinically significant change classifications yielded by Jacobson-Truax and Hageman-Arrindell methods. BMC Psychiatry, 16(1), 1–9. doi: 10.1186/s12888-016-0895-5.CrossRefGoogle Scholar
  28. Ronk, F. R., Korman, J. R., Hooke, G. R., & Page, A. C. (2013). Assessing clinical significance of treatment outcomes using the DASS-21. Psychological Assessment, 25(4), 1103–1110. doi: 10.1037/a0033100.CrossRefPubMedGoogle Scholar
  29. Rubel, J., Lutz, W., Kopta, S. M., Kock, K., Minami, T., Zimmermann, D., & Saunders, S. M. (2015). Defining early positive response to psychotherapy: An empirical comparison between clinically significant change criteria and growth mixture modeling. Psychological Assessment, 27(2), 478–488. doi: 10.1037/pas0000060.CrossRefPubMedGoogle Scholar
  30. Schulte, D., & Eifert, G. H. (2002). What to do when manuals fail? The dual model of psychotherapy. Clinical Psychology: Science and Practice, 9(3), 312–328.Google Scholar
  31. Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298–311. doi: 10.1037/a0019247.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Perth ClinicWest PerthAustralia
  2. 2.School of Psychological ScienceThe University of Western AustraliaCrawleyAustralia

Personalised recommendations