Overview
-
1.
The purpose of outcome research methodology is to evaluate the extent to which an intervention can be regarded as efficacious, and to evaluate the level of confidence professionals and consumers may place in such judgments. The efficacy reports and practice guidelines that result from the Panel activities will be based upon clinical expertise and comprehensive and systematic analysis of research data, which appears in peer-reviewed literature.
-
2.
The ability to meaningfully assess outcome (efficacy) studies assumes a basic knowledge of clinical experimental design and analysis methods as well as ethical standards for human research.
-
3.
The ability to utilize practice guidelines assumes a fundamental level of training and education, possession of a foundation of relevant knowledge, and effective clinical assessment skills that enable the clinical practitioner to set priorities and make sound decisions regarding treatment method and focus. It is recognized that no set of guidelines can be regarded as absolute criteria. When practice guidelines are developed, it will be recognized that at certain times clinical imperatives may require a modification of the “best practice” guidelines justified by a compelling professional rationale. The judgment of an experienced clinician is important, especially when research data are limited.
-
4.
Technological and clinical advances sometimes occur at a rapid rate, so it will be necessary to review the guidelines on a regular basis so they remain current and relevant. Reviews will be updated at least every 3 years.
-
5.
This document itself will be reviewed on a regular (annual) basis to determine whether it provides adequate and effective guidance in light of changes that may occur regarding treatment standards, diagnostic standards, ethical standards, or research standards.
Scientific Considerations
Clinical psychophysiology uses variables that are quantifiable. The diagnostic criteria, independent variables, dependent variables, and measurable intervening variables will be included in the Panel deliberations. In methodological terms, the independent variables are specific and subject to experimental manipulation (e.g., sensor placement, bandwidths and frequencies, reinforcement contingencies). The dependent variables (e.g., physiological event being measured, response to treatment) can also be clearly specified. Often, intervening variables (e.g., change in brainwave features, improved motor unit recruitment) can be quantified.
-
1.
The “condition of interest” (COI) will be clearly identified and operationally defined. Most often, the COI will be a diagnostic entity recognized in either the most current Diagnostic and Statistical Manual (DSM) published by the American Psychiatric Association, or in the most current International Classification of Diseases Handbook (ICD) published by the World Health Organization. If the COI is not a recognized diagnostic entity but addresses symptomatic, cognitive, or behavioral conditions (e.g., cognitive brightening, optimal functioning), the condition being treated will be operationally defined in a manner that permits objective assessment and replication.
-
2.
Guidelines developed by the Panels will reflect the technical features and parameters of the particular treatment modality under review.
-
3.
Panels will evaluate the extent to which reported interventions are amenable to empirical analysis and replication by independent researchers.
-
4.
If particular intervening variables (e.g., change in motor unit recruitment, shift in brainwave frequencies) have been hypothesized to be relevant to the clinical outcome, the Panel will evaluate the available empirical evidence for these mechanisms.
-
5.
Relevant variables such as age, gender, comorbidity, and treatment history will be identified.
-
6.
Outcome measures will be meaningfully related to the diagnostic criteria or COI operational definition.
Specificity
While the primary function of the Panels is to examine evidence for efficacy, the Panels will also examine whether treatment specificity has been demonstrated.
-
1.
Necessary or sufficient treatment components: The primary function of the various Panels is that of reviewing and reporting on evidence for treatment efficacy. Additionally, each Panel will report whether the study design permitted a determination as to whether a particular treatment component is necessary to the treatment outcome, rather than just sufficient to accomplish the treatment outcome. As an example, it may be questioned whether the manipulation of a particular brainwave frequency at a particular electrode location is necessary for successful treatment outcome. Would the manipulation of a different brainwave frequency achieve the same result? If not, then it would appear that manipulation of the particular frequency is a necessary component of the treatment protocol. If a different frequency serves as well, the variable may be sufficient to produce change, but other variable manipulations may facilitate symptomatic improvement as well. In that case, the presence of a particular variable in the clinical protocol may be sufficient to produce a desired result, but it is not a necessary component. If that question is not tested in the design, it cannot be assumed that the particular treatment variable is necessary to the treatment outcome.
A particular aspect of the intervention (independent variable) may be nonspecific as a component of the treatment protocol, but, at the same time, some element or characteristic of the class to which that independent variable belongs may be necessary to the desired outcome. For instance, a manipulation of a particular brainwave frequency may not be necessary to the desired outcome, but the manipulation of at least some brainwave frequency may be necessary to the desired outcome.
-
2.
The Panel will determine whether “Placebo” nonspecificity can be ruled out as the dominant effect. A more general concern about specificity is whether there is simply some general characteristic of the treatment protocol that is responsible for the desired change. In that case, none of the particular treatment variables are really necessary for symptomatic improvement. Rather, simply participating in the study and receiving empathy and encouragement from the therapist may be sufficient to produce symptomatic improvement. It is known that in some areas of applied psychophysiology, very different protocols are reported to produce similar beneficial results. Thus, the protocols cannot be differentiated on the basis of outcome. Similar problems are encountered in other therapies.2
Clinical Utility
Panels will recognize the distinction between “efficacy” and “clinical utility” and incorporate these distinctions into their deliberations and writings.
-
1.
Efficacy (having the desired effect) refers to the determination of treatment effect derived from a systematic evaluation obtained in a controlled clinical trial.
-
2.
Clinical utility refers to the practical value of an intervention, or the extent to which it is practical and possible to translate the findings from efficacy studies into normal clinical practice. As an example, an intervention may be found to be efficacious, but meet such consumer resistance, require such expensive equipment or extensive training to be implemented, or be so cost prohibitive that clinical utility is compromised on a practical level. Additionally, a particular intervention may be found to be efficacious in a highly controlled study in which other factors (comorbidity, treatment frequency, and so forth) are effectively controlled. The same protocol may be ineffective in the “real world” of clinical practice when those variables are uncontrolled.